Xorro P2P How we built a BitTorrent-like P2P network from scratch
Xorro P2P is a BitTorrent-like peer-to-peer file sharing network built by software engineers Ken Chen, David Kurutz, and Terry Lee.
Interested in P2P networks and distributed systems, we set out to build our own from scratch. The version available today is our alpha release, providing a working proof-of-concept. We want to continue improving upon Xorro — upcoming features are listed further down this page.
This site outlines our journey building Xorro P2P. We hope that readers leave with a better understanding of the P2P problem space.
As strictly end users of P2P systems, we started our journey facing a very large learning curve. Much research was needed for us to understand the history and internals of these systems. We collectively read up on P2P networks, old and new: Napster, Gnutella, Freenet, BitTorrent, IPFS...
Centralized vs Decentralized
An important concept to understand is the difference between centralized and decentralized systems. A comparison of first and second generation networks helps to illustrate the difference.
Napster was a P2P file sharing service that was mostly used to transfer music files, and was popular from 1999-2001, It’s estimated that there were about 80 million registered users at its peak. Napster worked by having all nodes connect to a central index server that contained all the information about who was in possession of which files.
Due to its centralized nature, Napster was vulnerable to attacks and lawsuits and Napster was shut down by court order after about 2 years of service. In addition to those vulnerabilities, the central index meant there was a single point of failure, as well as a lack of scalability.
BitTorrent, Gnutella, Freenet: Decentralized
The next generation of P2P networks were able to escape the same fate as Napster by moving to a distributed model.
In a decentralized system like BitTorrent, every computer/node acts as a client and server, maintaining its own segment of a file lookup index. Nodes can find out about the locations of files through other nodes, removing the reliance on a central server.
Focusing on P2P File Sharing Systems
Knowing the advantages of newer P2P systems, we went down the path of investigating their features more deeply. Luckily P2P networks have been around for a while so there were many resources available. White papers about distributed hash tables (DHTs) proved to be the most critical, as DHTs are the foundation of current P2P networks. Countless hours were spent reading and digesting white papers, specification documents, blog posts, and StackOverflow answers. We made sure to have a good understanding of DHTs before we started any coding.
A Kademlia network consists of many Nodes.
Has a unique 160 bit ID.
Maintains a routing table containing contact information for other nodes.
Maintains its own segment of the larger distributed hash table.
Communicates with other nodes via 4 remote procedure calls.
Each node's routing table is divided into 'buckets' — each bucket contains contact information for nodes of a specific 'distance' from the current node. We will discuss the concept of distance in more detail shortly.
Each contact contains the ID, IP address, and port number of the other node.
Because Xorro is a file sharing application, the DHT segment will contain key/value pairs where each key is a file id, and the corresponding value is the file location.https://github.com/xorro-p2p/xorrohttps://xorro-p2p.github.io/