The World of Peer-to-Peer (P2P)/Chapter P2P Implementations

P2P Networks and Protocols

This chapter will try to provide an overview of what is Peer-to-Peer, it's historical evolution, technologies and uses.

P2P and the Internet: A "bit" of History

P2P is not a new technology, P2P is almost as old as the Internet, it started with the email protocols and the next generation were called "metacomputing" or classed as "middleware". The concept of it took the Internet by storm only because of a general decentralization of the P2P protocols, that not only gave power to the simple user but also made possible savings on information distribution resources, a very different approach from the old centralization concept.

This can be a problem for security or control of that shared information, or in other words a "democratization" of the information (the well known use of P2P for downloading copies of MP3s, programs, and even movies from file sharing networks), and due to it's decentralizing nature the traffic patterns, are hard to predict, so, providing infrastructures to support it is a major problem most ISPs are now aware.

P2P has also been heralded as the solution to index the deep Web since most implantations of P2P technologies are based and oriented to wired networks running TCP/IP. Some are even being transfered to wireless uses (sensors, phones and robotic applications), you probably have already heard of some military implementation of intelligent mines or robotic insect hordes.

Ultimately what made P2P popular was that it created a leveled play field, due to the easy access to computers and networking infrastructures we have today in most parts of the world. We are free to easily become producers, replacing the old centralized models where most of the population remained as consumers dependent on a single entity (monopoly, brand, visibility) for the distribution or creation of services or digital goods. This shift will undoubtedly reduce the costs of production and distribution in general as the price of services and products that can be digital transfered, the cost is now also becoming evident the quality will also be downgraded until a new system for classification emerges, this can be seen today in relation to the written media after the Internet impact.

FidoNet

FIdoNet is still today a worldwide computer network that is used for communication between bulletin board systems (BBSes). It uses a store-and-forward system to exchange private (email) and public (forum) messages between the BBSes in the network, as well as other files and protocols in some cases.

The FidoNet system was started based in a number of small collaboratively interacting programs. These "peers" run alongside the BBS systems and interacted with them via scripts or some form of lower level inter-precess communications. Their function was to automatically pack/unpack and import/export content from one system's location to another. Being independent greatly eased porting, and FidoNet was one of the few networks that was widely supported by almost all BBS software, as well as a number of non-BBS online services. This modular construction also allowed FidoNet to easily upgrade to new data compression systems, which was important in an era using modem-based communications over telephone links with high long-distance calling charges.

The rapid improvement in modem speeds during the early 1990s, combined with the rapid decrease in price of computer systems and storage, made BBSes increasingly popular. By the mid-1990s there were almost 40,000 FidoNet systems in operation, and it was possible to communicate with millions of users around the world. Only UUCP came close in terms of breadth or numbers; FidoNet's user base far surpassed other networks like BITNET.

The broad availability of low-cost Internet connections starting in the mid-1990s lessened the need for FidoNet's store-and-forward system, as any system in the world could be reached for equal cost. Direct dialing into local BBS systems rapidly declined. The availability of internet connectivity is by no means universal, and although FidoNet has shrunk considerably since the early 1990s, it remains in use around the world.

eMail

Electronic mail (often abbreviated as e-mail or email), started as a centralized service for creating, transmitting, or storing primarily text-based human communications with digital communications systems with the first standardization efforts resulting in the adoption of the Simple Mail Transfer Protocol (SMTP), first published as Internet Standard 10 (RFC 821) in 1982.

Modern e-mail systems are based on a store-and-forward model in which e-mail computer server systems, accept, forward, or store messages on behalf of users, who only connect to the e-mail infrastructure with their personal computer or other network-enabled device for the duration of message transmission or retrieval to or from their designated server.

Originally, e-mail consisted only of text messages composed in the ASCII character set, today, virtually any media format can be sent today, including attachments of audio and video clips.

Peer2Mail

Peer to Mail ( http://www.peer2mail.com/ ) is a FreeWare application for Windows that lets you store and share files on any web-mail account, you can use Web-mail providers such as Gmail (Google Mail), Walla!, Yahoo and others, it will split the shared files into segments that will be compressed and encrypted and then sends the file segments one by one to an account you have administration access. To Download the files the process is reversed.

Security

The encryption was broken in Peer2Mail v1.4 (prior versions are also affected) - Peer2Mail Encrypt PassDumper Exploit.

Usenet

Usenet is the original peer to peer file-sharing application. It was originally developed to make use of UUCP (Unix to Unix Copy) to synchronize two computers' message queues. Usenet stores each article in an individual file and each newsgroup in its own directory. Synchronizing two peers is as simple as synchronizing selected directories in two disparate filesystems.

Usenet was created with the assumption that everyone would receive, store and forward the same news. This assumption greatly simplified development to the point where a peer was able to connect to any other peer in order to get news. The fragmentation of Usenet into myriad newsgroups allowed it to scale while preserving its basic architecture. 'Every node stores all news' became 'every node stores all news in newsgroups it subscribes to'.

Of all other peer-to-peer protocols, Usenet is closest to Freenet since all nodes are absolutely equal and global maps of the network are not kept by any subset of nodes. Unlike Freenet, which works by recursive pulling of a requested object along a linear chain of peers, Usenet works by recursive pushing of all news to their immediate neighbors into a tree.

FTP

The File Transfer Protocol (FTP), can be seen as a primordial P2P protocol. Even if it depends on a client/server structure the limitation is only on the type of application (client/server) one run since the roles are flexible.

Zero configuration networking

Zero configuration networking (zeroconf), is a set of techniques that automatically creates a usable Internet Protocol (IP) network P2P fashion, without manual operator intervention or special configuration servers.

Bonjour

Bonjour, formerly Rendezvous. A service discovery protocol by Apple Inc.'s. Bonjour locates in a P2P fashion, devices such as printers, as well as other computers and the services that those devices offer on a local network using multicast to maintain a Domain Name System record. The software is built into Apple's Mac OS X operating system from version 10.2 onward, and can be installed onto computers using Microsoft Windows operating systems. Bonjour also supports components that included other software, such as iTunes.

Bonjour for Windows ( http://support.apple.com/downloads/Bonjour_for_Windows )
Bonjour for Windows includes a plugin to discover advertised HTTP servers using Internet Explorer. If you have Bonjour devices on your local network with embedded HTTP (Web) servers, they will appear in the list.

Internet Relay Chat (IRC)

Internet Relay Chat, commonly abbreviated IRC is a real-time text-based multi-user communication protocol specification and implementation; it relays messages between users on the network. IRC was born sometime in 1988, from the mind of Jarkko Oikarinen. According to IRChelp.org ( http://www.irchelp.org/irchelp/rfc/ ), the official specification for IRC was written in 1993 in the RFC format. The protocol was defined in the "RFC 1459: Internet Relay Chat Protocol", a really excellent source for both an introduction to and detailed information about the IRC protocol.

IRC's largest unit of architecture is the IRC network. There are perhaps hundreds of IRC networks in the world each one running parallel and disjoint from the others. A client logged into one network can communicate only with other clients on the same network, not with clients on other networks. Each network is composed of one or more IRC servers. An IRC client is a program that connects to a given IRC server in order to have the server relay communications to and from other clients on the same network but not necessarily the same server.

Messages on IRC are sent as blocks. That is, other IRC clients will not see one typing and editing as one does so. One creates a message block (often just a sentence) and transmits that block all at once, which is received by the server and based on the addressing, delivers it to the appropriate client or relays it to other servers so that it may be delivered or relayed again, et cetera. For a look into the messages exchanged on an IRC network you can take a look into (http://www.alien.net.au/irc/irc2numerics.html), it clearly identifies the several implementations and functions.

Once connected to a server, addressing of other clients is achieved through IRC nicknames. A nickname is simply a unique string of ASCII characters identifying a particular client. Although implementations vary, restrictions on nicknames usually dictate that they be composed only of characters a-z, A-Z, 0-9, underscore, and dash.

Another form of addressing on IRC, and arguably one of its defining features, is the IRC channel. IRC channels are often compared to CB Radio (Citizen's Band Radio) channels. While with CB one is said to be "listening" to a channel, in IRC one's client is said to be "joined" to the channel. Any communication sent to that channel is then "heard" or seen by the client. On the other hand, other clients on the same network or even on the same server, but not on the same channel will not see any messages sent to that channel.

Updated information on IRC can be obtained at IRC.org, the move to support IPv6 and the new technical papers, the IETF (Internet Engineering Task-Force) approved the most current technical drafts ( April 2000 - authored by C Kalt):

     RFC 2810 : IRC Architecture
     RFC 2811 : IRC Channel-Management
     RFC 2812 : IRC Client-Protocol
     RFC 2813 : IRC Server-Protocol

These documents are already available on IRC.org's official FTP-server, reachable at ftp://ftp.irc.org/irc/server

While IRC is by definition not a P2P protocol, IRC does have some extensions that support text and file transmission directly from client to client without any relay at all. These extensions are known as DCC (Direct Client to Client) and CTCP (Client To Client Protocol).

Ident Protocol

The Ident Protocol, specified in RFC 1413, is an Internet protocol that helps identify the user of a particular TCP connection, and differentiate them from others sharing the same connection on the a server.

The Ident Protocol is designed to work it self as a server daemon, on a user's computer, where it receives requests to a specified port, generally 113. The server will then send a specially designed response that identifies the username of the current user.

Most standalone Windows machines do not have an Ident service running or present by default, in this case you may need to run your own Ident server (there are several stand alone servers available), on the other hand if you are on a Unix/Linux machine the service is there by default. Some Windows IRC clients have also an Ident server built into them.

The reason for having an running Ident server is due to IRC servers using the information for security reasons (not a particularly efficient way of doing), some going so far as blocking clients without an Ident response, the main reason being that it makes it much harder to connect via an "open proxy" or a system where you have compromised a single account of some form but do not have root.

CTCP (Client To Client Protocol) Protocol

With CTCP, clients can implement commands such as "ctcp nickname version" or "ctcp nickname ping" to get some interesting infos about other users (like mIRC does).

Bots or Robots

IRC systems also support (ro)bots, in this case they are not real users but a collection of commands that are loaded from a script (text) file into the IRC client, or even a stand alone program that connects to a IRC channel. They serve to ease the human interaction with the system, provide some kind of automation or even to test or implement some AI.

Basic Commands

Here are some basic commands for IRC:

Command	What it does	Example
/attach /server	Sign on to a server	/attach irc.freenode.net /server irc.freenode.net
/nick	Set your nickname	/nick YourName
/join	Join a channel	/join #wikibooks
/msg	Sends a message (can either be private or to the entire channel)	Message the channel: /msg #wikibooks hello world! Send a private message: /msg JohnDoe Hi john.
/whois	Display information about a user on the server	/whois JohnDoe
/clear /clearall	Clears a channel's text. Clears all open channel's text.	/clear /clearall
/away	Sets an away message. Note: Type /away again to return from away.	/away I'm away because...
/me	Sends an action to the channel. See example.	The following: /me loves pie. would output to the chat in the case of JohnDoe: JohnDoe loves pie.

Privileged User Commands

Commands for half-operators, channel operators, channel owners, and Admins:

Command	What it does	Example
/kick	Kicks, or boots a user from the channel. You must be a half-operator or greater to do this.	Kick a user from the channel with a reason: /kick JohnDoe I kicked you because...
/ban /unban	Bans a user from the channel. You must be a channel operator or greater to do this. Unbans a user from the channel. You must be a channel operator or greater to do this.	/ban JohnDoe /unban JohnDoe

IRC Networks

• EFnet
• Undernet
• Dalnet
• Rede Portuguesa de IRC (PTnet) ( http://www.ptnet.org/ ), is the biggest Portuguese IRC network, it was created in 1997, you can get an updated list of it's server ( http://www.ptnet.org/servidores ).

Security Risks

Software Implementations

• KVIrc ( http://www.kvirc.net/ ) an open source (GPL) portable IRC client based on the Qt GUI toolkit and coded in C++.
• Bersirc ( http://bersirc.free2code.net/index.php/home/ ), an open source IRC client (LGPL), coded in C, that runs on Windows (Linux and Mac OS X ports under development) by utilizing the Claro GUI Toolkit.
• XChat ( http://www.xchat.org/ ) is an IRC (chat) program for Windows and UNIX (Linux/BSD) operating systems. I.R.C. is Internet Relay Chat. XChat runs on most BSD and POSIX compliant operating systems. Open Source (GPL), coded in C.
• Irssi ( http://irssi.org/ ), an IRC client program originally written by Timo Sirainen, and released under the terms of the GNU General Public License. It is written in the C programming language and in normal operation uses a text-mode user interface.
• mIRC ( http://www.mirc.co.uk/ ), a shareware Internet Relay Chat client for Windows, created in 1995 and developed by Khaled Mardam-Bey. This was originally its only use, but it has evolved into a highly configurable tool that can be used for many purposes due to its integrated scripting language.

You can also check Wikipedia list of IRC clients and Comparison of Internet Relay Chat clients (not up-to-date)...

Invisible IRC Project

A technological advancement in relation to normal IRC networks, created by invisibleNET, a research & development driven organization whose main focus is the innovation of intelligent network technology. Its goal is to provide the highest standards in security and privacy on the widely used, yet notoriously insecure Internet.

Invisible IRC Project (http://invisibleip.sourceforge.net/iip/) is a three-tier, peer distributed network designed to be a secure and private transport medium for high speed, low volume, dynamic content. Features:

• Perfect Forward Security using Diffie-Hellman Key Exchange Protocol
• Constant session key rotation
• 128 bit Blowfish node-to-node encryption
• 160 bit Blowfish end-to-end encryption
• Chaffed traffic to thwart traffic analysis
• Secure dynamic routing using cryptographically signed namespaces for node identification
• Node level flood control
• Seamless use of standard IRC clients
• Gui interface
• Peer distributed topology for protecting the identity of users
• Completely modular in design, all protocols are plug-in capable

The IIP software is released under the GPL license and is available for Windows 98/ME/NT/2000/XP, *nix/BSD and Mac OSX, coded in C.

Instant Messaging

Instant messaging can be considered a subtype of P2P, in simple terms it consists on the act of instantly communicating between two or more people over a network (LAN or WAN) mostly using text. This requires the use of a client program so that when a message is sent, where a notification is shown a short time after on the destination application, enabling it's user to reply to the original messages. IM protocols can be centralized or distributed or a mix of the two.

Instant messaging allows users to send quick notes or reminders to other users in almost real time. IM can, but may not, include any P2P implementation or support extra P2P services like, file-sharing, VoIP or Video Conference the broad definition is that IM is the almost instantaneous trading of messages, whatever form it takes.

Since any P2P network depends on participation, supporting some kind of IM implementations is very important as a way to create a community and sustain the network.

IM Software Implementations

• Gaim/Pidgin ( http://pidgin.im/pidgin/home/ ) OpenSource (GPL) instant messaging client supporting Windows, GNU, BSD, and many Unix derivatives and compatible with AIM, ICQ, MSN, Yahoo!, IRC, Jabber, Gadu-Gadu, SILC, GroupWise Messenger, and Zephyr networks.
• Trillain ( http://www.ceruleanstudios.com/ ) skinnable chat client that supports AIM, ICQ, MSN, Yahoo!, and IRC, it also includes many features not included in those chat programs.
• BitWise IM (http://www.bitwiseim.com), Encrypted Cross-Platform (Windows, Mac OS X, and Linux) Instant Messaging, free but closed source, uses wxWidgets. Also supports a Whiteboards, Voice Chat.
• digsby (http://www.digsby.com), a closed source, windows only, multiprotocol IM client that lets you chat with all your friends on AIM, MSN, Yahoo, ICQ, Google Talk, and Jabber with one simple to manage buddy list.
• Google Talk (http://www.google.com/talk/), Windows XP+ only, closed source, supports IM and interacts with the Gmail (google WEB mail) platform.

VoIP

Voice over IP can also be seen like an extension of a IM were text is substituted by live audio or video, the technological challenges are very similar, if not considering the type data of data that needs to be transfered and specific considerations due to timings. It is not uncommon for IM applications to also support VoIP or video conferencing.

Security on VoIP faces the same vulnerabilities and security threats of other P2P protocols and applications, including fuzzing, floods, spoofing, stealth attacks and VoIP spam.

Napster

The Napster network was created at application-level using a client-server protocol over point-to-point TCP. The server was in this case a centralized directory that would hold an index of all files offered (MP3/WMA). The clients would connect to the server, identify themselves to the server (users had an account on the server) and send a list of MP3/WMA files they were sharing to it enabling other clients to search that central repository for any file on the network and then request it from any available source.

Napster protocol specifications

Software Implementation

• OpenNap ( http://opennap.sourceforge.net ), a Napster based peer-to-peer, created as open source (GPL), in C using Win32, and so for Windows. Intended to extend the Napster protocol to allow sharing of any media type, and adding the ability to link servers together. Discontinued.
• audioGnome ( http://www.audiognome.com ), closed source but as freeware for Windows.
• JNerve ( http://jnerve.sourceforge.net ), an open source (GPL) Java Napster Server Protocol implementation with a goal of cross-platform compatibility.
• Napsack ( http://napsack.sourceforge.net ) is a specialized multi-threaded client for broadcasting Napster queries across multiple servers; the list of target servers is retrieved from www.napigator.com, and is user-filterable (based on the number of users, files, or gigs indexed). Opensourced (GPL) using Java.

Gnutella

Gnutella is an open file sharing Network originally created by Justin Frankel of Nullsoft. That means, unlike most other networks, everyone can write a client which can access the GNet, if it fulfills the publicly available specifications.

The specifications are discussed and created by the GDF (the gnutella development forum), an open Mailinglist for developers, with by this date over 1000 members. After that they are documented in the rfc-gnutella. In this way all programs share a common base, while the protocol also allows for client specific options. The developers are careful to ensure the greatest possible backwards compatibility.

Despite the name, Gnutella isn't GNU-Software, though some Gnutella clients are GPL-licensed. It is an open network, and the origin of its name may be found more easily by eating too much Nutella, than at GNU. (That means: Gnutella is not a project of the FSF or related to GNU software tools). And while Gnutella was initially stated as a fully-distributed information-sharing technology, later versions of the protocol are a mix of centralized and distributed networks with "Servers" ( Ultra or Super peers) and "Clients" ( Leafs or Nodes ).

A Gnutella client software is basically a mini search engine (offering an alternative to web search engines) and file serving system in one. Gnutella in new implementations also supports Tiger tree hashing (TTH) for for file transfers.

One sibling of Gnutella deserves special attention, even though some developers of current clients would deny it. It is called MP (Mikes Protocol) or the Shareaza Protocol by most Gnutella developers, while its developer called it Gnutella2, a name which gave his program (Shareaza) much media coverage and created and creates much controversy and aversion against it in the_gdf.

Gnutella2 ( Mike's Protocol,G2 )

The result of a fork of the Gnutella protocol, due to the failing of the developers community to reach a consensus on the evolution of the protocol.

Gnutella2 is also called Mike's Protocol since the first changes and implementation resulted from a single developer Michael Stokes. In November 2002, Michael Stokes formally and unilaterally announced the creation Gnutella2 protocol to the Gnutella Developers Forum, which caused a schism among the developers, and lead for the modifications no to be supported in several Gnutella applications since the original proposal did conflict with other vendors concepts (in specific LimeWire and Bearshare).

The now resulting implementation drops all of the old Gnutella protocol except for the connection handshake and adopts an entirely new search algorithm. Gnutella2 is often abbreviated as G2.

Network model

The Original: FoF

You can imagine the original model of the Gnutella network as friends phoning each other to get information. One asks five others, each of whom asks 5 others and so on. After the first step the number of people reached is 5, after the second it is 25, after the 5th 3125, after the 7th 78,125 and after the 14th about 6.1 billion. That would be enough to reach every human being on this planet. The original Gnutella used 7 steps (called HTL: Hops To Live).

The biggest problem with this model (among others) is that you have to be a part of the clique before you can use it.

Problems of the FoF-model

The Friend-of-a-Friend model has certain disadvantages, which have their source in the way searches are performed. If a search brings too many results, the nodes, through which you are connected (your nearest 5 friends) can get overloaded, because every answer has to go through them, for they don't give out your \"phone number\", but their own and hand the answer to you. If you ask for the name of the head of University in the campus, you'll get hundreds of answers in reality, and thousands to millions on the web. Also, if every question is passed to every one in a 75,000 to 600,000 computer-network, and every computer asks only once an hour, each of them has to answer about 130 to 1600 questions per second. And they have to pass them on. While computers are fast, and today's Internet connections can handle quite a lot when compared to the connections a few years ago, this is too much even for them. Just imagine your phone ringing endlessly the whole day for all kind of questions.

Getting in

To address the connection problem several ideas to solve have come about.

The first way: Pong-Caching

Pong Caching means that the node (aka you) asks its friends who their friends are. It means your friends introduce you to their friends, especially friends whom they value highly, and you write all new addresses in your phone-book, so you know whom to phone when your original friends are on holiday (Somehow like being at a continuous cocktail party). It is easy and has the advantage of giving you very reliable contacts, but there is no way of getting into the network without knowing at least one contact who is already in the net. That means you can always get back in, but won't be able to connect if you never did before.

The second way: Remember who answers

The second way is really simple. When one of your 5 friends calls back to say Smith (whom you didn't know before) knows something, you note her number. When you call him the next time as one of your five direct contacts, the chance is greater that you will get your information more quickly, because he will likely have friends who have similar interests to you (where else should he have gotten the information?), and those are more likely to have your information than randomly picked persons (at least when you ask about something similar to your last question). The drawback is that those contacts might not be at home often, so it might be that you find a contact with great knowledge, but whom you'll never be able to reach again. Still no way of getting in the first time. And now we get to one of the recent developments in Gnutella: GWebCaches. I will discuss them in the next part.

The third way: GWebCaches

To stay within the picture, a GWebCache is a contact who puts her phone number into the newspapers and keeps a record of those who call. When you've been away for some time and are no longer certain if your contacts still have the same mobile-phone numbers, you call the publicly known contact. Before giving you numbers, he/she will ask you: "Do you know other publicly known contacts? If yes, please tell me their numbers." That is done because they can't read all the newspapers, and you do it all the time without working too hard for it. That way, they keep track of each other. Then the contact gives you some numbers to call and notes your number (to give it to someone else) and the addresses of other public contacts he/she knows (GWebCaches).

This is roughly the way GWebCaches work. GwebCaches are essential only for the first connections. You use them when your local address-book is empty. They must not be preferred over your local address-book. As I stated, they are one of the new developments in Gnutella, and thus I will now get to some more of the recent changes within Gnutella and to future plans.

Ultrapeers and Leafs

Change who calls whom

You'll surely have friends who know very many other people, and whom you can ask, and be sure they'll know exactly the person who can give you the answer. These are called Ultrapeers in Gnutella. An Ultrapeer doesn't have to know much himself/herself, he/she just needs to know who knows it. In Gnutella that means that a good Ultrapeer doesn't need to have many files to benefit the network. If you're afraid to share much, you should become an Ultrapeer in Gnutella.

In more detail

In the Computer World, as in the Real World, there are contacts who can cope with more calls, and those who can't phone often (or can't afford the bills). In the Real World this is because they have more free time. Whereas, in the Computer World, it is because they have faster connections (Like DSL, Cable, T1, T3 or similar broadband). Upon realizing this, the developers decided to change the topology, that means how the network looks from the outside when you draw it. Now you don't just call any of your friends, but only those whom you know have the time to take your call and to send it on to others. To save you from too many calls, they then ask you which kinds of informations you have or, to express it in a more human way, what your specialty is. In the Computer World that means your computer sends a list of all its files to the Ultrapeer, which is how we call these kinds of contacts. That list contains summaries (Hash-Strings) of all your shared files (those you decide to let others download) by which the downloader can verify that they are, indeed, the files he or she wants. Whenever a call reaches the Ultrapeer, it checks if you could know an answer and calls you only in that case.

These Ultrapeers have many connections to others, which means they have a really big address book. Normally they stay in contact with 16 other Ultrapeers whom they have in their address book and to whom they send questions, and who send them to 16 more, each. Also they have about 16 leafs, who can't or don't want to phone that much, from which they accept calls, and whose files or, for the human world, specialties they know.

This may seem like a foul bargain for the Ultrapeers, who devote far more resources to keeping the network intact than leafs, but in fact it isn't. While the Ultrapeer (UP) uses much of her time for keeping the network running, the leafs specialize on gathering and delivering information. So, when anyone, Ultrapeer or Leaf, wants to know something, he or she simply starts a call and a leaf specialist can explain it to them. That way people specialize to get more for all.

Intra-ultrapeer QRP

While with Ultrapeers not everyone needs to participate in sending questions to others, and people can specialize in sharing their information instead, the Ultrapeers would still send every question to everyone, without ever taking into account if that UP even has leaves, who have the files. This sounds normal, for how can an Ultrapeer know which files the other Ultrapeers have? The answer comes, again, from real life. A normal person knows her friends, and knows who of them might know the answer to a specific question, and who most surely will not. In Real Life this is mostly done through friendly chatting.

Now, computers normally don't chat idly, so they don't exchange this information by the way. Thus the Query Routing Protocol was developed. There each Leaf tells its Ultrapeers which files it has, but instead of taking the names, which would consume too much space, each word which is part of the name of a file is saved as numbers (these are computers after all). You can imagine this process like a game of BattleShips (the numbers form the board with two coordinates). An Ultrapeer doesn't send all questions to a leaf, but only those which it might be able to answer (which hit a ship), and so Leafs get far less needless calls.

Now when this takes so much pressure off the leaves, why not extend it? Exactly that was done. Now all Ultrapeers send their boards to their direct neighbors. They send only those searches, which have one more step to go, to other Ultrapeers on whose board they score a hit. That means, the last two steps of a search will only be taken when there is a chance that they give results. You can see quite simply why this heavily reduces the bandwidth usage: imagine a tree, a normal tree, not one of those mathematical constructs. If you try to count the leaves, you have almost no chance. But if you take the leaves away and count only the branches, you have far less work to do. If you now take away all those tiny branches, you can really begin to count them. QRP doesn't take all leaves and all tiny branches away, but it removes those of them who couldn't give you an answer. Since every part through which a question has to travel consumes bandwidth, and there are far more leaves than branches, taking away, in many cases, many of the last two steps (that means many of the leaves and the tiny branches) reduces the number of questions the computers have to send on (there are far more leaves, than branches). The example doesn't work for all of Gnutella, but here it fits nicely.

Searching: Dynamic Querying

Now, while the Ultrapeer model and QRP partly solve the problem that you don't have the time to explain something properly to someone else, or to get it explained, because the phone rings endlessly for questions to which you know no answer (or in Tech-Speech: because the network-traffic exceeds your connection-speed), there is still another problem which might normally not even be visible, if you look at it. In the Real World, an Ultrapeer will ask for a specialist who can give you the info until he/she finds one, and then stops. In the Computer-World, the question is sent on and on, to as many contacts as possible, without looking if there already are answers.

With dynamic Querying that changes. Now the Ultrapeers ask one other Ultrapeer at a time, and wait a bit, to see if they get answers. When they have enough answers to be satisfied, they stop asking for more. It sounds pretty natural, but was quite a big step for Gnutella because it saves resources which were wasted on very popular questions. I'll take the example of the head of university again: now, if you ask for the head of university, your Ultrapeers will first see if they know someone directly who can answer your question. Then they will simply give you some numbers of people they know who live on the campus. You will still get more than one answer because they will give you more than one number, as they can't be sure that you'll reach every number they gave you. But you won't get thousands of phone-numbers (one from every student on the campus), first because the Ultrapeers would waste their time with that on something which doesn't give you additional benefit, second, because you couldn't ever call all those people, and third, because then you might not reach your Ultrapeers anymore, because they would be so busy getting return calls from others who tell them numbers, and sending your question to other Ultrapeers.

Finding sources without searching aka the Download-Mesh

Now you might say, "but I can't download from those three, because others already do. I want to get all addresses from which I can download," (and you are not alone with this. I feel the same). By looking at the Real World, we can find a solution to this problem too, without having to waste too many resources on it. There (in the Real World), if you ask a specialist to explain something to you, and that specialist is busy, he or she will know some other specialists (because they know each other) who might have more time at the moment.

Getting this into Gnutella is not as easy as the Ultrapeer-Leaf Model nor as the Dynamic-Query Model. But the programmers found a way. As I stated at the Dynamic-Query-Model, you will get more than one number where you can ask. Now, when you call someone who should know the answer, you also give him or her the other numbers you know about. That way, the specialists will get to know each other (the same way, as the GWebCaches, which I mentioned before, learn of others of their kind). As everyone who asks also brings her own set of numbers, the specialists know more and more additional addresses, and when you ask them to explain, and they don't have time at the moment, they give them to you (they do it even if they have time, just in case they could be interrupted, and because in Gnutella you can download from more than one source at once, just like you can in the overnet-network (which does this to the extreme, but is only really efficient for big files). Additionally, the specialists also add you to their number of alternate-contacts, as soon as you know enough to teach others.

This is why often many people download files from you which you just downloaded yourself.

Downloading

Swarming and Partial File Sharing

Swarming is quickly explained (but hard to do in the friend-of-a-friend model, so I drop it for this part only). It works by simply getting one file from more than one person at once. The file is simply separated into several parts, as if you'd want to get a book from some friends and every one of them copied only a few pages of it. When you ask every one of them to copy a different part of the book, you'll get the complete book, and every one of them has only very little work to do (and if one doesn't have the time to do it, another one can).

Swarming works best with the Download Mesh and Partial File Sharing (PFS), which allows people to share files which they are downloading at the moment, because they can share those parts which they already have, while they still download from others. You can copy those pages which you have without having to have the whole book, since your pages are all numbered and friends can ask you for certain page numbers.

Downloading through Firewalls

Imagine there were people who couldn't be called, but could only call others (maybe because they only use public phones, or their number isn't displayed on your phone, and they don't like to give it out because they don't like being called by telemarketers, or by people terrorizing them over the phone). In Gnutella these are computers who are behind a firewall. They can call others and get information from them, but no one can call them.

The solution is to have the firewalled people call their Ultrapeers regularly and, when someone wants to call them, he or she simply calls the Ultrapeer who then holds two phones together, one to which the firewalled person (the one who can't be called) raised a call, and the one you called. That way you can talk to the firewalled person, but it takes two simultaneously running calls, which means, that it needs twice the bandwidth in the Computer-World. Firewalled persons always keep their connection to the Ultrapeers, who simply relay the information or data.

There are plans to save the Ultrapeers from this additional bandwidth usage by letting other people do the phone connecting. Then, when someone wanted to get information from a firewalled specialist, the Ultrapeer would tell the firewalled person and the asker to call a third person. That person would then hold the two phones together. In Gnutella most People have three to five phones, so this wouldn't be such a great problem. These phone-connectors will most likely be called routing-peers.

File-Magnets

File-Magnets stray from the Friend of a Friend model. They are links on Webpages, which you can simply click, and which will tell your file-sharing program to search Gnutella (in fact also other networks) for a specific file, and to download exactly this.

You can imagine it like an article in a newspaper which tells you information which gives your Ultrapeers the exact information that the specialist, from whom you want to learn, has to know. In the Real World you would most likely find one specialist and those who learned from him or her.

With a magnet-link you can avoid getting bad files because they use a hash-string, which is something like a summary of the information the specialist would give you. If he or she begins to tell you crap, you will see at once that it doesn't fit the summary. In Gnutella, the program asks for files to which the people who have them have assigned the same summary, aka Hash-string. After downloading, the program does its own summary and checks if they really match. If not, it tells you that the file is corrupt. The summaries from same files are always exactly the same because they are done via specific mathematic methods which always get to the same result when given the same data (aka information).

Different from Magnet-Links, KaZaA-Links and eDonkey-Links are not secure, because they use methods which can be betrayed with false files (for example a KaZaA-Link asks for a kind of summary which only checks the introduction and the first part of the information, but all the rest is ignored to make the summary quicker to create. Naturally it is very easy to give you false information, because they only have to tell the truth at the beginning, then they can lie or fantasize as much as they want). Further information on Magnet-Links can be found here: Magnet-Uri and on MagnetLink.org

There is now a new version of magnet-links: KaZaA magnets. Sadly those aren't secure, for they use the KaZaA hashing system (the incomplete summary) with some changes (they now add another smaller summary, which might tell you about the missing parts, but they didn't publish, how they create it). If KaZaA-Magnets provide information about a search term, they might work with Gnutella, but they won't ensure that you get what they offer to you. If you find the word \"kzhash\"in the link, it might not be secure (aside from having a somewhat misplaced name).

Lime Wire LLC

LimeWire a peer-to-peer file sharing client for the cross-platform Java Platform, Open Source (GPL), which uses the Gnutella network to locate and transfer files. It also encourages the user to pay a fee, which will then give the user access to LimeWire Pro. Support for BitTorrent protocol is also present by using the C++, Boost licensed, libtorrent library.

Software Implementations

To become part of the Gnutella Network, you can use one of the clients listed below :

• Deepnet Explorer ( http://www.deepnetexplorer.com/ ) a browser with RSS news reader, P2P client integration (Gnutella) and phishing alarm, closed source, Windows only, freeware.
• Phex cross-platform Java client.
• XoloX
• Gnucleus - Gnutella, Gnutella2 (G2) coded using C++ and Microsoft MFC libraries. The Core is LGPL and communicates to a GPL frontend using Windows COM-base.
• Gtk-Gnutella, GPL, for GNU/Linux.
• Hydranode ( multi-protocol, referenced on the eDonkey2000/eMule section )
• ezpeer, a Chinese client.
• pp365, a Chinese client.
• POCO, a Chinese, using GnucDNA.
• Bearshare, free closed source for Window.
• CocoGnut, for RISC OS.
• Swapper, free closed source for Windows utilizing .NET.
• TrustyFiles, for Windows, supports FatTrack (KaZaA), Gnutella and G2.
• Shareaza ( http://shareaza.sourceforge.net/ ), Open Source (GPL), coded in C++, MFC and ATL. Multi-network peer-to-peer file-sharing client supporting Gnutella2 (G2), Gnutella, eDonkey2000/eMule, BitTorrent, FTP and HTTP protocols.
• FrostWire ( http://sourceforge.net/projects/frostwire/ ), a Peer to Peer (P2P) information sharing client for the Gnutella network. This project is not affiliated with LimeWire LLC. It is a fork of Limewire's Java implementation committed to never including content filters. FrostWires' source code (Java) is Licensed under the GNU GPL Open Source license. Newer version have moved to use the BitTorrent protocol.
  • Acquisition, a Mac OS X client based on the Limewire core, written in Cocoa, shareware.
  • XNap, a multi-network program in Java using the Limewire Core for Gnutella.

Ares Network

Ares (software implementation) was developed in the middle of 2002, originally using the Gnutella network. After Six months of operation, it switched to its own network comprising the leaves-and-supernodes p2p architecture. Having a protocol that can be difficult to identify made Ares at times the only P2P client that could functions on restricted networks, such as some university campuses.

Software Implementations

• Ares ( http://aresgalaxy.sourceforge.net/ ), a Chat/File Sharing P2P implementation in Delphi/Kylix. It's based on a Network organized into leafs and supernodes into a topology featuring broadcast-type searches. Ares can deliver a broader search horizon by means of the DHT technology, using a mime filter to DHT engine. Ares users can also join chat rooms or host a channel. It is for 32-bit MS Windows Operating Systems (NT/2000/XP) and Open Source under the GPL License (GNU General Public License). From version 1.9.0, data sharing was enabled between two peers behind a firewall. From version 1.9.4, Ares included support for the BitTorrent protocol. From version 1.9.9, Ares Galaxy has support for the SHOUTcast internet radio stations.

Discontinued Implementations

• Warez P2P was a proprietary P2P filesharing service that uses the Ares network, and offers a service similar to that of Kazaa. Up to version 1.6, Warez P2P was a clone of Ares Galaxy, created by Italian developer Alberto Trevisan, but since then has been developed independently by Neoteric Ltd until recently when it was discontinued.

Direct Connect

Direct connect is a peer-to-peer file sharing protocol/network but it uses a central server, this reliance on a central point can also be seen on the old Napster network, in that each server build an independent network (not an hybrid like for instance with eMule). One should note that some clients are now also implementing DHTs that will result in unifying used networks. The Direct Connect protocol was originally developed by Jonathan Hess for use on the Neo-Modus Direct Connect (NMDC) v1, released September 2001 and partially in NMDC v2, released in July 2003.

Direct Connect defines the servers as HUBs. Clients connect to a central hub and that hub feature a list of clients or users connected to it. Users can then search for files to download, or as chat with other users present (on that server).

Direct Connect also implements Tiger tree hashing (TTH) for for file transfers.

NMDC Protocol

created by Jon Hess at Neo-modus protocol mirror ( http://www.teamfair.info/wiki/index.php )

ADC Protocol

The ADC protocol ( http://dcplusplus.sourceforge.net/ADC.html ) is similar to the Neo-Modus Direct Connect (NMDC) protocol. It consists of a text protocol for a client-server network, created with goal to be simple, yet extensible.

Jon Hess contributed to the creation of this protocol with the original Direct Connect idea through the Neo-Modus Direct Connect client / hub. Other major contributing source was Jan Vidar Krey's DCTNG draft that lead to subsequent work by Dustin Brody, Walter Doekes, Timmo Stange, Fredrik Ullner, Fredrik Stenberg and others.

HUB Software Implementations

• DConnect Daemon ( http://www.dc.ds.pg.gda.pl/ ), an open source Direct Connect's hub (working as daemon) written in C. Works currently under GNU Linux and FreeBSD, but is planned to be able to work on all Unixes and Windows. As a daemon it works in background and does not require any Xwindow system. Supports a telnet administration console.

Client Software Implementations

• NeoModus Direct Connect
• DC++ ( http://dcpp.net/ or http://sourceforge.net/projects/dcplusplus/ ), a project aimed at producing a file sharing client using the ADC protocol. It also supports connecting to the Direct Connect network, open source under GPL, using C++/MFC running on Windows. It is developed primarily by Jacek Sieka, nicknamed arnetheduck.
• BCDC++
• RevConnect ( http://www.revconnect.com/ ), supports also Kademlia, Open Source under GPL, using C++/MFC running on Windows.
• Strong DC++ ( http://strongdc.berlios.de/download.php?lang=eng ), Open Source under GPL, using C++/MFC running on Windows.
• ApexDC++ ( http://www.peerwebdc.tk/ - http://sourceforge.net/projects/apexdc/ ), based on Strong DC++. It has numerous features such as a plugin to block IP addresses (uses PeerGuardian blocklist), super seeding, customization, themes, and a nice Vista-type general user, Open Source under GPL, using C++/MFC running on Windows.
• TkDC++ ( http://tkdcpp.com ), Open Source under GPL, using C++/MFC running on Windows.
• GtkDC (http://gna.org/projects/gtkdc/), a Direct Connect client written in C, based on a GIMP toolkit 2.0 GUI. It is intended to run on UNIX-based platforms, but with a little port it should be executable on Windows platform with Gtk libraries. Licensed under the GNU General Public License V2 or later.
• FlylinkDC++ ( http://www.flylinkdc.com and http://www.flylinkdc.ru ), based on StrongDC++

eDonkey

eDonkey the original client for the eDonkey network (also known as eDonkey2000 network or eD2k), was created and managed by MetaMachines (Sam Yagan and Jed McCaleh) based on the city of new York. It had a stable P2P community and the protocol was older than BitTorrent it was created in 2002 shortly after the closing of Napster and competed with the FastTrack network. In June of 2005, the entertainment industry gained a victory in the Supreme Court (USA) that stated that every file-sharing developer could be sued for copyright infringement if they induced such behavior. In September 2005 the Recording Industry Association of America (RIAA) sent several commercial P2P developer cease and desist letters including to MetaMachines and with no funds to battle the interpretation of the Supreme Court decision, Sam Yagan conceded defeat as he testified to the United States Senate Judiciary Committee.

On September 11, 2006 users could not get the eDonkey2000 client software, in September 12, 2006 MetaMachines settles for $30 Million (US) and the agreement closes any avenue MetaMachines had in dealing with any P2P technology in the future...

The eDonkey networks is centralized (as it depends on serves) to provide decentralized sharing of content (not stored on the servers), there are still many software implementations that support the network the most popular is eMule.

Protocol

"The eMule Protocol Specification" ( http://sourceforge.net/project/showfiles.php?group_id=53489&package_id=145950 ) by Yoram Kulbak and Danny Bickson DANSS (Distributed Algorithms, Networking and Secure Systems) Lab - School of Computer Science and Engineering - The Hebrew University of Jerusalem, Israel - January 20, 2005, PDF document provided by the Emule Project.

Kademlia

Started as the Overnet project by Jed McCaleb, the creator of eDonkey2000 to overcome the need of servers. Overnet implemented the Kademlia algorithm. In late 2006, Overnet and all Overnet-owned resources were taken down as a result of legal actions from the RIAA and others. However, since the core of Overnet is decentralized, Overnet clients are still able to function with limited functionality.

The KadC library (http://kadc.sourceforge.net/ ) provides an OpenSource C library to publishing and retrieving records in Kademlia-based Distributed Hash Tables.

A some what old paper named Kademlia: A Peer-to-peer Information System Based on the XOR Metric by Petar Maymounkov and David Mazières can also be a source of information about the protocol.

The network is now known as Kademlia and is supported by many of the implementations of the old eDonkey/Overnet clients, especially by the eMule project. Kademlia is a research effort to implement a full-featured peer-to-peer system based on the XOR metric routing. Of special interest are the objectives for efficient data storage and query; anonymity; network, content and user security and authentication.

eMule content database

eMule mascot.

( http://content.emule-project.net/ ) a service provided by the eMule project team for the eDonkey2000 and Kad network users, to make free content available for download and easy to find. The content database has been on line since around new years 2004.

Software Implementations

• eMule ( http://www.emule-project.net/ ) a filesharing software implementation based on the eDonkey2000 network but offers more features than the standard client, open source C++/MFC and windows only, licensed under GPL ( http://sourceforge.net/projects/emule/ )
• Xmod ( http://savannah.nongnu.org/projects/x-mod/ ) The Xmod is a Project is based on the eMule Client, OpenSource under the GPL.
• xMule ( http://www.xmule.ws/ ), the X11 Mule, intended to bring a clone of eMule to virtually all the major Unix platforms, with a particular emphasis on Linux. C++ using wxWidgets for the GUI released as OpenSource under the GPL.
• MLdonkey ( http://mldonkey.sourceforge.net ) is a multi-platform, multi-network P2P implementation. It supports several large networks such as eDonkey, Overnet, Kademlia, Bittorrent, Gnutella (Bearshare, Limewire, etc.), Gnutella2 (Shareaza), or Fasttrack (Kazaa, Imesh, Grobster). Networks can be enabled or disabled. Searches are performed in parallel on all enabled networks. For some networks, each file can be downloaded from multiple clients concurrently.
• AMule ( http://www.amule.org/wiki/ ) this project is based on the eMule Client, using also C++ but also wxWidgets and crypto++. Opensource under the GPL, currently supports Linux, FreeBSD, OpenBSD, Windows, MacOS X and X-Box on both 32 and 64 bit computers.
• eMule Bowlfish ( http://pwp.netcabo.pt/DeepSea/ ), another eMule based project that aims to provide an restricted Network solution.
• Hydranode ( http://hydranode.com/ ) a modular, plugin-driven peer-to-peer client framework which is designed with true multi-network downloads in mind (Support for eDonkey2000 and Bittorrent networks). OpenSource under the GPL, supports Linux and Windows.
• Shareaza ( multi-protocol, referenced on the Gnutella section )

BitTorrent

BitTorrent is a protocol ( BitTorrent Protocol Specification v1.0 ) created by Bram Cohen derived from the Gnutella concept, but primarily designed to distribute large computer files over the Internet and permit WEB integration, in fact it aimed to be a substitute for the old centralized HTTP downloads, not a full P2P network. As such it initially avoided the limitations of transmitting search request across the network, something that recently has been implemented with the adoption of a DHT similar to the eDonkey solution, that permits searches across a network. Making BitTorrent a two tiers P2P.

BitTorrent is used to distribute legitimate content but in itself does not make any differentiation about the copyright status of the shared material, as any other decentralized network this permits infringement on a massive scale. Bram Cohen and Ashwin Navin founded on September 22, 2004, BitTorrent, Inc. headquartered in San Francisco, California, as a privately held American company that develops transformative technology and products to continue the advancement of a more efficient and open Internet also promotes development of the BitTorrent protocol through R&D and open specifications.

BitTorrent ( http://www.bittorrent.com ) is also the name of the original implementation of the protocol. It started as a Python ( source code and old versions ) application, now refereed as BitTorrent Mainline, that lead to a full featured commercial enterprise. BitTorrent.com part of BitTorrent Inc. is now a destination to download entertainment content using the BitTorrent protocol. The site provide fast, on-demand access to the most comprehensive licensed catalog of thousands of movies, TV shows, music and games, but it also provides content creators a publishing platform to list their works in high-quality alongside the most recognizable titles from major movie studios, TV networks, and record labels.

BitTorrent Inc. also contributes through more broadly focused standards bodies like the Internet Engineering Task Force (IETF) at the LEDBAT working group ( http://www.ietf.org/html.charters/ledbat-charter.html ). BitTorrent, Inc. owns the clients BitTorrent Mainline and µTorrent, as the BitTorrent DNA (Delivery Network Accelerator) which is a free content delivery service based on the BitTorrent protocol that brings the power of user-contributed bandwidth to traditional content publishers while leaving publishers in full control of their files.

The BitTorrent protocol is peer-to-peer in nature, its innovative approach in the beginning, was due to not be centered about the creation a real distributed network but around the specific shared resources, in this case files, preferably large files, as users connect to each other directly to send and receive portions of a large file from other peers who have also downloaded either the file or parts of the it. These pieces are then reassembled into the full file. Since the users are downloading from each other and not from one central server, the bandwidth load of downloading large files is divided between the many sources that the user is downloading from. This decreases the bandwidth cost for people hosting large files, and increases the download speeds for the people downloading large files, because the protocol makes use of the upstream bandwidth of every downloader to increase the effectiveness of the distribution as a whole, and to gain advantage on the part of the downloader. However, there is a central server (called a tracker) which coordinates the action of all such peers. The tracker only manages connections, it does not have any knowledge of the contents of the files being distributed, and therefore a large number of users can be supported with relatively limited tracker bandwidth.

The key philosophy of BitTorrent is that users should upload (transmit outbound) at the same time they are downloading (receiving inbound.) In this manner, network bandwidth is utilized as efficiently as possible. BitTorrent is designed to work better as the number of people interested in a certain file increases, in contrast to other file transfer protocols.

BitTorrent is redefining the way people share and search for content and is getting very popular for downloading movies, TV shows, full music albums and applications (it gains in performance with other alternatives) since it is very file specific and it gains on the "new" factor of P2P content, more users equals more speed, but it will not be the optimum solution to rare files or to distribute content that is not highly sought over.

To download files that are hosted using BitTorrent users must have a BitTorrent client and to publish a file one must run a tracker.

In November 2004, BitTorrent accounted for an astounding 35 percent of all the traffic on the Internet and in 2006 the BitTorrent protocol has risen to over 60 percent of all Internet traffic according to British Web analysis firm CacheLogic. Due to this some ISPs are doing traffic shaping also known as bandwidth throttling, meaning they are reducing the protocol priority inside their networks and reducing its overall performance this has resulted in two kind of responses, some ISPs are investing in upgrading their networks and provide local cache to the protocol and implementors of the protocol are starting to battle ISPs that refuse to adapt by encrypting and randomizing it, this kick need to adapt and the increasing popularity due to deviation from its creators vision is placing more and more its evolution on the hands of independent developers.

The BitTorrent system is highly dependent on active participation of peers since it only goal is the sharing of files. Rare and "old" content is not easy to find on the system, only highly sought after content benefit from this P2P implementation. Small files also don't fully benefit from it, since the needed time for replication is too short and in some extreme situations can even degrade the experience.

Content indexers

There are many different BitTorrent websites that index content, each providing information about files distributed via the BitTorrent protocol. They typically contain multiple torrent files and an index of those files. In a typical scenario, a user would enter such a site and browse or search for the content they desire, based on the torrent descriptions posted at the site by other users. If a torrent with the sought content is found, the user could download that torrent.

Legal Torrents ( http://www.legaltorrents.com ), a collection of Creative Commons-licensed, legally downloadable, freely distributable creator-approved files, from electronic/indie music to movies and books, which have been made available via BitTorrent. Everyone that grabs the BitTorrent client and downloads helps contribute more bandwidth.

There is also OpenBitTorrent ( http://openbittorrent.com ), OpenBitTorrent.kg ( http://www.openbittorrent.kg ), myTorrentTracker ( http://www.mytorrenttracker.com ) and trackhub ( http://trackhub.appspot.com ), all BitTorrent trackers free for anyone to use to share files. You don't need to register, upload or index a torrent anywhere, all you have to do is to include the tracker URL in the torrent file.

Other:

• bt.etree.org ( http://bt.etree.org/ ), a site provided by the etree.org community for sharing the live concert recordings of trade friendly artists.
• The Pirate Bay ( http://thepiratebay.org ).
• mininova ( http://www.mininova.org/ ).

Such websites each have different features to facilitate the user's search. See Wikipedia's Comparison of BitTorrent sites page. Several of the larger BitTorrent tracker sites were shut down citing concerns about problems with copyright holders, mostly representatives of large business interests. While in short it does prevent large scale copyright infringement, it also creates difficulty to legal uses and there is the issue with false notifications, that is claiming infringement of rights over works they do not own. In the long run this does little to solve the problem and pressures the protocol to evolve in ways to avoid this type of disruption. One way people have adapted to this pressure is to create private trackers that are only available by invitation.

Protocol

As already discussed the BitTorrent is a protocol for distributing files. It identifies content by URL and is designed to integrate seamlessly with the web. Its advantage over plain HTTP is that when multiple downloads of the same file happen concurrently, where each downloaders upload to each other, making it possible for the file source to support very large numbers of downloaders with only a modest increase in its load. ( http://www.bittorrent.com/protocol.html ). BitTorrent share some of the nomenclature of other P2P protocols but also creates new ones (see Wikipedia's page BitTorrent vocabulary for an extended list).

Torrent

A torrent can mean either a .torrent metadata file or all files described by it, depending on context. The torrent file, as defined in the BitTorrent specification, contains the URLs of multiple trackers, that coordinates communication between the peers in the swarm, and integrity metadata about all the files it makes down-loadable, including their names and sizes and checksums of all pieces in the torrent. It can also contain additional metadata defined in extensions to the BitTorrent specification, known as BitTorrent Enhancement Proposals. Examples of such proposals include metadata for stating who created the torrent, and when.

Index

An index is a list of .torrent files (usually including descriptions and other information) managed by a website and available for searches. An index website can also be sometimes refereed as a tracker, but as a "Torrent" tracked not BitTorrent tracker).

The protocol was partially dependent of centralized in do to the requirement of trackers.

Client

The program that enables p2p file sharing via the BitTorrent protocol. It denotes still the semi-centralized nature of the protocol, on the protocol definition sometimes the terms is substituted by Peer (ie: peer_id), for instance Gnutella refers to participants always as Peers or Nodes and to implementer as Vendor.

Tracker

A tracker is a server that keeps track of which seeds and peers are in the swarm. Clients report information to the tracker periodically and in exchange receive information about other clients to which they can connect. The tracker is not directly involved in the data transfer and does not have a copy of the file.

Scrape

This is when a client sends a request to the tracking server for information about the statistics of the torrent, such as with whom to share the file and how well those other users are sharing.

With the adoption of DHT (Distributed Hash Tables) the BitTorrent protocol starts to become more that a semi-centralized distribution network around a single resource, it becomes more decentralized and removes the static point of control, the tracker, this is done by relying in DHTs and the use of the PEX extension. Enabling the volatile Peer to operate also as a tracker, but even if this addressed the need for static tracker servers, there is still a centralization of the network around the content. Peers don't have any default ability to contact each other outside of that context.

Seeder

A seeder is a client that has a complete copy of the torrent and still offers it for upload. The more seeders there are, the better the chances of getting a higher download speed. If the seeder seeds the whole copy of the download they should get faster downloads

Seeding rules, like we will see with the special case of super-seeding, are variables and algorithms implemented locally by the client in a general configuration often open in some form to user control. These rules control and may serve to optimize the selection of what available torrent is seeded, instead of just starting the next one in the list, and sort torrents based on a Seeding Rank.

Seeding Rank

Is a priority rating, resulting from the calculations based on the active seeding rules of the client, serving to prioritizes your torrents based on how needy they are. It generates a priority queue, where the available torrents are given use of the available open slots for transfer. Several consideration can contribute to the Seeding Rank:

• Seed Ratio. The lower the ratio, the more scarce a the torrent is, the higher its Seeding Rank should be, giving priority to rare torrents.
• Seed Count. Similar to Seed ratio but considers not only complete seeds but the number of any client interested in the torrent, works in reverse, giving preference to larger swarms and torrents in high demand.
• Timed Rotation. Torrents will be rotated in and out of seeding mode. Each torrent is given a length of time they remain seeding.
• Default. Each torrents will be seeded based on their order they are added to the seed list.

Announce

Similar to "Scrape", but means that the client also announces that it wants to join the swarm and that the server should add it to the list of peers in that swarm.

Availability (also known as Distributed copies.)

This is a common word used on distributed systems, in this case it refers to the number of full copies of the file available to the client. Each seed adds 1.0 to this number, as they have one complete copy of the file. A connected peer with a fraction of the file available adds that fraction to the availability, if no other peer has this part of the file.

Example: a peer with 65.3% of the file downloaded increases the availability by 0.653. However, if two peers both have the same portion of the file downloaded - say 50% - and there is only one seeder, the availability is 1.5.

Interested

Describes a downloader who wishes to obtain pieces of a file the client has. For example, the uploading client would flag a downloading client as 'interested' if that client did not possess a piece that it did, and wished to obtain it.

Downloader

A downloader is any peer that does not have the entire file and is downloading the file. This term, used in Bram Cohen's Python implementation, lacks the negative connotation attributed to leech. Bram chose the term downloader over leech because BitTorrent's tit-for-tat ensures downloaders also upload and thus do not unfairly qualify as leeches.

Choked

Describes a client that has been refused file pieces. A client chokes another client in several situations:

• The second client is a seed, in which case it does not want any pieces (i.e., it is completely uninterested)
• The client is already uploading at its full capacity (it has reached the value of max_uploads)
• The second client has been blacklisted for being abusive or is using a blacklisted BitTorrent client.

Snubbed

An uploading client is flagged as snubbed if the downloading client has not received any data from it in over 60 seconds.

Extension protocol for BitTorrent

Created by Arvid Norberg and Ludvig Strigeus, (description http://www.rasterbar.com/products/libtorrent/extension_protocol.html ), it is an extension to the protocol that intends to provide a simple and thin transport for future extensions to the BitTorrent protocol. This protocol makes it easy to add new extensions without interfering with the standard bittorrent protocol or clients that don't support this extension.

The extension messages IDs are defined in the handshake is to avoid having a global registry of message IDs. Instead the names of the extension messages requires unique names, which is much easier to do without a global registry.

There seems also to be a concurrent implementation, or variation, by Vuze ( http://wiki.vuze.com/w/Azureus_messaging_protocol ) that is used both by Vuse and Transmission.

Peer exchange

Peer exchange or PEX is a communications protocol that augments the BitTorrent file sharing protocol. It allows a group of users (or peers) that are collaborating to share a given file to do so more swiftly and efficiently. PEX is implemented using one of two common extension protocols.

In the original design of the BitTorrent file sharing protocol clients, that users (Peers) in a file sharing group (known as a "swarm") relied upon a central computer server called a tracker to find each other and to maintain the swarm. PEX greatly reduces the reliance of peers on a tracker by allowing each peer to directly update others in the swarm as to which peers are currently in the swarm. By reducing dependency on a centralized tracker, PEX increases the speed, efficiency, and robustness of the BitTorrent protocol making it more decentralized.

As already explained, users wishing to obtain a copy of a file typically first download a .torrent file that describes the file(s) to be shared, as well as the URLs of one or more central computers called trackers that maintain a list of peers currently sharing the file(s) described in the .torrent file. In the original BitTorrent design, peers then depended on this central tracker to find each other and maintain the swarm. Later development of distributed hash tables (DHTs) meant that partial lists of peers could be held by other computers in the swarm and the load on the central tracker computer could be reduced. PEX allows peers in a swarm to exchange information about the swarm directly without asking (polling) a tracker computer or a DHT. By doing so, PEX leverages the knowledge of peers that a user is connected to by asking them for the addresses of peers that they are connected to. This is faster and more efficient than relying solely on a tracker and reduces the processing load on the tracker. It also keep swarms together when the tracker is down. In fact removing any control over the distribution once a peer keeps a complete copy of the file share.

Peer exchange cannot be used on its own to introduce a new peer to a swarm. To make initial contact with a swarm, each peer must either connect to a tracker using a ".torrent" file, or else use a router computer called a bootstrap node to find a distributed hash table (DHT) which describes a swarm's list of peers. For most BitTorrent users, DHT and PEX will start to work automatically after the user launches a BitTorrent client and opens a .torrent file. A notable exception is "private torrents" which are not freely available; these will disable DHT.

It was agreed between the Azureus and µTorrent developers that any clients which implement either of the mechanisms above try to obey the following limits when sending PEX messages:

• There should be no more than 50 added peers and 50 removed peers sent in any given PEX message.
• A peer exchange message should not be sent more frequently than once a minute.

Some clients may choose to enforce these limits and drop connections from clients that ignore these limits.

Permanent DHT tracking

With the PEX implementation and reliance on the distributed hash table (DHT), the evolution into creating a real P2P overlay network that is completely serverless was the next logical step, much like the eDonkey network has evolved as we have seen. The DHT works mostly the same way and will take information not only from old trackers by also from the PEX implementation, creating something like a distributed Database of shared torrents acting as backup tracker when all other trackers are down or can't deliver enough peers, as well as enabling trackerless torrents. The DHT acts and is added to torrents as a pseudo-tracker if the client has the option enabled and DHT trackers can be enabled and disabled per torrent just like regular trackers. Clients using this permanent DHT tracking are now a fully connected decentralized P2P network, they enter the DHT as a new node, this of course makes it necessary for private trackers (or non-public distributions) to exclude themselves from the participating.

Bootstrapping the DHT

Since the DHT is independent of any single tracker (and point of failure), the issue of how the DHT routing table is bootstrapped, the first time using DHT, has to be addressed. This is done in several ways:

1. Manually entering a host name and port number of a DHT node.
2. Connect to a tracker that has a .torrent file with a list of DHT nodes.
3. Downloading any torrent with peers who advertise that they support DHT. Not fully supported by all clients as it requires clients to advertise in the BitTorrent handshake DHT support.

Magnet links

Traditionally, .torrent files are downloaded from torrent sites. But several clients also support the Magnet URI scheme. A magnet link can provide not only the torrent hash needed to seek the needed nodes sharing the file in the DHT but may include a tracker for the file.

BitTorrent Enhancement Proposal (BEP)

The BitTorrent Enhancement Proposal Process (BEP) is a process started by John Hoffman. The process is defined in the public domain document ( http://www.bittorrent.org/beps/bep_0001.html ).

List of BitTorrent Enhancement Proposals is available ( http://www.bittorrent.org/beps/bep_0000.html ).

A BEP is a design document providing information to the BitTorrent community, or describing a new feature for the BitTorrent protocols. BEPs should provide a concise technical specification of the feature and a rationale for the feature, and are intended to be the primary mechanisms for proposing new features, for collecting community input on an issue, and for documenting the design decisions that have gone into BitTorrent.

The BEP author is responsible for building consensus within the community and documenting dissenting opinions. Because the BEPs are maintained as re-structured text files in a versioned repository, their revision history is the historical record of the feature proposal.

Super-seeding

Super-seeding specified in BEP 16 ( http://www.bittorrent.org/beps/bep_0016.html ) is a extension to the BitTorrent protocol (implemented without changes to the protocol), intended to be used when there is only one seed, it permits to manage the scarcity of a resource.

In a situation when a seeder detects that it is the only source for a file, it will attempt to minimize the amount of data uploaded, as to guarantee external sharing and optimize access to the scarce resource, until it detects that other complete seeders exist. The feature was conceived by John Hoffman and first implemented in the BitTornado client in 2003.

uTP

uTP or µTP (sometimes also referred as Micro Transport Protocol ) is an open source cross-platform protocol, created to be implemented on top of UDP protocol a TCP-like implementation of LEDBAT (a TCP congestion avoidance algorithm).

µTP was developed within BitTorrent, Inc. with no input from either the networking or BitTorrent communities as to be provide reliable, ordered delivery while maintaining minimum extra delay as too automatically slow down the rate at which packets of data are transmitted between users of peer-to-peer file sharing torrents when it interferes with other applications. For example, the protocol should automatically allow the sharing of an ADSL line between a BitTorrent application and a web browser. It was first introduced in the µTorrent 1.8.x beta branches, and publicized in the alpha builds of µTorrent 1.9. as is now the primary transport protocol for uTorrent peer-to-peer connections.

uTP is documented as a BitTorrent extension, in BEP 29 ( http://bittorrent.org/beps/bep_0029.html ). BitTorrent, Inc. has made a C++ implementation available under the MIT license ( http://github.com/bittorrent/libutp ), but the external interface is strictly C (ANSI C89).

BitTorrent protocol encryption

As of January 2005, BitTorrent traffic made up more than a third of total residential Internet traffic. Some ISPs decided to take different measures control and event to subvert P2P traffic, as covered in Shadow play section of this book.

This created a need for providing a BitTorrent protocol encryption. Obfuscation and encryption makes traffic harder to detect and monitor and therefore harder to throttle. BitTorrent protocol encryption is not designed to provide anonymity or confidentiality, even if some solutions will it increase confidentiality by obfuscating the content.

Bram Cohen, the inventor of BitTorrent, opposed adding encryption to the BitTorrent protocol. Cohen stated he was worried that encryption could create incompatibility between clients, stressing the point that the majority of ISPs don't block the torrent protocol. Cohen wrote "I rather suspect that some developer has gotten rate limited by his ISP, and is more interested in trying to hack around his ISP's limitations than in the performance of the Internet as a whole". After some criticism for this position, Cohen later added the ability to receive but not originate encrypted connections on his Mainline client. Notably, when µTorrent was purchased by BitTorrent, Inc. and then became the next mainline release, the ability to originate encrypted connections was retained, but it became turned off by default.

Encryption will not stop a traffic shaping system configured to universally slow down all encrypted, unidentifiable or unknown protocols using a method as simple as packet loss. Encrypting tracker communications prevents eavesdropping on peer lists and does not require upgrading both ends of peer-to-peer connections, but it requires imposing computational overhead on the tracker.

Protocol header encryption (PHE)

Created by RnySmile and first implemented in BitComet version 0.60 on 8 September 2005. The specifications was neither published, nor is it compatible with MSE/PE, and there are claims that it was already reverse engineered, reducing its usefulness.

Message stream encryption (MSE)/Protocol encryption (PE)

Developed by Azureus (now Vuze) in late January 2006, that later suffered several alterations permitting a broader acceptance by the creators of other BitTorrent clients.

As per the specifications ( http://wiki.vuze.com/w/Message_Stream_Encryption ), MSE/PE uses key exchange combined with the infohash of the torrent to establish an RC4 encryption key. The key exchange helps to minimize the risk of passive listeners, and the infohash helps avoid man-in-the-middle attacks. RC4 is chosen for its speed. The first kilobyte of the RC4 output is discarded to prevent a particular attack.

The specification allows for the users to choose between encrypting the headers only or the full connection. Encrypting the full connection provides more obfuscation but uses more CPU time. To ensure compatibility with other clients that don't support this specification, users may also choose whether unencrypted incoming or outgoing connections are still allowed. Supported clients propagate the fact that they have MSE/PE enabled through PEX and DHT.

Analysis of this method has shown that statistical measurements of packet sizes and packet directions of the first 100 packets in a TCP session can be used to identify the obfuscated protocol with over 96% accuracy, this makes this solution only effective against the efforts of ISPs that don't adopt state of the art traffic analysis, mostly smaller ISPs.

Various solutions exist to protect the BitTorrent network against attacks including encrypting both peer-to-tracker and peer-to-peer communication, using Microsoft's Teredo so that TCP connections are tunneled within UDP packets, filtering TCP resets before they reach the TCP layer in the end-host, or switching entirely from a TCP-based transport to a UDP-based transport. Each solution has its trade-offs. Filtering out attack TCP resets typically requires kernel access, and the participation of the remote peer since the attacker has to send the reset packet to the local and remote peers. Teredo is not available on all BitTorrent clients. Rewriting TCP reliability, in-order delivery and congestion control in a new UDP protocol represents a substantial engineering effort and would require upgrading both ends of any peer-to-peer connection.

Software Implementations

Wikipedia provides an some relevant information in articles like comparison of BitTorrent software and usage share of BitTorrent clients. The following list is given as to provide a general idea and comparison regarding the implementation details in relation to other P2P protocols.
(This should not to be considered a complete list of BitTorrent clients, no special order was used. All links have been verified, special attention has been given to the programming language and licenses of the software. Last update 11 September 2010)

• BitTorrent Queue Manager ( http://btqueue.sourceforge.net ), a console-based BitTorrent Client with built-in scheduler for handling multiple sessions. It is designed to manage sessions in queue easily without heavy-weight GUI. External module can search for new torrents in trackers and submit it automatically. OpenSource (Python Software Foundation License) project, using Python.
• Vuze previously called Azureus ( http://azureus.sourceforge.net or http://www.vuze.com/ ), an open source BitTorrent client in Java, probably the more advanced peer for the network (multiple torrent downloads, queuing/priority systems, start/stop seeding options, embedded tracker, Mainline DHT and a lot more) but a known resource hog, consuming large quantities of memory and CPU power.
• µTorrent ( http://utorrent.com ), a closed source, freeware BitTorrent client in C++, a very complete peer (includes bandwidth prioritization, scheduling, RSS auto-downloading and Mainline DHT and more) with a very low system footprint.
• BitTornado ( http://bittornado.com ), an open source BitTorrent client in Python based on the original BitTorrent client.
• BitComet ( http://www.bitcomet.com ), (originally named SimpleBT client from versions 0.11 to 0.37) is a closed source but freeware, BitTorrent client for the MS Windows OS only, it also supports HTTP/FTP download management.
• ABC (Yet Another BitTorrent Client) ( http://pingpong-abc.sourceforge.net ), an open source BitTorrent client, based on BitTornado.
• Transmission ( http://transmission.m0k.org/ ), an open source lightweight BitTorrent client with a simple graphic user interface on top of a cross-platform back-end. Transmission runs on Mac OS X with a Cocoa interface, Linux/NetBSD/FreeBSD/OpenBSD with a GTK+ interface, and BeOS with a native interface. Released under the MIT/X Consortium License.
• Warez ( http://www.warezclient.com ), a closed source, MS Windows only BitTorrent client from Neoteric Ltd. (previously supporting the Ares Network Warez P2P client).
• Bits on Wheels ( http://bitsonwheels.com ), a freeware but closed source, implementation, written in Objective-C and Cocoa for the Macintosh.
• Vidora ( http://www.videora.com/ ), a closed source, freeware implementation that also support Really Simple Syndication (RSS) feeds.
• sharktorrent ( http://sharktorrent.sourceforge.net/ ), an open source (GNU GPL) written in C++. This a cross-platform BitTorrent client uses QT , libtorrent and boost libraries.
• ted [Torrent Episode Downloader] ( http://www.ted.nu/ ), an open source (GNU GPL) BitTorrent client coded in Java, it also support torrent RSS feeds.
• rTorrent| ( http://libtorrent.rakshasa.no ) is a text-based ncurses BitTorrent client written in C++, based on the libTorrent libraries for Unix (not to be confused with libtorrent by Arvid Norberg/Rasterbar), whose author's goal is “a focus on high performance and good code. Both the client as the library are available under the GNU GPL.
• libtorrent ( http://www.rasterbar.com/products/libtorrent/ ) from Rasterbar Software, an open source C++ library, implementing the BitTorrent protocol and core necessities for an application, using zlib and Boost libraries, specifically Boost.Asio and shares the Boost license. This library is also commonly used embedded in devices. The library also provides support for UPnP configuration. libtorrent is used by the following implementations:
  • Halite ( http://www.binarynotions.com/halite-bittorrent-client ), an open-source, under the Boost Software License, this BitTorrent client uses the libtorrent library. Coded in C++ using the Boost library and WTL (Windows only).
  • FireTorrent ( https://addons.mozilla.org/en-US/firefox/addon/10931 ) by Pete Collins, Radical Software Ltd, Jan Varga, Matthew Gertner, open source in JavaScript using the libtorrent library, Mozilla Public License FireFox extension/add-on to download torrents.
  • Folx ( http://www.mac-downloader.com/ ), closed source, using libtorrent library (Mac only).
  • qBittorrent ( http://www.qbittorrent.org/ ), open source GNU GPL, developed by a Ph.D student (Christophe Dumez), a Bittorrent client using C++ / libtorrent and a Qt4 Graphical User Interface.
  • Deluge ( http://deluge-torrent.org ), an open source, using Python and libtorrent, lightweight, cross-platform BitTorrent client in Python released under the GNU GPL license.
  • Limewire, already covered as a well known Gnutella implementation also support the BitTorrent protocol by using the libtorrent library.
  • BTG ( http://btg.berlios.de ), Bittorrent client implemented in C++ and using the Rasterbar Libtorrent library, released under the GNU GPL. Provides a Ncurses, SDL, Gtkmm and WWW GUI, which communicate with a common backend running the actual BitTorrent operation, available only for OSX, BSD and Linux.
  • Free Download Manager (FDM), ( http://www.freedownloadmanager.org ), C++ open source software distributed under GNU GPL using the libtorrent library (Windows only).
  • torrent2exe.com, a web tool, that reportedly converts .torrents into executables (Windows) for distribution, closed source using the libtorrent library.
  • Flush ( http://sourceforge.net/projects/flush ) GTK-based BitTorrent client for Linux, open source C++/GTK+ using the libtorrent library.
  • Pump ( http://www.vipeers.com ), a closed source video manager that supports the BitTorrent protocol by using the libtorrent library.
  • Lince ( http://lincetorrent.sourceforge.net ), open source C++/GTK+/libtorrent BitTorrent client, release under the GNU GPL (Linux/BSD/UNIX-like OSes).
  • Miro, previously known as Democracy Player and DTV ( http://getmiro.com ) is an designed to automatically download videos from RSS-based “channels”, manage them and play them. Open source in Python/GTK/libtorrent, released under the terms of the GNU General Public License.
  • tvitty ( http://tvitty.com ) closed source BitTorrent download add-in for vista media center using libtorrent (Windows only).
  • FatRat ( http://fatrat.dolezel.info ), is an open source download manager for Linux written in C++ using Qt4 and libtorrent libraries.
  • LeechCraft ( http://leechcraft.org ), open source BitTorrent client (supporting also HTTP/FTP downloads), create using C++, Qt and libtorrent. Released under the GNU General Public License.
  • MooPolice ( http://www.moopolice.de ), BitTorrent client for Windows, having an unorthodox GUI. Open source (without a specific license) C++ using MFC and libtorrent BitTorrent client library and MiniUPnP.
  • Linkage ( http://code.google.com/p/linkage ), a lightweight BitTorrent client written in C++ using gtkmm and libtorrent, open source under the GNU General Public License (no longer maintained).
  • Arctic Torrent ( http://int64.org/projects/arctic-torrent ), a small BitTorrent client for Windows (includes a 64bits version). Open Source C++ under the MIT License, using libtorrent.

Specific BitTorrent Papers

• May 22, 2003 - Incentives Build Robustness in BitTorrent (PDF), Bram Cohen
  The BitTorrent file distribution system uses tit-for-tat as a method of seeking pareto efficiency. It achieves a higher level of robustness and resource utilization than any currently known cooperative technique. We explain what BitTorrent does, and how economic methods are used to achieve that goal.

Other Software Implementations

JXTA

JXTA™ technology, created by Sun™ ( http://www.jxta.org ), is a set of open protocols that allow any connected device on the network ranging from cell phones and wireless PDAs to PCs and servers to communicate and collaborate in a P2P manner. JXTA peers create a virtual network where any peer may interact with other and their resources directly even when some of the peers and resources are behind firewalls and NATs or are on different network transports. The project goals are interoperability across different peer-to-peer systems and communities, platform independence, multiple/diverse languages, systems, and networks, and ubiquity: every device with a digital heartbeat. The technology is licensed using the Apache Software License (similar to the BSD license).

Most of the implementation is done in Java (with some minor examples in C).

iFolder

iFolder ( http://www.ifolder.com ) is an still in early development open source application, developed by Novell, Inc., intended to allow cross-platform file sharing across computer networks by using the Mono/.Net framework.

iFolder operates on the concept of shared folders, where a folder is marked as shared and the contents of the folder are then synchronized to other computers over a network, either directly between computers in a peer-to-peer fashion or through a server. This is intended to allow a single user to synchronize their files between different computers (for example between a work computer and a home computer) or share files with other users (for example a group of people who are collaborating on a project).

The core of the iFolder is actually a project called Simias. It is Simias which actually monitors files for changes, synchronizes these changes and controls the access permissions on folders. The actual iFolder clients (including a graphical desktop client and a web client) are developed as separate programs that communicate with the Simias back-end.

The iFolder client runs in two operating modes, enterprise sharing (with a server) and workgroup sharing (peer-to-peer, or without a server).

Freenet

The Freenet Project ( http://freenetproject.org ), designed to allow the free exchange of information over the Internet without fear of censorship, or reprisal. To achieve this Freenet makes it very difficult for adversaries to reveal the identity, either of the person publishing, or downloading content. The Freenet project started in 1999, released Freenet 0.1 in March 2000, and has been under active development ever since.

Freenet is unique in that it handles the storage of content, meaning that if necessary users can upload content to Freenet and then disconnect. We've discovered that this is a key requirement for many Freenet users. Once uploaded, content is mirrored and moved around the Freenet network, making it very difficult to trace, or to destroy. Content will remain in Freenet for as long as people are retrieving it, although Freenet makes no guarantee that content will be stored indefinitely.

The journey towards Freenet 0.7 began in 2005 with the realization that some of Freenet's most vulnerable users needed to hide the fact that they were using Freenet, not just what they were doing with it. The result of this realization was a ground-up redesign and rewrite of Freenet, adding a "darknet" capability, allowing users to limit who their Freenet software would communicate with to trusted friends. This would make it far more difficult for a third-party to determine who is using Freenet.

Freenet 0.7 also embodies significant improvements to almost every other aspect of Freenet, including efficiency, security, and usability. Freenet is available for Windows, Linux, and OSX. It can be downloaded from:

Software Implementations

All software is available on The Freenet Project page.

Frost an application for Freenet that provides usenet-like message boards and file uploading/downloading/sharing functionalities. It should get installed with Freenet 0.7 automatically if you used the standard Freenet installers.

jSite is a graphical application that you can use to create, insert and manage your own Freenet sites. It was written in Java by Bombe.

Thaw is a filesharing utility and upload/download manager. It is used as a graphical interface for Freenet filesharing.

KaZaa

KaZaa ( http://www.kazaa.com )

Software (FastTrack) Implementations

• Kazaa
• Kazaa Lite
• Diet Kaza
• giFT
• Grokster
• iMesh

GNUnet

GNUnet ( http://gnunet.org/ ), was started in late 2001, as a framework for secure peer-to-peer networking that does not use any centralized or otherwise trusted services. The framework provides a transport abstraction layer and can currently encapsulate the network traffic in UDP (IPv4 and IPv6), TCP (IPv4 and IPv6), HTTP, or SMTP messages. All peer-to-peer messages in the network are confidential and authenticated.

The primary service build on top of the framework is anonymous file sharing, implemented on top of the networking layer allows anonymous censorship-resistant file-sharing. GNUnet uses a simple, excess-based economic model to allocate resources. Peers in GNUnet monitor each others behavior with respect to resource usage; peers that contribute to the network are rewarded with better service.

GNUnet is part of the GNU project. Our official GNU website can be found at ( http://www.gnu.org/software/gnunet/ ), there is only an existing client, OpenSource, GPL, written in C, that shares the same name as the network. GNUnet can be downloaded from this site or the GNU mirrors.

MANOLITO (MP2P)

MANOLITO or MP2P is the internal protocol name for the proprietary peer-to-peer file sharing network developed by Pablo Soto. MANOLITO uses UDP connections on port 41170 for search routing and is based on Gnutella. In addition file transfers use a proprietary protocol based on TCP.

MANOLITO hosts obtain an entry into the network by contacting an HTTP network gateway, which returns a list of approximately one-hundred MANOLITO hosts. Hosts can also be manually connected to. Servents maintain contact with a fixed number of peers (depending on the Internet connection) that are sent search queries and results.

Software Implementations

• Blubster ( http://www.blubster.com ), first implementation of the MP2P protocol, closed source but freeware for Windows.
• Piolet ( http://www.piolet.com ), closed source but freeware for Windows.

Mute File Sharing

MUTE File Sharing ( http://mute-net.sourceforge.net ) is an anonymous, decentralized search-and-download file sharing system. MUTE uses algorithms inspired by ant behavior to route all messages, include file transfers, through a mesh network of neighbor connections.
Author Jason Rohrer - jcr13 (at) cornell (dot) edu Created using C++ and Crypto++ Library, support is provided for multiple OSs there is a frontend for Windows created with MFC, Mute is Open Source and released under the GPL License.

iMesh

iMesh ( http://www.imesh.com ), a free but closed source P2P network (IM2Net) operating on ports 80, 443 and 1863, for Widows. iMesh is owned by an American company iMesh, Inc. and maintains a development center in Israel. An agreement with the MPAA had also been reached. Video files more than 50mb in size and 15 minutes in length can no longer be shared on the iMesh network, guaranteeing feature-length releases cannot be transferred across the network.

BitCoop

BitCoop (http://bitcoop.sourceforge.net/) created by Philippe Marchesseault is a console (Text Based) peer to peer backup system that enables the storage of files on remote computers with cryto and compression support. The size of files depends on the quantity you wish to share with the other peers. It is intended for server farms that wish to backup data among themselves. Supports various Operating Systems including Windows, Linux and Mac OS X, it's implemented in Java (Open Source under the GPL).

CSpace

CSpace (http://cspace.in/) provides a platform for secure, decentralized, user-to-user communication over the Internet. The driving idea behind the CSpace platform is to provide a connect(user,service) primitive, similar to the sockets API connect(ip,port). Applications built on top of CSpace can simply invoke connect(user,service) to establish a connection. The CSpace platform will take care of locating the user and creating a secure, nat/firewall friendly connection. Thus the application developers are relieved of the burden of connection establishment, and can focus on the application-level logic! CSpace is developed in Python. It uses OpenSSL for crypto, and Qt for the GUI. CSpace is licensed under the GPL.

I2P

I2P is a generic anonymous and secure peer to peer communication layer. It is a network that sits on top of another network (in this case, it sits on top of the Internet). It is responsible for delivering a message anonymously and securely to another location.

p300

p300 (http://p300.eu/) is a P2P application created in Java with the intention of provide a just-works-single-download solution for a multitude of Operating Systems without the need to deal with user accounts or specific protocol and security configurations (ie. samba). Another aspect is that p300 is primarily meant to be used in LANs or over VPNs. This application is OpenSource released under the GNU GPL v3.

Netsukuku

Netsukuku (http://netsukuku.freaknet.org/) is a p2p (in mesh) network system, originally developed by FreakNet MediaLab, that can generate and sustain itself autonomously. It is designed to handle an unlimited number of nodes with minimal CPU and memory resources. It seems that it can be easily used to build a worldwide distributed, anonymous decentralized network, above the Internet, without the support of any servers, ISPs or authority controls. Netsukuku replaces the network level 3 of the OSI model with another routing protocol. An open source Python implementation was finished in October 2007.

Netsukuku is based on a very simple idea: extending the concept of Wi-Fi mesh networks to a global scale, although not necessarily using that medium. With the use of specialized routing protocols and algorithms, the current Wi-Fi technologies can be exploited to allow the formation of a global P2P wireless network, where every peer (node) is connected to its neighbors.

Other media will be equally functional to interconnect nodes, as the interaction is independent of that which it is transmitted through, but it is believed that Wi-Fi will be the most practical for ordinary users to take advantage of. Once greater proliferation has been achieved, it may become common to see some nodes establishing high-speed land-line connections between each other in the interest of increasing network bandwidth for connections over it and lowering latencies.

Adobe's RTMFP (Real Time Media Flow Protocol) Groups

The RTMFP is a closed protocol/implementation based on the creation of Amicima, a start-up founded in 2004 focused in the development of improved Internet protocols for client-server and peer-to-peer networking and derived applications ( p2p-hackers - amicima's MFP - preannouncement, Jul 2005, MFP - The Secure Media Flow Protocol - version 1), that was acquired by Adobe for inclusion into the Flash platform, that gives developers the ability to stream data to endpoints without going through a central server (Flash Media Server). This addition to the Flash player v10.1+ capabilities enables most P2P network needs to be performed on Flash. No much information is available on the implementation yet. The presentation is available in a flash video ( http://tv.adobe.com/watch/max-2009-develop/p2p-on-the-flash-platform-with-rtmfp ).

Other

• Thunderbolt (aka Thunder) (http://www.xunlei.com/) was created by Xunlei Network Technology Ltd. Thunderbolt proprietary P2P network supports Multi-protocol P2P resources (supports BitTorrent, eDonkey, Kad, and FTP) and also HTTP downloads (a download accelerator) as it web caches to aid in accelerating of downloads. It is mainly used in the Mainland China, recently an English translation has been released. Of particularly interest is that on January 5, 2007, Google acquired a 4% stake on the company.

• Cassandra (http://code.google.com/p/the-cassandra-project/) a Structured Storage System on a P2P Network for managing structured data while providing reliability at a massive scale. Made in Java under the Apache License 2.0.

• Aimini (http://www.aimini.com/), windows freeware.

• TinyP2P ( http://www.freedom-to-tinker.com/tinyp2p.html ), claims to be The World's Smallest P2P Application, written in fifteen lines of code, in the Python programming language.

• Secure Content Downloader, closed source, Windows and Microsoft specific P2P application.

• XNap ( http://xnap.sourceforge.net/ ) OpenSource (GPL), written in Java. The client features a modern Swing based user interface and console support. Able to work in several P2P Networks OpenNap, Gnutella, Overnet and OpenFT (and other networks supported by giFT like FastTrack). It also supports ICQ and IRC, viewers for MP3 tags, images, PDF, ZIP files and Text-To-Speech.

• Filetopia ( http://www.filetopia.com ), a free but closed source server/clients P2P application for Windows. It includes, instant messaging, chat and file sharing system with a search engine, online friends list and message boards. It also supports the use of a bouncer (open source,Java) as an anonymity layer, that enables indirect connections.

• Carracho ( http://www.carracho.com ), freeware but closed sourced P2P application to MacOS X.

• SockeToome ( http://www.blackdiamond.co.za/bdsock.html ), shareware P2P application for Mac and Windows.

• FilePhile ( http://www.filephile.net ), closed source but freeware, Java-based and so multi-platform application.

• Napster network
  • WinMX
  • Napigator
  • FileNavigator

• WPNP network
  • WinMX

• other networks
  • InterPlanetary File System- protocol and network designed to create a content-addressable, peer-to-peer method of storing and sharing hypermedia in a distributed file system. Initially designed by Juan Benet, and now open-sourced.
  • MojoNation
  • Carracho
  • Hotwire
  • Chord peer-to-peer lookup service|Chord
  • Dexter
  • Swarmcast
  • Alpine program|Alpine
  • Scribe
  • Groove
  • Squid_Soft|Squid
  • Akamai
  • Evernet
  • Overnet network
  • Audiogalaxy network
  • SongSpy network
  • The Circle
  • OpenFT

• Acquisition
• Cabos
• Swapper
• SoulSeek