BitTorrent

• 1. Introduction

  ›

  BitTorrent arrived in the summer of 2001 as a small text file and a radical idea: what if the people downloading a file also became the people delivering it? Bram Cohen, a programmer and University at Buffalo alumnus, released the first version on the 2nd of July 2001. Within three years, his protocol accounted for a third of all internet traffic on the planet.

  That is a remarkable number for something that most people never consciously installed. BitTorrent spread through the pipes of the early internet so thoroughly that by some measures a quarter of a billion users a month were touching it. And yet it was not a company, not a product people paid for. It was a protocol, a set of rules for how computers talk to each other when they want to share something large.

  The questions BitTorrent raises go beyond technology. How do you build something that distributes itself? How do you hold a network together when no single server is in charge? And what happens when a tool built to solve a bandwidth problem becomes the engine of one of the largest copyright disputes in the history of the internet?

• 2. The Swarm Idea

  ›

  Cohen designed the protocol in April 2001 with one specific frustration in mind: downloading large files from a single server was slow, and the more popular the file, the worse it got. His answer was to break each file into pieces and let every downloader also become an uploader of the pieces they had already received.

  In BitTorrent's model, each peer that receives a new piece immediately becomes a source of that piece for other peers. A file could theoretically be sent by the original uploader just once, and then spread to an unlimited number of recipients through the swarm. The more popular the file, the faster it moved, because more seeders meant more people serving it simultaneously. This flipped the logic of traditional servers, where popularity created bottlenecks.

  The pieces are downloaded in random order, or in a "rarest-first" approach that keeps the most scarce pieces in circulation. The client then reassembles them into the correct sequence. A download can stop at any point and resume without losing the pieces already gathered, which made the protocol especially suited to large video and audio files where an interrupted session would otherwise mean starting over.

  Each piece is protected by a cryptographic hash contained in the torrent descriptor. Any corruption, accidental or deliberate, is caught when a piece is checked against that hash. If a node starts with an authentic copy of the torrent descriptor, it can verify every piece it receives all the way to the complete file.

• 3. Building the Infrastructure

  ›

  The first BitTorrent client shipped with no search engine and no way for peers to find each other except through a central tracker. Users created a small text file, the torrent file, uploaded it to an index site, and the tracker then handed out lists of IP addresses to anyone who wanted to join the swarm.

  That dependency on a central tracker was a structural weakness. In 2005, Azureus, later renamed Vuze, released version 2.3.0.0 with a distributed hash table system that let clients find each other without a working tracker at all. The following month, BitTorrent, Inc. released its own DHT implementation, called Mainline DHT, which was incompatible with Azureus's version. Both are based on an architecture called Kademlia. By 2014, Mainline DHT had between 10 million and 25 million concurrent users at any given time, with a daily churn of at least 10 million.

  Peer exchange, added in 2006, gave clients another way to discover participants: asking already-connected peers whether they knew of others. Web seeding, also implemented in 2006, allowed clients to pull individual pieces from a standard HTTP server in addition to the swarm. A site could seed its own torrent from a regular web host, then stop serving the file once the swarm grew large enough to sustain itself.

  Cohen and Ashwin Navin founded BitTorrent, Inc. in 2004 to continue developing the technology. The company was later renamed Rainberry, Inc., and it remains the entity that maintains the protocol today.

• 4. Legal Battles and The Pirate Bay

  ›

  The Pirate Bay, formed by a Swedish group, became the most visible emblem of BitTorrent's legal entanglements. On the 31st of May 2006, Swedish police raided its servers in Sweden, acting on allegations by the MPAA of copyright infringement. The tracker was back online three days later.

  The site is notable for maintaining a public section where letters alleging copyright infringement, and replies to those letters, are displayed. In a study prepared to value NBC Universal in its merger with Comcast, analysts examined the 10,000 torrent swarms managed by PublicBT with the most active downloaders. After excluding pornographic and unidentifiable content, only one swarm offered legitimate content.

  In the United States, more than 200,000 lawsuits for copyright infringement on BitTorrent have been filed since 2010. In the United Kingdom, on the 30th of April 2012, the High Court of Justice ordered five internet service providers to block The Pirate Bay.

  The BitTorrent protocol itself is legal. The controversy has centered on trackers and index sites that link to copyrighted material without authorization, since the metafiles themselves do not store file content. The sites Suprnova.org, TorrentSpy, LokiTorrent, BTJunkie, Mininova, Oink's Pink Palace, and What.cd have all been shut down at various points as a result of legal action.

• 5. ISPs, Encryption, and Net Neutrality

  ›

  Because BitTorrent accounted for such a large share of internet traffic, some internet service providers began throttling it, slowing transfers for all users identified as running the protocol. In August 2007, Comcast went further, actively preventing BitTorrent seeding by monitoring and interfering with communications between peers.

  The response from developers was encryption. Protocol header encrypt and message stream encryption, known as MSE/PE, were built into clients to make BitTorrent traffic harder to detect. As of November 2015, a wide range of clients including Vuze, BitComet, KTorrent, Transmission, Deluge, and the official BitTorrent client supported MSE/PE encryption.

  In 2008, Comcast reached a settlement with BitTorrent, Inc., agreeing to manage traffic in a protocol-agnostic way. The episode fed directly into the debate over net neutrality in the United States, raising questions about whether an ISP could legally discriminate between types of traffic. Encryption, however, is not a complete defense; BitTorrent remains vulnerable to traffic analysis, and an ISP can still determine whether a system has stopped downloading and is only uploading, and terminate the connection by injecting TCP RST packets.

  By early 2015, AT&T estimated that BitTorrent accounted for 20% of all broadband traffic. The protocol also created hardware problems for home users: because BitTorrent frequently contacts between 20 and 30 servers per second, it can rapidly fill the network address translation tables of consumer-grade routers, causing them to stop working correctly.

• 6. Legitimate Uses

  ›

  Sub Pop Records used BitTorrent, Inc.'s platform to distribute more than a thousand albums. Nine Inch Nails distributed albums through BitTorrent frequently. The CBC became the first major broadcaster in North America to distribute a television program, Canada's Next Great Prime Minister, via BitTorrent after broadcast.

  Blizzard Entertainment previously used the protocol to distribute content and patches for Diablo III, StarCraft II, and World of Warcraft. Facebook and Twitter both use BitTorrent to push updates to their own servers. The British government used it to distribute details about how tax money was spent. Florida State University uses it to deliver large scientific data sets to researchers, and the developing Human Connectome Project uses it to share its open dataset.

  In August 2012, the Internet Archive added BitTorrent to its download options for more than 1.3 million existing files, describing it as the fastest means of downloading media from the Archive. The protocol has also been adopted for folder synchronization through Resilio Sync, which uses BitTorrent as the underlying engine for an alternative to services like Dropbox.

  GSC Game World shared files between offices in Czechia and Ukraine via BitTorrent during development of S.T.A.L.K.E.R. 2, a practice that led to game assets being leaked because the transfers were not adequately secured.

• 7. BitTorrent v2 and Forward

  ›

  In 2017, BitTorrent, Inc. released the v2 protocol specification, updating the cryptographic foundation of the entire system. The original protocol used SHA-1 to generate a hash for each piece of data. By 2017, SHA-1 was no longer considered safe from collision attacks, so v2 replaced it with SHA-256.

  The new format also introduced a Merkle tree structure, where each piece hash forms part of a hash tree rooted in the torrent's infohash. This allows more precise checks for file corruption and speeds up the transition from adding a torrent to actually beginning the download. Each file is now hashed individually, enabling deduplication across different torrents: if two separate torrents share an identical file, a downloader of one can obtain that file from peers seeding the other.

  To protect users who had not yet upgraded, the v2 specification includes a hybrid torrent format that contains both SHA-1 and SHA-256 hashes, so a single torrent file can serve both v1-only and v2-capable clients simultaneously. Magnet links for v2 carry the same hybrid approach, embedding both hash types to maintain compatibility with older software.

  The 2019 Sandvine data offered one measure of how much the landscape had shifted since 2004: BitTorrent generated 2.46% of downstream traffic and 27.58% of upstream traffic, a dramatic drop from its third-of-all-traffic peak, reflecting competition from streaming services that had built the kind of centralized, convenient infrastructure BitTorrent had deliberately set out to replace.