Colossus computer

• 1. Introduction

  ›

  Colossus was a machine built to win a war that its operators were forbidden to admit existed. Developed by British codebreakers between 1943 and 1945, it worked on one of the most closely guarded problems of the Second World War: breaking the Lorenz cipher, the encryption system used by Germany's highest military command. When the first prototype was switched on in December 1943, it became something that no one had ever built before. It was the world's first programmable, electronic, digital computer. And almost no one outside Bletchley Park would know that for thirty years.

  The questions worth sitting with are these: how did a telephone engineer from the Post Office come to build the most sophisticated computing machine of his era? What mathematical leap made the machine possible? And why, after the war was won, did the British government order the man who built it to burn every drawing, every plan, every record he had?

  341481

• 2. The Cipher That Sparked It

  ›

  In August 1941, a German operator made a catastrophic blunder. Two versions of the same message were transmitted with identical machine settings. British signals interceptors caught both, and the ripple effects of that single error would eventually produce Colossus.

  John Tiltman, a highly regarded cryptanalyst at the Government Code and Cypher School, worked the intercepts first. He extracted a keystream of almost four thousand characters. That raw material then landed with Bill Tutte, a newly arrived member of the Research Section. Working from Tiltman's keystream, Tutte deduced the entire logical structure of the German encryption machine without ever seeing it.

  Tutte worked out that the machine had twelve wheels arranged in two groups of five, which he named the chi and psi wheels, plus two further wheels he called the mu or motor wheels. The chi wheels advanced with every encrypted character; the psi wheels moved irregularly, driven by the motor wheels. This irregular motion was the heart of the cipher's strength.

  The machine, which the British called Tunny, used a technique known as Vernam ciphering. It combined plaintext characters with a stream of key characters using an XOR Boolean operation to produce ciphertext. Done correctly, this obliterated any trace of the natural frequency patterns that appear in ordinary language.

  Tutte then found the crack. By examining how characters changed from one position to the next, rather than examining the characters themselves, patterns appeared in the noise. This technique, called differencing, gave cryptanalysts a way into the system. It also gave them a precise mathematical target for any machine they might build to attack it.

  4572474456

• 3. Tommy Flowers and the Leap of Faith

  ›

  Tommy Flowers held the title of senior electrical engineer and Head of the Switching Group at the Post Office Research Station at Dollis Hill. He was not a mathematician or an academic. He was a man who understood how electronic components actually behaved under operational conditions.

  Flowers had been brought in to design the combining unit for an earlier machine called Heath Robinson, a less ambitious predecessor that proved the concept of machine-based cryptanalysis but had serious practical problems. Tapes stretched and broke at high speed. Two looped tapes had to be kept in precise synchronisation, and the mechanical components could not keep up with the demands of the work.

  Flowers knew otherwise. His pre-war research had shown him that most valve failures happened at power-up, when thermal stress was highest. If the machine was never switched off, failure rates dropped to very low levels. Slow voltage ramping at start-up reduced stress further. And valves could be soldered directly into the circuit rather than fitted with plug-in bases, eliminating a common source of unreliability.

  He persisted, and found support from W. Gordon Radley, Director of the Research Station. Flowers then spent eleven months, from early February 1943, directing a team of about fifty people in the Switching Group to build the machine. He used some of his own money to fund the project. The resulting prototype, Mark 1 Colossus, contained 1,600 thermionic valves.

  5152251640

• 4. From Dollis Hill to D-Day

  ›

  Mark 1 Colossus performed satisfactorily at Dollis Hill on the 8th of December 1943. It was then dismantled and shipped north to Bletchley Park, arriving on the 18th of January 1944 and reassembled by Harry Fensom and Don Horwood. A memo written by Max Newman on that date, now held in the National Archives, simply records: "Colossus arrives today." The machine successfully attacked its first message on the 5th of February 1944.

  While the prototype was still being built, an improved design had already been developed. The Mark 2 Colossus incorporated a six-character shift register and twelve thyratron ring stores that simulated each wheel of the Lorenz machine. Four of these improved machines were ordered in March 1944. By the end of April, the number on order had risen to twelve.

  Dollis Hill faced intense pressure to have the first Mark 2 operational by the 1st of June. Allen Coombs took over leadership of the production Colossi. The first Mark 2, containing 2,400 valves, became operational at 08:00 on the 1st of June 1944. The Allied invasion of Normandy fell on the 6th of June. The machine that arrived just in time to support that operation was five times faster than the prototype and simpler to operate.

  Afterwards, new Colossi arrived at roughly one per month. By V-E Day, ten were running at Bletchley Park and an eleventh was being assembled. Seven were dedicated to wheel setting and three to wheel breaking. The tape reader for the Mark 2 had been tested up to 9,700 characters per second during development, before the tape disintegrated. Regular operational speed was set at 5,000 characters per second, equivalent to 40 feet per second. With the five-way parallelism built into the design, effective processing speed reached 25,000 characters per second.

  4739165022

• 5. The Women Who Ran the Machines

  ›

  The Newmanry, the section responsible for machine-based cryptanalysis, was staffed by cryptanalysts, engineers who stayed permanently on hand for maintenance, and operators drawn from the Women's Royal Naval Service, known as Wrens. By the end of the war the staffing stood at 272 Wrens and 27 men.

  Every run on Colossus began with the Wrens preparing the paper tape loop. They joined the two ends of a tape using Bostik glue, leaving a 150-character blank section between the end and the start of the message. Using a hand punch, they inserted a start hole between the third and fourth channel sprocket holes from the end of the blank section, and a stop hole between the fourth and fifth channel sprocket holes from the end of the message. Specially positioned photocells read these markers to control the timing of the run.

  The Wren operators would then thread the tape through the gate, around the pulleys of what was called the bedstead, and adjust the tension. The two-tape bedstead design, carried over from Heath Robinson, allowed one tape to be loaded while the previous one was still running. Working under instruction from the cryptanalyst, the Wrens set the algorithm using the decade switches on the selection panel and the K2 panel switches, then started the tape motor. A long run, involving two chi wheels as in Tutte's foundational algorithm, took on average eight minutes. A short run involving a single chi wheel took about two minutes. Over time, decision trees were developed that allowed Wren operators to select the next algorithm to try without waiting for the cryptanalyst, in a proportion of cases.

  Howard Campaigne, a mathematician and cryptanalyst from the US Navy's OP-20-G, captured the rhythm of working with the machine in a foreword to Flowers' 1983 paper: "I told the machine to make certain calculations and counts, and after studying the results, told it to do another job."

  3450513847

• 6. Secrecy, Destruction, and Rediscovery

  ›

  After Germany surrendered, the Colossus machines were dismantled. All but two were broken into parts so small that no one examining them could infer what they had been used for. Two Colossi, along with two Tunny machines, were moved first to GCHQ's headquarters at Eastcote in April 1946, then relocated to Cheltenham between 1952 and 1954. The machine known as Colossus Blue was dismantled in 1959; the other followed in the 1960s.

  Tommy Flowers was ordered to destroy all documentation. He complied, burning his drawings and plans in a furnace. He described the experience later in words that stayed with those who heard them: "That was a terrible mistake. I was instructed to destroy all the records, which I did. I took all the drawings and the plans and all the information about Colossus on paper and put it in the boiler fire. And saw it burn."

  For thirty years, Colossus did not exist in the history of computing. Flowers and his colleagues received none of the recognition that was their due. In 1972, Herman Goldstine, who was unaware of Colossus and its legacy, wrote admiringly about Britain's post-war vitality in computing, not knowing that the source of much of that vitality was classified.

  The secrecy began to crack when Group Captain Winterbotham published his book The Ultra Secret in 1974. Professor Brian Randell, researching British computer history for a conference at the Los Alamos Scientific Laboratory in New Mexico in June 1976, obtained permission to present a paper on the wartime Colossi. The British government had released a series of captioned photographs from the Public Record Office in October 1975. The response to Randell's presentation was intense enough that a special evening meeting was convened. In 1977, Randell published his account, titled The First Electronic Computer, in several journals.

  In October 2000, GCHQ released a 500-page technical report to the Public Record Office called the General Report on Tunny. It included a passage written by the cryptographers who had worked with Colossus: "It is regretted that it is not possible to give an adequate idea of the fascination of a Colossus at work; its sheer bulk and apparent complexity; the fantastic speed of thin paper tape round the glittering pulleys."

  2441675559

• 7. Rebuilding the Machine

  ›

  Tony Sale led a team that began reconstructing a fully functional Mark 2 Colossus in 1993. The original blueprints had been burned, the hardware destroyed. What remained was scattered through engineers' notebooks and, to a considerable degree, materials that had survived in the United States.

  The optical tape reader presented the greatest potential obstacle. Dr. Arnold Lynch, who had originally designed it for the wartime machine, was still alive and was able to redesign it to his own original specification. The reconstruction was completed in 2008 and is now on display in H Block at Bletchley Park in Milton Keynes, Buckinghamshire, in the historically correct location for Colossus No. 9 at The National Museum of Computing.

  In November 2007, before the project was formally complete, a Cipher Challenge pitted the rebuilt Colossus against radio amateurs worldwide. Three messages were enciphered using a Lorenz SZ42 and transmitted from radio station DL0HNF at the Heinz Nixdorf MuseumsForum computer museum. The challenge was won by radio amateur Joachim Schuth, who had prepared carefully for the event and written his own signal processing and code-breaking software in Ada. His 1.4 GHz laptop found the settings for all twelve wheels in under a minute, processing ciphertext at 1.2 million characters per second, which was 240 times faster than Colossus. Schuth calculated that this implied an equivalent clock speed of 5.8 MHz for the 1944 machine, and described that as remarkable for a computer of that era.

  The Colossus team was slowed by their choice to use original World War II radio equipment, which put them a day behind due to poor reception. Tony Sale commented on the result: "On the strength of today's performance Colossus is as good as it was six decades ago." In January 2024, GCHQ released new photographs showing a re-engineered Colossus in an environment that appears to be GCHQ Cheltenham, a reminder that the machine's story kept accumulating new chapters long after the war was over.

  267313330