Joseph Weizenbaum

• 1. Introduction

  ›

  Joseph Weizenbaum built one of the most unsettling programs in the history of computing, and it unsettled him most of all. In 1966, he released a simple chatbot called ELIZA, named after the ingenue in George Bernard Shaw's Pygmalion. He expected people to see through it. Instead, they poured out their hearts to it. His own secretary, who knew perfectly well she was talking to a machine, asked him to leave the room so she could continue the conversation in private. That moment changed Weizenbaum's life. A man who had helped automate the American banking system, who had built a programming language from scratch, who had earned a tenured professorship at MIT, found himself asking a question his colleagues would rather not touch: just because a computer can do something, does that mean it should? The answers he gave would make him one of the most important critics of artificial intelligence in the twentieth century, and a figure whose warnings grow more relevant with every passing decade.

• 2. Escape and Education

  ›

  Weizenbaum was born in Berlin on the 8th of January 1923 to Jewish parents, and the family fled Nazi Germany in January 1936. They arrived in the United States, and Weizenbaum began studying mathematics in 1941 at Wayne State University in Detroit, Michigan. A year later he interrupted that work to serve in the U.S. Army Air Corps as a meteorologist. He had applied for cryptology work but was turned down because his background made him an "enemy alien" in the eyes of the military. That label, applied to a man who had escaped a regime that called him an enemy by birth, carried its own particular irony. After the war ended, he returned to Wayne State in 1946 and earned his bachelor's degree in mathematics in 1948, followed by a master's degree in 1950. Around 1952, still at Wayne as a research assistant, he turned toward the emerging world of digital computers, helping to build one after working on analog machines. That hands-on engineering experience would define the next phase of his career.

• 3. Building the Infrastructure

  ›

  In 1956, Weizenbaum took a job with General Electric working on a project called ERMA. The system introduced the magnetically encoded fonts printed along the bottom border of bank checks, making it possible for machines to read and sort them automatically. This process, known as magnetic ink character recognition or MICR, allowed check volumes to expand far beyond what human clerks could have handled. A short paper he published in the journal Datamation in 1962 described a strategy for making a Gomoku-playing program that could beat novice players, titled "How to Make a Computer Appear Intelligent." That title turned out to be a preview of his future preoccupations. By 1963, the SLIP programming language he had developed, which stood for Symmetric List Processing, was impressive enough to earn him an associate professorship at MIT. Within four years he had tenure, and in 1970 he received a full professorship in computer science and engineering. He also held appointments at Harvard, Stanford, and the University of Bremen, among other institutions.

• 4. ELIZA and Its Aftermath

  ›

  ELIZA ran on a script called DOCTOR, and its design was drawn from the therapeutic method of Carl Rogers, who championed open-ended questions as a way to draw patients into deeper conversation with their therapists. The program responded to typed statements by applying pattern-matching rules that reflected the user's words back as questions. It was, by Weizenbaum's own account, comparatively simple. What it produced was not. Users treated it as though it understood them. They confided in it. They became attached to it. Weizenbaum was shocked. He had not built a mind; he had built a mirror, and people were seeing companionship in the reflection. Enthusiasts declared ELIZA a forerunner of thinking machines, an interpretation Weizenbaum spent much of his subsequent career working to dismantle. The program was written in SLIP, the language of his own creation, which had already secured his position at MIT. ELIZA is now recognized as an early example of what are called chatbots, but at the time of its release in 1966 it raised questions about human credulity that no one had quite anticipated.

• 5. The Conservative Machine

  ›

  Weizenbaum drew on his years at General Electric when he explained why he feared computers as a social force. He pointed to what ERMA had done for banking. By allowing banks to process an ever-growing volume of checks automatically, the computer had relieved pressure that might otherwise have forced the industry to restructure itself, perhaps through decentralization. Efficiency had been gained, but a deeper reckoning had been deferred. He called the computer a fundamentally conservative force. It did not disrupt; it preserved. It extended existing arrangements rather than questioning them. He also called it "a child of the military," a phrase that captured how thoroughly the technology had been shaped by defense priorities. He was careful to say he was not a pacifist; he believed there were circumstances that called for arms. But he argued that the euphemisms surrounding military language, words like "defense" rather than descriptions of what defense actually involved, made societies quicker to accept violence. He told the MIT publication The Tech that his fears extended beyond artificial intelligence to the computer itself as a force in human life.

• 6. Deciding Versus Choosing

  ›

  In 1976, Weizenbaum published Computer Power and Human Reason, the book that laid out his central argument most completely. He was not, he insisted, making the simple claim that machines could not do what humans do. Whether artificial intelligence was achievable or not was beside the point. The real question was whether computers ought to be assigned particular tasks, and that was a question no computer could answer. Weizenbaum built his case on a distinction between deciding and choosing. Deciding, he wrote, is a computational activity. Given the right inputs and the right rules, a machine can reach a decision. Choosing is different. Choice comes from judgment, which is inseparable from values. When an organization deploys a computer to make decisions that humans once made, someone has already made a prior choice grounded in their own values. The outcomes of that computerized process are not neutral. They fall on real people in particular ways, and those consequences reflect the values of whoever set the system in motion, not some objective logic. The book gave Weizenbaum's ideas their widest audience and remains a reference point for debates about algorithmic decision-making.

• 7. Return and Legacy

  ›

  In 1996, Weizenbaum moved back to Berlin and settled near the neighborhood where he had grown up. He died on the 5th of March 2008, and a memorial service was held in Berlin on the 18th of March 2008. He was buried at the Weissensee Jewish cemetery in Berlin. A German documentary about his life, titled Weizenbaum. Rebel at Work., had been released the previous year in 2007 and was later dubbed into English. A second film, Plug & Pray, focusing on his views on artificial intelligence and ethics, followed in 2010. The interdisciplinary German Internet Institute was named the Weizenbaum Institute for the Networked Society in his honor. The Weizenbaum Award also carries his name. Among his published works was Islands in the Cyberstream: Seeking Havens of Reason in a Programmed Society, published by Litwin Books in 2015. The man who once built tools to make banking faster and combat seem surgical ended up arguing, with precise and uncomfortable clarity, that the hardest human problems are not the ones machines solve but the ones they help us avoid.