Computer graphics

• 1. Introduction

  ›

  In 1960, two researchers at Boeing put a name to something that barely existed yet. The graphic designer William Fetter coined the phrase "computer graphics," and he in turn credited it to his colleague Verne Hudson. At the time, the field they were describing was little more than glowing dots on a few experimental screens. Today the same idea drives digital photography, film, video games, digital art, and the displays on nearly every phone and computer.

  How does a discipline go from a traced finger on a display scope to images that fall into the uncanny valley? Who were the people who taught a machine to draw a square, to hide a surface, to make a satellite tumble through space on film? And why did a single university department in Utah become the place where so many of these answers were found? This is the story of how computers learned to make pictures, told through the moments and the minds that pushed it forward.

• 2. Before The Screen Could Draw

  ›

  The Lumiere brothers used mattes to create special effects for the earliest films, dating from 1895, but those displays were limited and not interactive. Real interactivity needed different hardware. The first cathode ray tube, the Braun tube, was invented in 1897, and it opened the path to the oscilloscope and the military control panel. These were the first two-dimensional electronic displays that could respond to programmatic or user input.

  During the first half of the twentieth century, advances in electrical engineering, electronics, and television laid the groundwork. The discipline stayed largely unknown until the 1950s and the post-World War II period. It grew out of university and laboratory research into more advanced computers, alongside the United States military's development of radar, aviation, and rocketry. These wartime projects produced a flood of information, and new kinds of displays were needed to process it. That need is what turned computer graphics into a field of its own.

• 3. Teaching A Machine To Draw

  ›

  Douglas T. Ross, working on the Whirlwind SAGE system, ran a personal experiment. He wrote a small program that captured the movement of his finger and displayed its vector, his traced name, on a display scope. In 1959, while at MIT transforming mathematical statements into computer-generated 3D machine tool vectors, he created a display scope image of a Disney cartoon character. The Whirlwind and SAGE projects had already introduced the CRT as a usable display and the light pen as an input device.

  In 1959, the TX-2 computer was developed at MIT's Lincoln Laboratory, integrating new man-machine interfaces. On it ran Ivan Sutherland's Sketchpad software, which let a user draw simple shapes with a light pen, save them, and recall them later. The light pen held a small photoelectric cell in its tip that emitted a pulse whenever the screen's electron gun fired at it. By timing that pulse against the gun's location, the computer could pinpoint the pen and draw a cursor there.

  Sketchpad introduced ideas that still shape graphics interfaces. With drawing constraints, a user did not have to draw four perfect lines to make a square. They could specify a box, give its location and size, and the software would build a perfect one. Sutherland's software also modeled objects rather than mere pictures of them. With a model of a car, you could change the size of the tires without affecting the rest, or stretch the body without deforming the wheels.

• 4. When Industry Took Notice

  ›

  In the early 1960s, Pierre Bezier at Renault used Paul de Casteljau's curves, now called Bezier curves, to develop 3D modeling techniques for car bodies. Curves, unlike polygons, are mathematically complex entities to draw and model well, and these became the foundation for much curve-modeling work in the field. The automobile industry was an early source of momentum.

  Major corporations followed. TRW, Lockheed-Georgia, General Electric, and Sperry Rand were among the many companies getting started by the mid-1960s. IBM responded by releasing the IBM 2250 graphics terminal, the first commercially available graphics computer. In 1966, Ralph Baer, a supervising engineer at Sanders Associates, came up with a home video game later licensed to Magnavox and called the Odyssey. Simple and built from inexpensive parts, it let a player move points of light around a screen, and it was the first consumer computer graphics product.

  David C. Evans served as director of engineering at Bendix Corporation's computer division from 1953 to 1962, then spent five years as a visiting professor at Berkeley. In 1966, the University of Utah recruited him to form a computer science program, and computer graphics quickly became his primary interest. In 1967 he recruited Ivan Sutherland to join him. In 1968, the two founded Evans & Sutherland, the first computer graphics hardware company. Sutherland had wanted Cambridge, Massachusetts, but Salt Lake City was chosen for its closeness to their research group at Utah.

• 5. The Utah Pipeline

  ›

  Sutherland and Evans taught an advanced computer graphics class at the University of Utah that produced students who would found Pixar, Silicon Graphics, and Adobe Systems. Tom Stockham led the image processing group there, working closely with the computer graphics lab. The department became the world's primary research center for the field through the 1970s.

  Edwin Catmull arrived from The Boeing Company with a degree in physics and a love of animation he could not pursue by drawing. The first computer animation he ever saw was his own, an animation of his hand opening and closing. In 1974 he pioneered texture mapping to paint textures onto three-dimensional models, now a fundamental technique in 3D modeling. His goal was to make a feature-length motion picture using computer graphics, which he achieved two decades later through his founding role in Pixar. In the same class, Fred Parke created an animation of his wife's face, and both works appeared in the 1976 film Futureworld.

  John Warnock was another of these pioneers. He later founded Adobe Systems and created a revolution in publishing with his PostScript page description language. James Clark was there too, and he founded Silicon Graphics, whose advanced rendering systems dominated high-end graphics until the early 1990s. A major advance also came from Utah: hidden surface determination, which decides which surfaces sit behind an object and should be hidden when the image is rendered.

• 6. The Language Of Light And Surface

  ›

  In 1968, Arthur Appel described the first ray casting algorithm, the first of a class of ray tracing-based methods that model the paths rays of light take from a source, to surfaces, and into the camera. These algorithms became fundamental to achieving photorealism. The pursuit of realistic light would occupy the field for decades.

  In the 1970s, Henri Gouraud, Jim Blinn, and Bui Tuong Phong built the foundations of shading through the Gouraud and Blinn-Phong models, moving graphics beyond a flat look toward real depth. In 1978, Jim Blinn introduced bump mapping, a technique for simulating uneven surfaces and a predecessor to many later forms of mapping. In 1986, the general rendering equation of David Immel and James Kajiya was developed, an important step toward global illumination. In 1996, Krishnamurty and Levoy invented normal mapping, an improvement on Blinn's bump mapping.

  E. E. Zajac, a scientist at Bell Telephone Laboratory, created a film in 1963 called "Simulation of a two-giro gravity attitude control system," showing how a satellite's attitude could be altered as it orbits the Earth, animated on an IBM 7090 mainframe. At the same lab, Ken Knowlton, Frank Sinden, Ruth A. Weiss, and Michael Noll worked in the field, with Sinden making a film called Force, Mass and Motion to illustrate Newton's laws. At Lawrence Radiation Laboratory, Nelson Max created Flow of a Viscous Fluid and Propagation of Shock Waves in a Solid Form.

• 7. Pictures On A Mass Scale

  ›

  In the early 1980s, metal-oxide-semiconductor very-large-scale integration technology brought 16-bit CPU microprocessors and the first graphics processing unit chips. NEC's microPD7220 was the first GPU, built on a fully integrated NMOS VLSI chip, supporting up to 1024x1024 resolution and laying foundations for the PC graphics market. In 1984, Hitachi released the ARTC HD63484, the first CMOS GPU, and in 1986 Texas Instruments introduced the TMS34010, the first fully programmable graphics processor. Japan's Osaka University built the LINKS-1 system in 1982, a supercomputer using up to 257 Zilog Z8001 microprocessors to render realistic 3D images, and it was the world's most powerful computer as of 1984.

  The arcade matured alongside the hardware. Pong in 1972 was one of the first hit arcade cabinet games. Speed Race in 1974 featured sprites moving along a vertically scrolling road, Gun Fight in 1975 added human-looking animated characters, and Space Invaders in 1978 filled the screen with animated figures. In 1988, the first dedicated real-time 3D graphics boards for arcades arrived with the Namco System 21 and Taito Air System. Lucasfilm and Industrial Light & Magic became the go-to house for film graphics, and Dire Straits' near-fully-CGI video for "Money for Nothing" in 1985 popularized CGI among music fans. Young Sherlock Holmes that same year featured the first fully CGI character in a feature movie, an animated stained-glass knight.

  In the 1990s, home computers took on rendering once limited to costly workstations, and Autodesk products like 3D Studio rose as Silicon Graphics declined. Pixar released Toy Story in 1995, a critical and commercial success. In 1992, Virtua Racing on the Sega Model 1 popularized real-time 3D polygonal graphics, while id Software released Wolfenstein 3D, Doom, and Quake using an engine innovated mainly by John Carmack. In 1999, Nvidia released the GeForce 256, the first home video card billed as a GPU. The Final Fantasy: The Spirits Within, released in 2001, was the first fully computer-generated feature film with photorealistic CGI characters made with motion capture, though it was not a box-office success.

• 8. The Artists And The Algorithms

  ›

  Charles Csuri created the first computer art in 1964, and the Smithsonian recognized him as the father of digital art and computer animation. The Museum of Modern Art and the Association for Computing Machinery-SIGGRAPH named him a pioneer of computer animation. A. Michael Noll began creating digital art in 1962, among the earliest digital artists, and in 1965 he, Frieder Nake, and Georg Nees were the first to publicly exhibit computer art. That April, the Howard Wise Gallery showed Noll's work alongside random-dot patterns by Bela Julesz.

  Jack Bresenham developed Bresenham's line algorithm in 1962, his best-known invention, along with the midpoint circle algorithm. He retired after 27 years at IBM as a Senior Technical Staff Member, taught 16 years at Winthrop University, and holds nine patents. Donald P. Greenberg mentored figures such as Robert L. Cook, Marc Levoy, Brian A. Barsky, and Wayne Lytle, many of whom won Academy Awards for technical achievements, and he founded the NSF Center for Computer Graphics and Scientific Visualization.

  In 1969, the ACM started the Special Interest Group on Graphics, known as SIGGRAPH, to organize conferences, standards, and publications. By 1973 the first annual SIGGRAPH conference was held. In 1977, a group of 25 experts from SIGGRAPH published the 3D Core Graphics System, the first graphical standard and a foundation for later developments. The next frontier the source points to is the 2020s, where ray-tracing moved into real-time rendering and Nvidia pushed ray-tracing cores while AMD followed with FSR, Tensor cores, and ray-tracing cores.