Seppo Linnainmaa

• 1. Introduction

  ›

  Seppo Ilmari Linnainmaa was born on the 28th of September 1945, in Pori, Finland. He grew up to become a mathematician and computer scientist, and today he is known for creating the modern version of backpropagation. In 1974, he received the first doctorate ever awarded in computer science at the University of Helsinki. It is an achievement most people outside the field have never heard of. What led a young Finnish student to that milestone? And what idea had he already put on paper years before anyone called it backpropagation?

• 2. Decades In Finnish Academia

  ›

  In 1976, Linnainmaa became Assistant Professor, a title that placed him formally within Finland's academic system. From 1984 to 1985, he crossed the Atlantic to serve as Visiting Professor at the University of Maryland, in the United States.

  Between 1986 and 1989, Linnainmaa chaired the Finnish Artificial Intelligence Society. From 1989 until 2007, he was Research Professor at the VTT Technical Research Centre of Finland, before retiring in that final year.

  That research post at VTT lasted until his retirement in 2007. But the discovery he is remembered for had been written down decades earlier, in a thesis filed while he was still a student.

• 3. The Thesis Behind It

  ›

  In his 1970 master's thesis, Linnainmaa introduced what would later be called the reverse mode of automatic differentiation. The technique was designed to compute the derivative of a differentiable composite function, one that can be represented as a graph. It worked by applying the chain rule again and again to the function's individual building blocks.

  The thesis already described explicit, efficient error backpropagation for networks that were arbitrary, discrete, and possibly only sparsely connected. These were structures resembling neural networks, even though Linnainmaa never used that term.

  Not everyone chasing this idea worked in computer science. Some were solving very different problems in very different industries entirely.

• 4. A Question Of Priority

  ›

  Gerardi Ostrowski had already used the same underlying idea, applying it to certain process models in chemical engineering, some five years before Linnainmaa's thesis. He never published his version. Linnainmaa's account, in contrast, made it into print. That made his the first published description of the method, even though it was not the first use of the idea.

  The chemical engineering world where Ostrowski worked never got the same credit, even though process models is where the idea first proved useful.