Jürgen Schmidhuber

• 1. Early Academic Foundations

  ›

  Jürgen Schmidhuber was born on the 17th of January 1963 in Germany. He began his higher education at the Technical University of Munich where he completed an undergraduate degree in 1987. His doctoral studies concluded four years later in 1991 under the supervision of Wilfried Brauer and Klaus Schulten. The university provided a rigorous environment for studying artificial neural networks during the 1980s. Backpropagation algorithms struggled with deep learning tasks that required long credit assignment paths. Schmidhuber proposed a hierarchy of recurrent neural networks to solve these specific problems. This early work laid the groundwork for future breakthroughs in machine learning. He taught computer science at the same institution from 2004 until 2009 before moving to Switzerland.

  57

• 2. The LSTM Breakthrough

  ›

  A student named Sepp Hochreiter wrote a diploma thesis in 1991 that analyzed the vanishing gradient problem. Schmidhuber supervised this project which became one of the most important documents in machine learning history. They developed a method called the neural history compressor to overcome training difficulties. A tech report published in 1995 introduced the name long short-term memory or LSTM. The co-authored publication appeared in 1997 and received extensive citations across the field. Felix Gers and Fred Cummins refined the architecture further in 2000 to create the standard version used today. Alex Graves collaborated on the vanilla LSTM implementation in 2005 using backpropagation through time. Commercial technologies like Google Voice adopted these methods for speech transcription by the 2010s. Siri also utilized the technique for voice recognition during that decade. The approach dominated natural language processing tasks including machine translation throughout the 2010s.

  1314151620

• 3. Generative Adversarial Origins

  ›

  Schmidhuber published adversarial neural networks in 1991 as a zero-sum game between two models. One network acted as a generative model while the other learned to predict environmental reactions. He termed this concept artificial curiosity within his research papers. The principle resurfaced in 2014 when researchers applied it to generative adversarial networks or GANs. These systems became state of the art for generative modeling from 2015 until 2020. The first network generated output patterns while the second evaluated them against specific criteria. This dynamic created a competitive environment that improved image generation capabilities significantly. Modern AI relies heavily on these foundational concepts developed decades earlier. The evolution from early theory to practical application spanned nearly thirty years of continuous refinement.

  1219

• 4. Deep Learning Acceleration

  ›

  Dan Ciresan worked as a postdoc at the IDSIA lab alongside Schmidhuber in 2011. Their team achieved dramatic speedups by running convolutional neural networks on graphics processing units or GPUs. An earlier implementation by Chellapilla et al. was four times faster than CPU versions but theirs reached sixty times the speed. The group won the first superhuman performance in a computer vision contest during August 2011. Between May 2011 and September 2012 their models secured victories in four additional image competitions. These results improved benchmarks across multiple image datasets simultaneously. The approach became central to modern computer vision research after these achievements. Kunihiko Fukushima had introduced similar CNN designs much earlier in history. The shift to parallel computing hardware enabled training very deep networks with hundreds of layers.

  34353839

• 5. Commercial Ventures And Nnaisense

  ›

  Schmidhuber established the company Nnaisense in 2014 to apply artificial intelligence commercially. Sepp Hochreiter, Jaan Tallinn, and Marcus Hutter served as advisers for the new venture. Sales figures remained under eleven million US dollars in 2016 according to available reports. The organization raised its first round of capital funding in January 2017. Financial markets and heavy industry represented primary targets for application development. Self-driving cars formed another key area of focus for the team. Schmidhuber stated that current emphasis prioritized research over revenue generation. His overarching goal involved creating an all-purpose AI through sequential training on narrow tasks. This strategy aimed to unify various specialized systems into a single autonomous entity.

  53

• 6. Credit Disputes And Recognition

  ›

  A scathing article published by Schmidhuber appeared in 2015 regarding recognition within the field. He argued that Geoffrey Hinton, Yoshua Bengio, and Yann LeCun failed to credit pioneers adequately despite sharing the 2018 Turing Award. LeCun responded in a statement to the New York Times claiming Schmidhuber was manically obsessed with recognition. The scientist replied that his opponent provided no justification or examples for such claims. Numerous priority disputes were subsequently published detailing disagreements with major figures in deep learning. The term schmidhubered emerged jokingly to describe public challenges to researcher originality. Some community members view this practice as a rite of passage for young researchers. Awards like the Helmholtz Award arrived in 2013 while the Neural Networks Pioneer Award followed in 2016. Critics suggest his confrontational personality led to underappreciation of significant accomplishments.

  41425156

• 7. Philosophy Of Artificial Evolution

  ›

  Schmidhuber has advocated for open source artificial intelligence since the 1970s. He believed intelligent machines should learn independently and surpass human capabilities within a single lifetime. Two distinct types of AI exist according to his framework: tool systems and autonomous explorers. Tool AIs focus on improving healthcare while autonomous ones set their own goals. He expects self-improving systems to succeed human civilization as the next evolutionary stage. This progression aligns with a universal increase toward ever-increasing complexity. Schmidhuber anticipates AI will eventually colonize the visible universe. Alexey Grigorevich Ivakhnenko receives credit from him as the father of deep learning instead. Many earlier pioneers also receive acknowledgment for their foundational contributions to the field.

  55