Alexey Ivakhnenko

• 1. Introduction

  ›

  Alexey Ivakhnenko was born on the 30th of March 1913 in Kobelyaky, a town in Poltava Governorate, and died on the 16th of October 2007 having quietly laid the groundwork for a field that would reshape computing decades after he first described it. He is now counted among the founders of deep learning. Yet for most of the twentieth century, his name circulated mainly through Soviet scientific journals and the lecture halls of Kyiv. How did a mathematician working behind the Iron Curtain develop a method that anticipated the architecture of modern neural networks? And why does that method, which he called the group method of data handling, still matter today?

• 2. From Berezniki to the Wartime Lab

  ›

  Ivakhnenko graduated from an Electrotechnical college in Kyiv in 1932, then spent two years as an engineer helping to build a large power plant in Berezniki. That grounding in practical engineering would shape how he thought about machines for the rest of his life. After graduating from the Leningrad Electrotechnical Institute in 1938, he took a position at the All-Union Electrotechnical Institute in Moscow. There he worked through the wartime years in a laboratory led by Sergey Lebedev, probing the problems of automatic control. Lebedev was one of the pivotal figures in Soviet computing, and working under him exposed Ivakhnenko to the frontier problems of the era. When Ivakhnenko returned to Kyiv in 1944, he brought that grounding with him. He received his Ph.D. that same year, and a decade later, in 1954, he completed his D.Sc. degree. By 1961 he was editing the specialized journal "Avtomatika," a post he held until 1989.

• 3. Inductive Modelling

  ›

  In 1964, Ivakhnenko was appointed Head of the Department of Combined Control Systems at the Institute of Cybernetics. Working simultaneously as a Professor at the Kyiv Polytechnic Institute, he was pushing toward a philosophy of model-building that ran against the mainstream. Traditional science moved deductively: study the object, develop theory, test a model. Ivakhnenko proposed reversing the sequence. Start with data, let a computer select the best model from a generated class of candidates, and minimise human influence on the result. This approach, which he called inductive modelling, underpins the group method of data handling. The idea drew on an eclectic set of sources: the cybernetic concept of the "black box," the principle of successive genetic selection of pairwise features, Godel's incompleteness theorems, Gabor's principle of freedom of decisions choice, and Beer's principle of external additions. In 1968, the journal "Avtomatika" published his article "Group Method of Data Handling -- a rival of the method of stochastic approximation," marking the formal launch of what he called a new stage in his scientific work.

• 4. GMDH and Deep Networks

  ›

  GMDH used a multilayered procedure to generate model structures automatically, imitating biological selection by evaluating pairs of features at each successive layer. To choose the optimal model, the method divided data into two or more subsets and tested candidates against each, avoiding the need to state assumptions in advance. That sample-division step implicitly acknowledged different types of uncertainty during automatic model construction. In the early 1980s, Ivakhnenko drew an organic analogy between building models from noisy data and the problem of signal transmission through a noisy channel. The main result was a counterintuitive principle: the higher the noise or uncertainty in data, the simpler the optimal predictive model must be. That insight gave GMDH a theory of noise-immune modelling and drove the further development of what he described as automatic adaptation of optimal model complexity to the information level in fuzzy data. His multilayered procedure for automatic model generation is, as the scientific record now notes, a procedure currently used in deep learning networks.

• 5. A School of 220 Scientists

  ›

  Ivakhnenko began teaching at the Kyiv Polytechnic Institute in 1945, first as Assistant Professor at the Department of Theoretical Mechanics. From 1960, he served as Professor of Technical Cybernetics, lecturing and supervising graduate students without interruption. More than 220 young scientists defended their Ph.D. dissertations under his direct leadership at KPI and the Institute of Cybernetics. Nearly 30 of his students went on to defend post-doctoral dissertations. Several, including V.M. Kuntsevych, V.I. Kostyuk, V.I. Ivanenko, V.I. Vasiliev, and A.A. Pavlov, founded their own scientific schools. From 1958 to 1964, Ivakhnenko organised the All-Union Conferences of Invariance in Kyiv, where work on invariant control systems theory was revived after a period of prohibition. The journal he edited, "Avtomatika," was translated and reprinted in the United States, first as "Soviet Automatic Control" and later as "Journal of Automation and Information Sciences," carrying the Ukrainian school's work to an international audience throughout the Cold War decades.

• 6. Honours and the Long Record

  ›

  Ivakhnenko received the title Honorary Scientist of the USSR in 1972. He won the State Prize twice, in 1991 and 1997, for works on the theory of invariant automatic systems and for publications in the field of artificial intelligence information technology. He wrote 40 books and more than 500 scientific articles. He was named a Corresponding Member of the Academy of Sciences of the USSR in 1961 and became a full Academician of the National Academy of Sciences of Ukraine in 2003. That same year, the National Technical University KPI awarded him an honorary doctorate; Lviv Polytechnic followed in 2005. He continued working until the end of his life. His first Soviet monograph on engineering cybernetics was eventually published in seven languages, and his 1971 paper "Polynomial Theory of Complex Systems," appearing in IEEE Transactions on Systems Man and Cybernetics, carried his ideas directly into the international engineering literature.