Short answers, pulled from the story.
A neural network is a computational model inspired by the structure and functions of biological neural networks. It consists of connected artificial neurons arranged in layers, where signals travel from an input layer through hidden layers to an output layer, with weights on each connection adjusted during training to improve accuracy.
Warren McCulloch and Walter Pitts published the first mathematical model of artificial neurons in 1943, capable of representing logical functions. Frank Rosenblatt described the perceptron in 1958, one of the first implemented neural networks, funded by the United States Office of Naval Research.
Backpropagation is an algorithm that trains neural networks by propagating error gradients backward from the output layer to the input layer to adjust connection weights. Seppo Linnainmaa published the modern form in his master's thesis in 1970; David E. Rumelhart and colleagues popularized it in 1986.
Convolutional neural networks, whose deep architecture originated with Kunihiko Fukushima's neocognitron in 1979, are specialized for spatially structured data like images and are the essential tool for computer vision. Recurrent neural networks allow connections between neurons in the same or previous layers, making them suited for sequential data such as speech and time series; the LSTM architecture, introduced by Sepp Hochreiter and Jürgen Schmidhuber, became the default recurrent design after the forget gate was added in 1999.
The transformer architecture was introduced in 2017 in the paper "Attention Is All You Need" and uses attention mechanisms to model long-range dependencies in data. Large language models including GPT, Gemini, Grok, DeepSeek, and Qwen are all built on this architecture.
As of 2026, training a commercial large language model such as GPT, Grok, or Gemini typically required hundreds of thousands of computers and cost tens of millions of dollars. Training these models requires data centers drawing hundreds of megawatts of power, compared to the roughly 20 watts the human brain consumes.