— Ch. 1 · The First Algorithmic Voice —
Generative artificial intelligence.
~5 min read · Ch. 1 of 6
In 1906, Russian mathematician Andrey Markov introduced a concept that would eventually birth generative artificial intelligence. He analyzed vowel and consonant patterns in the novel Eugeny Onegin to model natural language. This early work laid the foundation for systems that could generate probabilistic text after training on a corpus of words. By the early 1970s, artists began using computers to extend these techniques beyond simple Markov models. Harold Cohen developed AARON, a pioneering computer program designed to autonomously create paintings. These works were exhibited and marked an early attempt at machine creativity.
The terms generative AI planning or generative planning emerged in the 1980s and 1990s to describe systems generating sequences of actions. Military planners used these tools to generate crisis action plans while manufacturing sectors utilized them for process planning. By the early 1990s, this technology was considered relatively mature within its specific domains. The field then shifted focus toward deep learning technologies beginning in the late 2000s. Neural networks trained as discriminative models dominated the era due to the difficulty of generative modeling.
Architectures That Generate Reality
Advancements in 2014 produced the first practical deep neural networks capable of learning generative models for complex data like images. Variational autoencoders and generative adversarial networks enabled systems to output entire images rather than just class labels. DeepDream became one of the earliest examples of such deep generative models producing visual content. In 2017, the Transformer network replaced older long short-term memory models to enable advancements in generative capabilities. This architecture allowed models to process entire sequences simultaneously and capture long-range dependencies more efficiently.
OpenAI developed the first generative pre-trained transformer known as GPT-1 in 2018 following the Transformer breakthrough. Generative adversarial networks consist of two neural networks trained simultaneously in a competitive setting. A generator creates synthetic data by transforming random noise into samples resembling training datasets. A discriminator learns to distinguish authentic data from the synthetic data produced by the generator. These two models engage in a minimax game where the generator aims to fool the discriminator while the discriminator improves its detection abilities. Transformers allow models to capture the significance of every word in a sequence when predicting subsequent words.