Short answers, pulled from the story.
LSTM was developed by Sepp Hochreiter and Jürgen Schmidhuber. Hochreiter analyzed the vanishing gradient problem in his 1991 German diploma thesis, and the two researchers published the foundational LSTM paper in 1997 in the journal Neural Computation. Felix Gers, Schmidhuber, and Fred Cummins later added the forget gate in 1999.
LSTM addresses the vanishing gradient problem that affects classic recurrent neural networks during training. When error signals are backpropagated through many time steps, they tend toward zero, preventing the network from learning long-range dependencies. LSTM's Constant Error Carousel allows gradients to flow backward with little to no attenuation.
An LSTM unit contains an input gate, an output gate, and a forget gate. Each gate assigns a value between 0 and 1 to regulate information flow: the forget gate discards outdated information, the input gate decides what new information to store, and the output gate controls what information is passed to the next layer.
Google began using LSTM trained by Connectionist Temporal Classification for speech recognition on Google Voice in 2015. The deployment cut transcription errors by 49% according to the official blog post.
LSTM powers a wide range of applications including speech recognition (Google Voice, Siri), machine translation (Google Translate, which reduced translation errors by 60% using LSTM), text-to-speech (Amazon Polly and Alexa), handwriting recognition, drug design, financial forecasting, and reinforcement learning in video games such as Dota 2 and Starcraft II.
The Gated Recurrent Unit (GRU), published by Kyunghyun Cho and others in 2014, is a simplified variant of the forget gate LSTM architecture. GRU consolidates the LSTM's separate input and forget gates into a more compact design, reducing the number of learned parameters while preserving similar sequence-modeling capabilities.