Recurrent neural network
Recurrent neural networks carry memory. Every other kind of neural network reads an input, produces an output, and forgets everything that came before. An RNN does something different: it feeds its own output back as an input at the next step, building a hidden state that accumulates a record of everything the network has seen so far. That single design choice makes RNNs capable of something feedforward networks cannot do on their own: learning from the order of things, not just the things themselves.
The consequences of that choice stretch across an enormous range of problems. Speech recognition, handwriting recognition, machine translation, music composition, protein detection, even predicting plasma disruptions inside fusion reactors. The list runs from the deeply practical to the barely imaginable. But how did a feedback loop in a brain diagram become one of the central architectures of modern computing? And why, after decades of dominance, do RNNs coexist uneasily with a newer class of models that threw out recurrence entirely? Those are the questions this documentary sets out to answer.
In 1901, the neuroanatomist Santiago Ramón y Cajal looked at tissue from the cerebellar cortex and described what he called "recurrent semicircles", loops formed by parallel fibers, Purkinje cells, and granule cells. That observation planted a word, "recurrent", in the vocabulary of brain science. It would take most of the twentieth century for that word to fully migrate into computing.
In 1933, Lorente de Nó used Golgi's staining method to map out what he called "recurrent, reciprocal connections" and proposed that excitatory loops could explain parts of the vestibulo-ocular reflex, the mechanism that keeps your vision stable when your head moves. The idea that the brain was not purely a feedforward device, not simply a one-way relay from sensation to response, became a serious research agenda in the 1940s. Donald Hebb proposed the concept of a "reverberating circuit" as a possible explanation for short-term memory. The landmark 1943 paper by McCulloch and Pitts, which introduced the McCulloch-Pitts neuron model, explicitly considered networks containing cycles, noting that the current activity of such networks could be shaped by inputs from indefinitely far in the past. Both authors were interested in closed loops as possible explanations for conditions like epilepsy and causalgia.
The Macy Conferences brought neural feedback loops into open discussion across disciplines, and recurrent inhibition was formally proposed in 1946 as a negative feedback mechanism in motor control. By mid-century, the loop was no longer an anatomical curiosity. It was a theoretical tool.
A parallel intellectual lineage ran through statistical mechanics rather than neuroscience. Wilhelm Lenz and Ernst Ising developed the Ising model in the 1920s as a stripped-down description of magnets at thermal equilibrium, a lattice of interacting spins settling toward a stable state. In 1963, Glauber extended this to a system evolving through time, what became known as Glauber dynamics.
The Sherrington-Kirkpatrick model of spin glass, published in 1975, proved that the energy landscape of such systems was likely to contain many local minima. In 1982, John Hopfield applied that insight directly to networks with binary activation functions, showing that the same mathematics that described magnets could describe associative memory. A 1984 follow-up paper extended the analysis to continuous activation functions, making the Hopfield network a standard reference model for studying neural systems through the lens of statistical mechanics.
Frank Rosenblatt had already reached a related destination from the perceptron side. In 1960 he published "close-loop cross-coupled perceptrons", three-layer networks whose middle layer held recurrent connections that changed by a Hebbian learning rule. In Principles of Neurodynamics (1961), he went further, noting that a fully cross-coupled perceptron network is equivalent to an infinitely deep feedforward network. Kaoru Nakano in 1971 and Shun'ichi Amari in 1972 published similar networks, and Amari's 1972 and 1974 work was later acknowledged by Hopfield in his 1982 paper.
At the resurgence of interest in neural networks during the 1980s, recurrent networks were studied again, sometimes under the name "iterated nets". Two architectures from this period became foundational. The Jordan network, developed in 1986, and the Elman network, developed in 1990, both applied RNNs to questions in cognitive psychology. The Elman network is a three-layer structure with an additional set of context units. The hidden layer connects to those context units with a fixed weight of one, saving a copy of the hidden state at each time step. This lets the network maintain a kind of working memory without any explicit memory mechanism. Jordan networks are similar but feed context from the output layer rather than the hidden layer.
In 1993, a neural history compressor system solved what was described as a "Very Deep Learning" task requiring more than a thousand subsequent layers in an RNN unfolded in time. The system worked by stacking RNNs such that unpredictable inputs at one level became inputs to the next level up, compressing the sequence into progressively higher abstractions. The architecture could be distilled into two components: a "conscious" chunker at the higher level and a "subconscious" automatizer below, each teaching the other to handle events at different timescales. This compression idea anticipated later hierarchical approaches in deep learning.
The central obstacle blocking RNNs from learning over long sequences had a precise name: the vanishing gradient problem. When a network trains by adjusting its weights in response to error signals propagated backward through time, those signals shrink exponentially with the distance between the event that caused the error and the weight being corrected. Long-range dependencies, patterns separated by many time steps, become effectively invisible to the learning algorithm.
Hochreiter and Schmidhuber invented Long Short-Term Memory in 1995 specifically to solve this problem. LSTM uses a set of recurrent gates, including what is called a "forget gate", to regulate the flow of information through the hidden state. Errors can propagate backward through unlimited numbers of virtual layers unfolded in space, meaning an LSTM can in principle learn from events thousands or even millions of discrete time steps in the past. The architecture became the default choice for RNN design across the field.
Around 2006, bidirectional LSTM began transforming speech recognition, outperforming earlier approaches in certain applications. It then improved large-vocabulary speech recognition, text-to-speech synthesis, and was used in Google voice search and Android dictation. The bidirectional design processes the same input sequence in two directions simultaneously, one forward and one backward, concatenating the results so each position in the sequence has access to its full surrounding context. The ELMo model of 2018 is a stacked bidirectional LSTM that takes character-level input and produces word-level embeddings.
The encoder-decoder idea, where one RNN compresses an input sequence into a representation and a second RNN expands that representation into an output sequence, developed in the early 2010s. The papers most commonly identified as the starting point for this approach appeared in 2014, and the architecture became state of the art in machine translation during the 2014-2017 period.
In this configuration, the encoder RNN processes the input into a series of hidden vectors. The decoder RNN then reads those hidden vectors, with an optional attention mechanism that allows the decoder to selectively emphasize different parts of the encoded input at each step of generation. Teacher forcing, the training technique where the decoder is shown the correct output at each step rather than its own previous prediction, helps avoid the compounding errors that arise when an early mistake sends subsequent predictions in the wrong direction.
Gated Recurrent Units, introduced in 2014, arrived as a simpler alternative to LSTM. GRUs have fewer parameters because they lack an output gate, and on tasks like polyphonic music modeling and speech signal modeling, their performance was found to be comparable to LSTM without the additional computational weight.
The encoder-decoder work of this period was instrumental in the development of the attention mechanism, and from attention came the transformer architecture, which abandoned recurrence entirely in favor of self-attention over the full sequence at once.
Beyond LSTM and GRU, the field generated a considerable range of specialized architectures. The Independently Recurrent Neural Network (IndRNN) addresses the vanishing and exploding gradient problems by restricting each neuron to receive only its own past state as context, rather than connecting to all other neurons in the layer. This allows the network to be trained with non-saturating functions like ReLU and supports deep networks built with skip connections.
PixelRNN applied RNN logic to two-dimensional data, processing an image grid row by row or diagonally using two LSTMs scanning from opposite corners. The Hierarchical Recurrent Neural Network (HRNN) decomposes behavior into subprograms at multiple levels; applied to forecasting disaggregated inflation components in the US CPI-U index, it outperformed established inflation prediction methods. Neural Turing Machines couple an RNN to an external memory resource, producing a system that is differentiable end-to-end while being analogous in structure to a Turing machine or Von Neumann architecture. Differentiable Neural Computers extend this by allowing fuzzy addressing and maintaining a record of access chronology.
Memristive networks, funded partly through DARPA's SyNAPSE project in collaboration with IBM Research, HP Labs, and Boston University, implement cortical computing using thin-film materials whose resistance is tuned by ion transport. Their dynamics share properties with Hopfield networks and are analyzed using variations of the Caravelli-Traversa-Di Ventra equation.
Transformers now dominate most of the tasks that once belonged to RNNs. Their self-attention mechanism handles long-range dependencies more cleanly, and they parallelize across the sequence in a way that recurrent architectures cannot match, because each step of an RNN depends on the step before it.
Yet recurrent networks have not disappeared. For applications where computational efficiency matters, where processing must happen in real time, or where the data is inherently sequential in a way that favors the recurrent inductive bias, RNNs remain a practical choice. The continuous-time variant, which models neuron dynamics using ordinary differential equations and is analyzed using dynamical systems theory, retains particular relevance for neuroscience modeling. Applications today still span robot control, brain-computer interfaces, time series anomaly detection, energy forecasting, protein homology detection, subcellular localization prediction, and the prediction of plasma disruptions in fusion reactors.
The software infrastructure now supporting all of this spans PyTorch, TensorFlow, Keras, MXNet, Deeplearning4j, and several other libraries, each providing optimized or just-in-time-compiled implementations. The LSTM architecture remains the most widely used RNN design in production, more than three decades after Hochreiter and Schmidhuber introduced it to solve a problem that had blocked the field for years.
Up Next
Common questions
What is a recurrent neural network and how does it differ from a feedforward neural network?
A recurrent neural network (RNN) is designed for processing sequential data where the order of elements matters. Unlike feedforward neural networks, which process each input independently, RNNs use recurrent connections that feed the output of a neuron at one time step back as input at the next, maintaining a hidden state that captures temporal patterns across the sequence.
Who invented long short-term memory (LSTM) and why was it created?
Hochreiter and Schmidhuber invented LSTM in 1995 to solve the vanishing gradient problem in traditional RNNs. The vanishing gradient problem prevented networks from learning long-range dependencies, because error signals shrank exponentially with the distance between an event and the weight being corrected. LSTM uses recurrent gates, including a forget gate, to allow errors to flow backward through effectively unlimited virtual layers.
What real-world applications use recurrent neural networks?
Recurrent neural networks have been applied to speech recognition, machine translation, handwriting recognition, text-to-speech synthesis, music composition, robot control, time series anomaly detection, energy forecasting, protein homology detection, brain-computer interfaces, and prediction of plasma disruptions in fusion reactors. Bidirectional LSTM was used in Google voice search and Android dictation.
What is the vanishing gradient problem in RNNs?
The vanishing gradient problem is the tendency of error signals to shrink exponentially as they are propagated backward through many time steps during training. This makes it difficult for a standard RNN to learn patterns that depend on events separated by a large number of time steps. LSTM and GRU architectures were specifically designed to address this limitation.
What is a gated recurrent unit (GRU) and how does it compare to LSTM?
Gated Recurrent Units were introduced in 2014 as a simplified alternative to LSTM. GRUs have fewer parameters because they lack an output gate, making them more computationally efficient. Performance on tasks like polyphonic music modeling and speech signal modeling was found to be similar to LSTM, with no consistent advantage identified for either architecture.
What is the historical origin of the concept of recurrence in neural networks?
The concept traces to two parallel lines of work in the early twentieth century. In 1901, Cajal observed recurrent loop structures in the cerebellar cortex, and in 1933, Lorente de Nó described recurrent reciprocal connections using Golgi's staining method. Separately, the Ising model of the 1920s and Hopfield's 1982 paper applied statistical mechanics to network memory. Frank Rosenblatt published close-loop cross-coupled perceptrons in 1960, describing recurrent connections in three-layer networks.
All sources
136 references cited across the entry
- 1journalTime series forecasting using artificial neural networks methodologies: A systematic reviewAhmed Tealab — 2018-12-01
- 2journalA Novel Connectionist System for Improved Unconstrained Handwriting RecognitionAlex Graves et al. — 2009
- 3webLong Short-Term Memory recurrent neural network architectures for large scale acoustic modelingHaşim Sak et al. — Google Research — 2014
- 4arxivConstructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech RecognitionXiangang Li et al. — 2014-10-15
- 5journalA thorough review on the current advance of neural network structures.Samuel Dupond — 2019
- 6journalState-of-the-art in artificial neural network applications: A surveyOludare Isaac Abiodun et al. — 2018-11-01
- 7journalThe Importance of Cajal's and Lorente de Nó's Neuroscience to the Birth of CyberneticsJuan Manuel Espinosa-Sanchez et al. — 2023-07-05
- 8bookHistologie du système nerveux de l'homme & des vertébrésSantiago Ramón y Cajal — Paris : A. Maloine — 1909
- 9journalVestibulo-Ocular Reflex ArcR. Lorente de NÓ — 1933-08-01
- 10journalSome predictions of Rafael Lorente de Nó 80 years laterJorge A. Larriva-Sahd — 2014-12-03
- 12journalA logical calculus of the ideas immanent in nervous activityWarren S. McCulloch et al. — December 1943
- 13journalOn the legacy of W.S. McCullochRoberto Moreno-Díaz et al. — April 2007
- 14journalWarren McCulloch's Search for the Logic of the Nervous SystemMichael A Arbib — December 2000
- 15journalCentral Effects of Centripetal Impulses in Axons of Spinal Ventral RootsBirdsey Renshaw — 1946-05-01
- 16journalRecurrent Neural NetworksStephen Grossberg — 2013-02-22
- 17bookDTIC AD0256582: PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMSFrank Rosenblatt — Defense Technical Information Center — 1961-03-15
- 18bookPattern Recognition and Machine LearningKaoru Nakano — 1971
- 19journalAssociatron-A Model of Associative MemoryKaoru Nakano — 1972
- 20journalLearning patterns and pattern sequences by self-organizing nets of threshold elementsShun-Ichi Amari — 1972
- 21journalThe Existence of Persistent States in the BrainW. A. Little — 1974
- 22citationBeiträge zum Verständnis der magnetischen Eigenschaften in festen KörpernW. Lenz — 1920
- 23citationBeitrag zur Theorie des FerromagnetismusE. Ising — 1925
- 24journalHistory of the Lenz-Ising ModelStephen G. Brush — 1967
- 25journalRoy J. Glauber "Time-Dependent Statistics of the Ising Model"Roy J. Glauber — February 1963
- 26journalSolvable Model of a Spin-GlassDavid Sherrington et al. — 1975-12-29
- 27journalNeural networks and physical systems with emergent collective computational abilitiesJ. J. Hopfield — 1982
- 28journalNeurons with graded response have collective computational properties like those of two-state neuronsJ. J. Hopfield — 1984
- 29bookStatistical mechanics of learningA. Engel et al. — Cambridge University Press — 2001
- 30journalStatistical mechanics of learning from examplesH. S. Seung et al. — 1992-04-01
- 31bookDive into deep learningAston Zhang et al. — Cambridge University Press — 2024
- 32journalLearning representations by back-propagating errorsDavid E. Rumelhart et al. — October 1986
- 33bookHabilitation thesis: System modeling and optimizationJürgen Schmidhuber — 1993
- 34journalLong Short-Term MemorySepp Hochreiter et al. — 1997-11-01
- 36journalFramewise phoneme classification with bidirectional LSTM and other neural network architecturesAlex Graves et al. — 2005-07-01
- 37conferenceAn Application of Recurrent Neural Networks to Discriminative Keyword SpottingSantiago Fernández et al. — Springer-Verlag — 2007
- 38conferencePhoto-Real Talking Head with Deep Bidirectional LSTMBo Fan et al. — 2015
- 39webGoogle voice search: faster and more accurateHaşim Sak et al. — September 2015
- 40journalSequence to Sequence Learning with Neural NetworksIlya Sutskever et al. — 2014
- 41arxivExploring the Limits of Language ModelingRafal Jozefowicz et al. — 2016-02-07
- 42arxivMultilingual Language Processing From BytesDan Gillick et al. — 2015-11-30
- 43arxivShow and Tell: A Neural Image Caption GeneratorOriol Vinyals et al. — 2014-11-17
- 44arxivLearning Phrase Representations using RNN Encoder-Decoder for Statistical Machine TranslationKyunghyun Cho et al. — 2014-06-03
- 45arxivSequence to sequence learning with neural networksIlya Sutskever et al. — 14 Dec 2014
- 46arxivDeep contextualized word representationsPeters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L — 2018
- 47journalAttention is All you NeedAshish Vaswani et al. — Curran Associates, Inc. — 2017
- 48journalPixel Recurrent Neural NetworksAäron van den Oord et al. — PMLR — 2016-06-11
- 50journalFinding Structure in TimeJeffrey L. Elman — 1990
- 51conferenceNeural-Network Models of Cognition — Biobehavioral FoundationsMichael I. Jordan — 1997-01-01
- 52journalLearning Precise Timing with LSTM Recurrent NetworksFelix A. Gers et al. — 2002
- 53bookArtificial Neural Networks – ICANN 2009Justin Bayer et al. — Springer — 2009-09-14
- 54conferenceSequence labelling in structured domains with hierarchical recurrent neural networksSantiago Fernández et al. — 2007
- 55arxivSimplified Minimal Gated Unit Variations for Recurrent Neural NetworksJoel Heck et al. — 2017-01-12
- 56arxivGate-Variants of Gated Recurrent Unit (GRU) Neural NetworksRahul Dey et al. — 2017-01-20
- 57webRecurrent Neural Network Tutorial, Part 4 – Implementing a GRU/LSTM RNN with Python and Theano – WildMLDenny Britz — October 27, 2015
- 58arxivEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence ModelingJunyoung Chung et al. — 2014
- 59citationAre GRU cells more specific and LSTM cells more sensitive in motive classification of text?N. Gruber et al. — 2020
- 60journalBidirectional associative memoriesBart Kosko — 1988
- 61journalExponential stability for markovian jumping stochastic BAM neural networks with mode-dependent probabilistic time-varying delays and impulse controlRajan Rakkiyappan et al. — 2 January 2015
- 62bookNeural networks: a systematic introductionRául Rojas — Springer — 1996
- 63journalHarnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless CommunicationHerbert Jaeger et al. — 2004-04-02
- 64journalReal-time computing without stable states: a new framework for neural computation based on perturbationsWolfgang Maass et al. — 2002
- 65bookProceedings of International Conference on Neural Networks (ICNN'96)Christoph Goller et al. — 1996
- 66thesisThe representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errorsSeppo Linnainmaa — University of Helsinki — 1970
- 67bookEvaluating Derivatives: Principles and Techniques of Algorithmic DifferentiationAndreas Griewank et al. — SIAM — 2008
- 68citation28th International Conference on Machine Learning (ICML 2011)Richard Socher et al.
- 69journalRecursive Deep Models for Semantic Compositionality Over a Sentiment TreebankRichard Socher et al.
- 70arxivNeural Turing MachinesAlex Graves et al. — 2014
- 71journalHybrid computing using a neural network with dynamic external memoryAlex Graves et al. — 2016-10-12
- 72bookAdaptive Processing of Sequences and Data StructuresGuo-Zheng Sun et al. — Springer — 1998
- 73journalTuring machines are recurrent neural networksHeikki Hyötyniemi — 1996
- 74bookThe Utility Driven Dynamic Error Propagation NetworkAnthony J. Robinson et al. — Department of Engineering, University of Cambridge — 1987
- 75bookBackpropagation: Theory, Architectures, and ApplicationsRonald J. Williams et al. — Psychology Press — 1 February 2013
- 76journalA Local Learning Algorithm for Dynamic Feedforward and Recurrent NetworksJürgen Schmidhuber — 1989-01-01
- 77bookNeural and adaptive systems: fundamentals through simulationsJosé C. Príncipe et al. — Wiley — 2000
- 78arxivTraining recurrent networks online without backtrackingOllivier Yann et al. — 2015-07-28
- 79journalA Fixed Size Storage O(n3) Time Complexity Learning Algorithm for Fully Recurrent Continually Running NetworksJürgen Schmidhuber — 1992-03-01
- 80reportComplexity of exact gradient computation algorithms for recurrent neural networksRonald J. Williams — Northeastern University, College of Computer Science — 1989
- 81journalLearning State Space Trajectories in Recurrent Neural NetworksBarak A. Pearlmutter — 1989-06-01
- 82bookA Field Guide to Dynamical Recurrent NetworksSepp Hochreiter — John Wiley & Sons — 15 January 2001
- 83journalOn-Line Learning Algorithms for Locally Recurrent Neural NetworksPaolo Campolucci et al. — 1999
- 84journalDiagrammatic derivation of gradient algorithms for neural networksEric A. Wan et al. — 1996
- 85journalA Signal-Flow-Graph Approach to On-line Gradient CalculationPaolo Campolucci et al. — 2000
- 86conferenceConnectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networksAlex Graves et al. — 2006
- 87journalSequence Modeling with CTCAwni Hannun — 2017-11-27
- 88citationIJCAI 99Faustino J. Gomez et al. — Morgan Kaufmann — 1999
- 89thesisApplying Genetic Algorithms to Recurrent Neural Networks for Learning Network Parameters and ArchitectureOmar Syed — Department of Electrical Engineering, Case Western Reserve University — May 1995
- 90journalAccelerated Neural Evolution Through Cooperatively Coevolved SynapsesFaustino J. Gomez et al. — June 2008
- 91arxivIndependently Recurrent Neural Network (IndRNN): Building a Longer and Deeper RNNShuai Li et al. — 2018
- 92journalLearning complex, extended sequences using the principle of history compressionJürgen Schmidhuber — 1992
- 93journalDeep LearningJürgen Schmidhuber — 2015
- 94thesisUntersuchungen zu dynamischen neuronalen NetzenSepp Hochreiter — Institut f. Informatik, Technical University of Munich — 1991
- 95journalLearning and Extracting Finite State Automata with Second-Order Recurrent Neural NetworksC. Lee Giles et al. — 1992
- 96journalConstructing Deterministic Finite-State Automata in Recurrent Neural NetworksChristian W. Omlin et al. — 1996
- 97journalHow Hierarchical Control Self-organizes in Artificial Adaptive SystemsRainer W. Paine et al. — 2005-09-01
- 99journalForecasting CPI inflation components with Hierarchical Recurrent Neural NetworksOren Barkan et al. — 2023
- 100bookRecurrent Multilayer Perceptrons for Identification and Control: The Road to ApplicationsKurt Tutschku — University of Würzburg Am Hubland — June 1995
- 101journalEmergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot ExperimentYuichi Yamashita et al. — 2008-11-07
- 102journalThe hierarchical and functional connectivity of higher-order cognitive mechanisms: neurorobotic model to investigate the stability and flexibility of working memoryFady Alnajjar et al. — 2013
- 104citationCortical computing with memristive nanodevicesGreg Snider — 2008
- 105journalThe complex dynamics of memristive circuits: analytical results and universal slow relaxationFrancesco Caravelli et al. — 2017
- 106citation3rd international conference on Simulation of adaptive behavior: from animals to animats 3Inman Harvey et al. — 1994
- 107conferenceEvolving communication without dedicated communication channelsMatt Quinn — 2001
- 108journalThe dynamics of adaptive behavior: A research programRandall D. Beer — 1997
- 109conferenceDeriving the Recurrent Neural Network Definition and RNN Unrolling Using Signal ProcessingAlex Sherstinsky — 2018-12-07
- 110journalComputational Capabilities of Recurrent NARX Neural NetworksHava T. Siegelmann et al. — 1995
- 111journalComparative analysis of Recurrent and Finite Impulse Response Neural Networks in Time Series PredictionMilos Miljanovic — Feb–Mar 2012
- 112journalBrain inspired neuronal silencing mechanism to enable reliable sequence identificationShiri Hodassman et al. — 2022-09-29
- 113newsGoogle Built Its Very Own Chips to Power Its AI BotsCade Metz — May 18, 2016
- 114book2006 IEEE/RSJ International Conference on Intelligent Robots and SystemsHermann Mayer et al. — October 2006
- 115conferenceEvolino: Hybrid Neuroevolution/Optimal Linear Search for Sequence LearningDaan Wierstra et al. — 2005
- 116arxivRecurrent neural networks for time series forecastingGábor Petneházi — 2019-01-01
- 117journalRecurrent Neural Networks for Time Series Forecasting: Current Status and Future DirectionsHansika Hewamalage et al. — 2020
- 118journalFramewise phoneme classification with bidirectional LSTM and other neural network architecturesAlex Graves et al. — 2005
- 119book2013 IEEE International Conference on Acoustics, Speech and Signal ProcessingAlex Graves et al. — 2013
- 120journalSpeech synthesis from neural decoding of spoken sentencesEdward F. Chang et al. — 24 April 2019
- 121journalNeuroprosthesis for Decoding Speech in a Paralyzed Person with AnarthriaDavid A. Moses et al. — 2021-07-15
- 122conferenceLong Short Term Memory Networks for Anomaly Detection in Time SeriesPankaj Malhotra et al. — Ciaco — April 2015
- 124arxivForecasting Energy Consumption using Recurrent Neural Networks: A Comparative AnalysisAbhishek Maity et al. — 23 January 2026
- 126bookArtificial Neural Networks — ICANN 2002Douglas Eck et al. — Springer — 2002-08-28
- 127journalLearning nonregular languages: A comparison of simple recurrent networks and LSTMJürgen Schmidhuber et al. — 2002
- 128journalLSTM Recurrent Networks Learn Simple Context Free and Context Sensitive LanguagesFelix A. Gers et al. — 2001
- 129journalKalman filters improve LSTM network performance in problems unsolvable by traditional recurrent netsJuan Antonio Pérez-Ortiz et al. — 2003
- 130conferenceOffline Handwriting Recognition with Multidimensional Recurrent Neural NetworksAlex Graves et al. — MIT Press — 2009
- 131conferenceUnconstrained Online Handwriting Recognition with Recurrent Neural NetworksAlex Graves et al. — Curran Associates — 2007
- 132bookHuman Behavior UnterstandingMoez Baccouche et al. — Springer — 2011
- 133journalFast model-based protein homology detection without alignmentSepp Hochreiter et al. — 2007
- 134journalBidirectional Long Short-Term Memory Networks for Predicting the Subcellular Localization of Eukaryotic ProteinsTrias Thireou et al. — July 2007
- 135bookAdvanced Information Systems EngineeringNiek Tax et al. — 2017
- 136journalDoctor AI: Predicting Clinical Events via Recurrent Neural NetworksEdward Choi et al. — 2016