LSTM — Long Short-Term Memory Networks#
A recurrent network with gating that captures long-range dependencies in sequences.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
An LSTM (Long Short-Term Memory network) is a special recurrent neural network for sequential data that overcomes the vanishing- and exploding-gradient problem of vanilla RNNs. It adds a gated memory structure that learns what to keep, update and forget across long sequences.
The gated cell#
A cell state runs through the sequence like a conveyor belt, and three gates — sigmoids in \([0, 1]\) that decide how much to let through — govern it: a forget gate \(f_t = \sigma(W_f \cdot [h_{t-1}, x_t] + b_f)\), an input gate \(i_t\) with candidate values \(\tilde{C}_t = \tanh(\cdot)\), and an output gate \(o_t\). The updates are
with \(C_t\) the long-term cell state and \(h_t\) the short-term hidden state.
Variants#
Common variants include the bidirectional LSTM (reads a sequence forwards and backwards), the stacked LSTM (several layers for depth), the peephole LSTM (gates can see the cell state), and the GRU, a simpler cousin that merges the forget and input gates.
Uses, strengths, weaknesses#
LSTMs power NLP (text generation, translation, sentiment), speech recognition, time-series forecasting (demand, stock prices, anomalies) and control systems. They capture dependencies across 50-100+ steps, but they are computationally heavy, slower than GRUs, and now often outperformed by Transformers on very long sequences.
Theme: Signal Processing & Time Series · All terminology
Hint
Mind map — connected ideas
Prophet — Time Series Forecasting by Facebook (Meta) · ARIMA (AutoRegressive Integrated Moving Average) · Autoencoder · Embedding · Bayesian Neural Networks (BNNs) · Time Series
Hint
More in Signal Processing & Time Series
ARIMA (AutoRegressive Integrated Moving Average) · Bayesian Time Series · Forecast Error · Forecasting Benchmarks · Forecasting Competitions · Log-Space · Low-pass Filtering · M-Competitions (Makridakis Competitions) · Naïve Baseline Forecast · Prophet — Time Series Forecasting by Facebook (Meta) · Seasonal Lag · Seasonality · Signal Processing · Simple Baseline Methods
See also
Source article Adapted (context, re-expressed) in our own words from: LSTM — Long Short-Term Memory Networks (insightful-data-lab.com).