.. _dl-geoffrey-hinton-interview: ======================================================================== Geoffrey Hinton Interview ======================================================================== **Stage 1 · 🧠 Introduction to Deep Learning** · Lesson 04 of 17 · *beginner* :doc:`◀ Previous · Why Deep Learning is Taking Off <03-why-deep-learning-is-taking-off>` · :doc:`Next · Binary Classification and Logistic Regression (Neural Network Basics) ▶ <05-binary-classification-and-logistic-regression-neural-network-basics>` · :doc:`↑ Section ` .. important:: **✨ AI-generated content.** This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it. Heroes of Deep Learning ------------------------ This lesson is a short detour from the mathematics: a look at the ideas from Andrew Ng's **"Heroes of Deep Learning"** conversation with **Geoffrey Hinton**, one of the researchers most responsible for the field existing at all. It is history and perspective rather than a technique — but the history explains *why* the tools in this course look the way they do. Backpropagation and representations ------------------------------------- Hinton is best known for the 1986 paper with **Rumelhart and Williams**, *"Learning representations by back-propagating errors"*, which **popularised backpropagation** for training multi-layer networks. He is careful that they were **not the first** to the idea — versions were proposed years earlier — and that the paper's real contribution was showing backprop could learn useful **internal (distributed) representations**: hidden units that come to stand for meaningful features, exactly the "learned features" idea from Lesson 1. Through the winter -------------------- Backpropagation is just **gradient descent plus the chain rule**, and in the 1990s it hit a wall: in deep networks the gradients **shrank** as they propagated back through the layers (the **vanishing-gradient** problem), and interest drifted to other methods. Through that "neural-network winter" Hinton kept the flame alive with **Boltzmann machines** (with Sejnowski) and later **restricted Boltzmann machines**. His 2006 work on **deep belief networks** — pre-training a deep net one layer at a time, then fine-tuning with backprop — is widely credited with sparking the modern **deep-learning** revival. Newer directions, and advice ------------------------------ Hinton never stopped pushing past the standard recipe — proposing **dropout**, and **capsule networks** aimed at capturing part–whole structure in images — and argued that **unsupervised learning** would ultimately matter more than the supervised setting this course begins with. His advice to newcomers is quietly encouraging: **trust your intuitions**, read enough but not so much that you only reproduce others' thinking, **replicate results** to learn them deeply, and keep going on the problems that feel right. .. hint:: **Related lessons:** :doc:`What is a Neural Network? <01-what-is-a-neural-network>` · :doc:`Why Deep Learning is Taking Off <03-why-deep-learning-is-taking-off>` · :doc:`Logistic Regression Gradient Descent <13-logistic-regression-gradient-descent>` · :doc:`Computation Graph <11-computation-graph>` .. seealso:: **Source article** Adapted (context, re-expressed) in our own words from: `https://insightful-data-lab.com/2025/04/07/geoffrey-hinton-interview/ `__ (insightful-data-lab.com). .. tags:: purpose: reference, topic: deep learning, level: beginner