🧠 Deep Learning
From a single neuron to deep networks
0.5.dev0+git.20260808.d6e9440 - August 08, 2026 18:54 UTC

Deep Learning

Deep learning stacks simple, differentiable units into networks trained end-to-end by gradient descent. This course follows the ground-up path the source corpus takes — view logistic regression as a single neuron, learn the computation graph and backpropagation that train it, then vectorise it into the efficient building block that scales to deep networks — as an ordered, self-contained course of 17 lessons.

Read it at any depth:

• newcomers — what a neural network is, and a single neuron;

• practitioners — the sigmoid model, the cross-entropy cost, and gradient descent;

• researchers / engineers — the computation graph, backpropagation, and full vectorisation with numpy.

Warning

These lessons build the mathematics from scratch in numpy to show what a framework does under the hood. In production you would use a framework (PyTorch / Keras) — its automatic differentiation computes exactly these gradients for you.

Note

Follow the lessons in order with Next ▶, or jump in by stage below. Snippets use real numpy / PyTorch / Keras calls and connect to scikit-plots’ visualkeras architecture views. This course pairs with the Terminology reference (classification metrics for evaluating networks).

---

🧠 Stage 1 — Introduction to Deep Learning · 4 lessons

What a neural network is, where supervised deep learning applies, and why it took off. · beginner

• 01 · What is a Neural Network? — How stacked neurons map inputs to outputs, learning the mapping from examples by gradient descent.

• 02 · Supervised Learning and Neural Networks — Learning input→output mappings from labelled data, and which network suits structured, image or sequence inputs.

• 03 · Why Deep Learning is Taking Off — The three drivers — data scale, compute, and algorithms — that made deep networks practical.

• 04 · Geoffrey Hinton Interview — Perspectives from a pioneer of backpropagation on how the field’s key ideas emerged.

🔵 Stage 2 — Logistic Regression as a Neuron · 3 lessons

A single sigmoid unit — the binary-classification model, its loss and its cost function. · beginner

• 05 · Binary Classification and Logistic Regression (Neural Network Basics) — Framing a yes/no prediction, the notation for one training example, and stacking data into matrices.

• 06 · Logistic Regression (Binary Classification Model) — A single neuron: a linear score passed through the sigmoid to output a probability.

• 07 · Logistic Regression – Loss Function and Cost Function — The per-example cross-entropy loss and the averaged cost that training minimises.

📉 Stage 3 — Derivatives & the Computation Graph · 5 lessons

The calculus that trains a neuron: gradients, the computation graph and the chain rule. · intermediate

• 08 · Gradient Descent in Logistic Regression — Iteratively stepping the parameters downhill along the cost gradient toward a minimum.

• 09 · Derivatives — The slope of a function — the building block of every gradient used to train a network.

• 10 · More Derivative Examples — Worked derivatives that build intuition for how small input changes move the output.

• 11 · Computation Graph — Breaking a computation into elementary steps — the structure backpropagation runs on.

• 12 · Derivatives with a Computation Graph — Applying the chain rule backward through the graph to get every partial derivative efficiently.

⚙️ Stage 4 — Backprop & Vectorization · 5 lessons

Backpropagation across a whole training set, then vectorising it for speed with numpy. · intermediate

• 13 · Logistic Regression Gradient Descent — Backpropagation for one example: the derivatives of the loss w.r.t. each weight and the bias.

• 14 · Gradient Descent on m Training Examples — Averaging the single-example gradients over the whole training set for one update step.

• 15 · Vectorization in Logistic Regression — Replacing explicit loops with matrix operations so the whole batch computes at once.

• 16 · More Vectorization Examples — Further numpy patterns — broadcasting and whole-array functions — that remove Python loops.

• 17 · Vectorizing Logistic Regression — A fully vectorised forward and backward pass over all examples — the gateway to deep networks.

🔤 Every lesson, A–Z index

🔠 Open the full alphabetical index

Binary Classification and Logistic Regression (Neural Network Basics) Computation Graph Derivatives Derivatives with a Computation Graph Geoffrey Hinton Interview Gradient Descent in Logistic Regression Gradient Descent on m Training Examples Logistic Regression (Binary Classification Model) Logistic Regression Gradient Descent

Logistic Regression – Loss Function and Cost Function More Derivative Examples More Vectorization Examples Supervised Learning and Neural Networks Vectorization in Logistic Regression Vectorizing Logistic Regression What is a Neural Network? Why Deep Learning is Taking Off

Tags: purpose: reference topic: deep learning