Binary Classification and Logistic Regression (Neural Network Basics)#

Stage 2 · 🔵 Logistic Regression as a Neuron · Lesson 05 of 17 · beginner

◀ Previous · Geoffrey Hinton Interview · Next · Logistic Regression (Binary Classification Model) ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

The task#

Binary classification asks a yes/no question: given an input, output a label \(y \in \{0, 1\}\). The running example is a cat classifier — an image goes in, and the model should output 1 for “cat” and 0 for “not cat”. Logistic regression, the subject of this stage, is the simplest model for this — and, read the right way, a single neuron.

One example, as a vector#

A model needs numbers, so an image is unrolled into a feature vector. A 64×64 colour image has three channels (red, green, blue), giving \(n_x = 64 \times 64 \times 3 = 12{,}288\) values stacked into one column \(x \in \mathbb{R}^{n_x}\). A single labelled example is the pair \((x, y)\) with \(x \in \mathbb{R}^{n_x}\) and \(y \in \{0, 1\}\).

Stacking the whole set#

With \(m\) training examples \((x^{(1)}, y^{(1)}), \dots, (x^{(m)}, y^{(m)})\), Ng stacks them into matrices. Each example becomes a column, so the data form

\[X \in \mathbb{R}^{n_x \times m}, \qquad Y \in \mathbb{R}^{1 \times m},\]

with \(X\) holding one example per column and \(Y\) the matching row of labels.

Why columns#

Putting examples in columns rather than rows is a deliberate convention: it makes the vectorised forward and backward passes later in this stage line up as clean matrix products, with no transposes to track. A small choice now, much tidier code from Lesson 15 onward. In numpy the shapes are X.shape == (n_x, m) and Y.shape == (1, m).

Hint