Vectorizing Logistic Regression#

Stage 4 · ⚙️ Backprop & Vectorization · Lesson 17 of 17 · intermediate

◀ Previous · More Vectorization Examples · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

The whole step, no loops#

Everything in this stage now combines into a single gradient-descent iteration with no Python loop over the data — not over the \(m\) examples, not over the \(n\) features. Both the forward pass and the gradient computation become a handful of matrix operations on \(X \in \mathbb{R}^{n_x \times m}\) and \(Y \in \mathbb{R}^{1 \times m}\).

Forward, then backward#

The forward pass predicts all examples at once; the backward pass forms all gradients at once:

Z = np.dot(w.T, X) + b           # (1, m)   all pre-activations
A = sigmoid(Z)                   # (1, m)   all predictions
dZ = A - Y                       # (1, m)   the clean "prediction - truth"
dw = (1 / m) * np.dot(X, dZ.T)   # (n_x, 1) averaged weight gradient
db = (1 / m) * np.sum(dZ)        # scalar   averaged bias gradient
w = w - alpha * dw
b = b - alpha * db

The vectorised \(\mathrm{d}Z = A - Y\) carries the per-example result \(\mathrm{d}z = a - y\) across the whole set, and np.dot(X, dZ.T) sums \(x^{(i)}\,\mathrm{d}z^{(i)}\) over all examples in one product.

One loop remains#

This is one step of gradient descent. To actually train, you repeat it — and that outer loop over iterations (epochs) is the one loop you cannot vectorise away, because each step depends on the parameters the previous step produced. Everything inside the step, though, is loop-free.

The gateway to deep networks#

That compact block is the whole of logistic regression — a single sigmoid neuron, trained by vectorised gradient descent. A deep network is the same pattern repeated: stack more layers, run the forward pass and backpropagation through each, and reuse exactly these ideas — the sigmoid (or ReLU) activation, the cross-entropy cost, the computation graph, and vectorisation. With this stage complete, you have built every piece a neural network is made of.

Hint