Logistic Regression (Binary Classification Model)#

Stage 2 · 🔵 Logistic Regression as a Neuron · Lesson 06 of 17 · beginner

◀ Previous · Binary Classification and Logistic Regression (Neural Network Basics) · Next · Logistic Regression – Loss Function and Cost Function ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

From score to probability#

Given an input \(x\), logistic regression predicts \(\hat{y} = P(y = 1 \mid x)\) — the probability the label is 1. A plain linear score \(\mathbf{w}^{\!\top}\mathbf{x} + b\) can be any real number, from large negative to large positive, so it cannot serve as a probability directly. It has to be squashed into \([0, 1]\).

The sigmoid#

The squashing function is the sigmoid (logistic) function:

\[\hat{y} = \sigma(\mathbf{w}^{\!\top}\mathbf{x} + b), \qquad \sigma(z) = \frac{1}{1 + e^{-z}},\]

with parameters \(\mathbf{w} \in \mathbb{R}^{n_x}\) (a weight per feature) and a bias \(b \in \mathbb{R}\).

Reading it#

The sigmoid is an S-curve between 0 and 1: as \(z \to +\infty\) it approaches 1, as \(z \to -\infty\) it approaches 0, and at \(z = 0\) it is exactly 0.5. So a large positive score means “confidently 1”, a large negative score “confidently 0”, and a score near zero an undecided, halfway probability.

One neuron#

Put together, logistic regression is precisely a single neuron: a linear combination of the inputs followed by a nonlinear activation — here the sigmoid. That is the exact template from Lesson 1, and stacking many such units is all a neural network is. The next lesson gives this neuron a loss, so it can learn \(\mathbf{w}\) and \(b\) from data.

Hint