🏋️ Model Weights

Model Weights#

The learned parameters that define a trained model’s behaviour.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Model weights are a model’s trainable parameters — the numbers that determine how input features are turned into predictions. Each weight encodes the importance of a feature, and training adjusts them to minimise loss.

By model type#

In linear regression, $y = w_1 x_1 + w_2 x_2 + b$, the weights $w_1, w_2$ say how much each feature contributes — if $w_1 = 200$, every extra square foot adds $200 to the predicted price. In a neural network, every connection between neurons has a weight; the forward pass multiplies inputs by weights and applies an activation, and CNN filter weights learn patterns like edges. In logistic regression, weights are the log-odds contribution — positive pushes toward the positive class, negative away.

How they’re learned#

Weights start random (or from a heuristic), then a loop refines them: forward pass to predict, a loss against the truth, backpropagation for the gradients of loss with respect to each weight, and an optimiser (SGD, Adam) to update them — repeated until the loss settles.

Why they matter#

Weights are the model’s learned knowledge: saving or loading a model is saving or loading its weights. In transfer learning we often freeze the encoder’s weights and fine-tune only the last layers. Concretely, a spam classifier might learn a weight of +2.5 for “free” (strongly spammy) and -1.0 for “invoice” (less so).

Theme: Model Training & Optimization · All terminology

Hint

Mind map — connected ideas

Hyperparameter · Frozen Encoder · Regression Coefficient · Epochs · Neural Networks · FLOPs

Hint

More in Model Training & Optimization

Active Learning · Binary Cross-Entropy (BCE) · Deep Ensembles · Early Stopping · Ensemble · Epochs · FLOPs · Full Annotation · Hyperparameter · Label Noise · Log-Odds · Logit Space · Logits · Loss Functions