🏋️ Quantization

Quantization#

Reducing numeric precision of weights to shrink and speed up models.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Quantization lowers the numerical precision of a model’s weights (and often activations) — from 32-bit floating point (FP32) down to INT8, FP8, or even INT4. Fewer bits per number means a smaller, faster, cheaper model.

The payoff#

An INT8 model uses about 75% less memory than FP32, and because decoding is often memory-bandwidth-bound, 4-bit weights can be read up to 4× faster than 16-bit — directly cutting latency and inference cost on hardware that supports low precision.

Managing the trade-off#

Naive quantization degrades accuracy. Post-training quantization (PTQ) converts a trained model quickly (with a small calibration set), while quantization-aware training (QAT) bakes precision loss into training to preserve accuracy; advanced schemes like AWQ and GPTQ protect the most sensitive weights to reach near-FP16 quality at INT4 speeds.

Theme: Model Training & Optimization · All terminology

Hint

Mind map — connected ideas

Caching · TPU Clusters · Inference Cost (Inference $) · Neural Networks · Compute budgets · ONNX (Open Neural Network Exchange)

Hint

More in Model Training & Optimization

Active Learning · Binary Cross-Entropy (BCE) · Deep Ensembles · Early Stopping · Ensemble · Epochs · FLOPs · Full Annotation · Hyperparameter · Label Noise · Log-Odds · Logit Space · Logits · Loss Functions