Sensitivity in Feature Engineering#
How much a model’s output responds to changes in a feature.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
Sensitivity in feature engineering is the degree to which a model’s predictions depend on how features are represented — their scale, distribution, and encoding. Some algorithms are highly sensitive to these choices; others are nearly invariant — and that difference decides how much preprocessing you must do.
Who’s sensitive#
Distance-based models (KNN, SVM), gradient-descent learners (linear / logistic regression, neural nets), and regularized models are scale-sensitive — a feature with a large range will dominate distances or gradients unless it’s normalized. Tree-based models (decision trees, random forests, gradient boosting) split one feature at a time and are essentially scale-invariant.
Why it matters#
Knowing a model’s sensitivity tells you what preprocessing is required versus wasted — you must scale for KNN or a neural net, but scaling for a random forest changes little. The same lens underlies feature-sensitivity analysis: measuring how much the output moves when a feature changes reveals which features the model actually relies on.
Theme: Data Preparation & Features · All terminology
Hint
Mind map — connected ideas
Encode (in Feature Engineering) · Normalize (in Feature Engineering) · Feature Values · Outlier · Support Vector Machines (SVMs) · Decision Trees
Hint
More in Data Preparation & Features
Advanced Sorting in Spreadsheets · Encode (in Feature Engineering) · Normalize (in Feature Engineering)
See also
Source article Adapted (context, re-expressed) in our own words from: Sensitivity in Feature Engineering (insightful-data-lab.com).