Motivation of Decision Trees: An Incremental Model of Decision-Making#

Stage 7 · 🌳 Decision Trees · Lesson 44 of 56 · advanced

◀ Previous · Interpreting and Assessing a Forward-Selection Logistic Regression Model for College Student Retention · Next · The CART Algorithm ▶ · ↑ Section

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

How people decide#

Humans often decide by asking a sequence of simple questions. To triage a patient: Is there chest pain? If so, is it severe? Radiating to the arm? Each answer narrows the possibilities until a decision is reached. A decision tree formalises exactly this — an incremental model that reaches a prediction by asking one yes/no question at a time. Its appeal is that it thinks the way people do.

A tree of questions#

Structurally, a decision tree is a flowchart. Each internal node poses a test on one feature (“distance > 3 miles?”); each branch is an answer that leads onward; and each leaf delivers a prediction — a class for classification, a number for regression. To predict for a new observation, you start at the root and follow the branches its feature values dictate until you land in a leaf. The path from root to leaf reads as a plain chain of if-then rules.

What trees are good at#

This structure has real strengths. Trees capture non-linear relationships and interactions between features automatically — a split on one feature can lead to different splits on another, so the effect of one variable can depend on another with no special terms. They handle numeric and categorical features side by side, need no scaling or standardisation, and are unbothered by the linearity assumptions that constrain regression. They cope naturally with the messiness real data brings.

White-box models#

Above all, trees are interpretable — a “white-box” model whose every decision can be traced and explained, in contrast to “black-box” methods like neural networks. You can read a tree, show it to a domain expert, and check whether its logic makes sense. That transparency is why trees are a favourite when a decision must be justified, not merely made — and it sets up their use later in the stage to explain the clusters of Stage 4. The next lesson gives the algorithm that actually builds a tree from data: CART.

Hint