AI Decision Trees Tutorial: Gini, Entropy

TL;DRDecision trees split data on feature thresholds — interpretable but prone to overfitting.

A tree asks a series of yes/no questions ("age > 30?"), splitting data to maximize purity (measured by Gini or entropy) until leaves are mostly one class. They're highly interpretable (you can read the rules) and handle mixed feature types, but a deep tree memorizes the training data (overfits). Pruning and depth limits help — and ensembles fix it entirely.

Key points

Yes/no splits maximizing purity (Gini/entropy)
Highly interpretable — readable rules
Handle mixed feature types
Overfit easily; prune/limit depth

Common mistakes

Letting trees grow unbounded (overfit)
Reading one deep tree as stable truth
Ignoring that small data changes reshape the tree

Try it: Explain why a deep, unpruned tree overfits.

Example code

<!doctype html><html><head><meta charset="utf-8"></head>
<body style="background:#06040d;color:#e6e0ff;font-family:monospace;padding:20px"><pre>age &gt; 30?
  ├ yes → income &gt; 50k? → ...
  └ no  → ...
deep tree → memorizes training data → overfit</pre></body></html>

Open the interactive lesson →

← Logistic Regression · Binary Classifier Random Forests · Ensemble of Trees →