overfitting

Overfitting

When a model learns the training data too well — including its noise and random variation — and performs poorly on new, unseen data.

When you'd see it: Any ML model evaluation showing high training accuracy but low test/validation accuracy. More complex models (deep neural networks, unpruned decision trees, heavily boosted models) are more prone to overfitting than simpler ones.

Why it matters: A model that only works on its training data is useless in production. Overfitting is the primary failure mode in machine learning — the model that looks best in development fails in deployment. Solutions include cross-validation, regularization, early stopping, and holding out a test set the model never sees during training.

Common mistakes: Evaluating model performance only on training data. A model should always be validated on data it hasn't seen. Training accuracy is almost always higher than test accuracy; the gap is the overfitting signal.

Related underfitting, Random Forest, gradient boosting, decision tree, variance