principal component analysis

Principal Component Analysis

A dimensionality reduction technique that finds the directions of maximum variance in high-dimensional data and projects the data onto a smaller set of uncorrelated components.

When you'd see it: Preprocessing step before ML models when there are many correlated features. Visualizing high-dimensional data in 2D or 3D (first two principal components plotted against each other). Common in genomics, image processing, and any domain with many correlated measurements.

Why it matters: PCA reduces the number of features while preserving as much variance as possible, which can improve model performance when many features are correlated or when data is sparse. It's also the standard technique for visualizing how high-dimensional data clusters.

Common mistakes: Applying PCA to all features indiscriminately. PCA loses interpretability — you can no longer say 'feature X predicted outcome Y,' only 'component 1 predicted outcome Y.' It can also hurt performance if the variance PCA captures isn't the variance that predicts the outcome.

Related covariance, k-means clustering, overfitting, variance