Cross-Validation

Resampling techniques for estimating how well a model performs on unseen data, most commonly k-fold cross-validation.

Category: Machine Learning · Created: 2026-08-18 · Updated: 2026-08-18

Cross-validation is a family of resampling techniques for estimating how well a machine learning model will perform on data it has never seen. The naive approach — train and evaluate on the same data — is meaningless, because models can memorize; the standard remedy is to hold out a portion of the data for testing. But a single holdout split wastes data and gives an estimate that depends on the luck of the split. Cross-validation addresses both problems by repeating the train–test process many times on different splits.

The most common form is k-fold cross-validation. The dataset is divided into k equal parts (folds); the model is trained on k−1 folds and evaluated on the remaining fold; the process repeats k times, each fold serving as the test set exactly once; the k scores are averaged. Typical choices are k = 5 or k = 10. Leave-one-out cross-validation is the extreme k = n case, training on all but one point — expensive but nearly unbiased. Stratified cross-validation preserves class proportions in each fold, which matters for imbalanced classification problems. For time series, folds must respect temporal order, since random splits leak the future into the past.

The purpose is honest evaluation and model selection. Cross-validation scores guide the choice of hyperparameters — the depth of a decision tree, the k of k-nearest neighbors, the regularization strength — and detect the overfitting and regularization problems that arise when models are tuned too aggressively on training data. A common discipline is to keep a final untouched test set: use cross-validation on the training portion to select hyperparameters, then evaluate the chosen model once on the held-out test set to obtain an unbiased estimate of real-world performance.

Cross-validation has costs and caveats. It multiplies training time by k, which is why practitioners use fewer folds for large models. The estimate is a random variable: repeated runs with different random splits give different scores, and with small datasets the variance can be large. The assumptions are that folds are independent and representative — violated when data are clustered (the same patient appearing in multiple folds) or non-stationary. Used properly, cross-validation is the standard evidence that a reported accuracy is real rather than an artifact of memorization.

Cross-Validation

Tags

Related articles