Predictive Modelling
A good training fit does not establish good performance on new observations. This section defines the prediction target and develops methods for estimating prediction error.
Read the overview first, then bias and variance. Optimism explains the gap between training and test performance; cross-validation and GCV provide ways to assess the fitted procedure.
Predictive Modelling
Overview and model setup.
Prediction Error & Bias-Variance
Separate prediction error into bias, variance, and noise.
Optimism & Model Complexity
Compare training error with expected prediction error.
Cross-Validation
Use held-out predictions to assess and select models.
LOO & Generalized Cross-Validation
Estimate prediction error efficiently from fitted models.