Regularization & Penalized Regression
Correlated predictors and flexible models can produce unstable coefficient estimates. Regularization constrains a fit to trade some bias for greater stability.
Start with the overview and least-squares geometry, then examine collinearity through the spectrum of the design matrix. The ridge notes introduce the penalty and explain how to choose its strength.
Regularization & Penalized Regression
Overview and model setup.
Least Squares & Projection
Geometry, fitted values, and residuals.
Spectral Analysis & Collinearity
Understand unstable directions in the design matrix.
Ridge Regression
Control coefficient size with a quadratic penalty.
Ridge Tuning & GCV
Choose penalty strength and measure effective complexity.