Regression
Regression studies how a numerical response changes with its predictors and how well a fitted relationship predicts new observations. These topics connect prediction error, regularization, flexible curves, and sparse models.
Begin with Predictive Modelling for prediction targets and assessment. Continue to regularization for unstable or correlated predictors, nonlinear regression for flexible curves, and sparse regression for variable selection and trees.
Predictive Modelling
Prediction targets and assessment of new-observation error.
Regularization & Penalized Regression
Least squares, collinearity, ridge penalties, and tuning.
Nonlinear Regression
Function spaces, spline bases, and smoothing.
Sparse Regression and Trees
LASSO, coordinate descent, selection, and regression trees.