Classification
Classification predicts a category from observed features. The methods here differ in how they represent a decision boundary, estimate probabilities, and control model complexity.
Start with Logistic Regression and k-NN to compare a parametric model with a local method. Then explore Naive Bayes, trees and ensembles, and SVM. Compare methods using held-out predictions and a metric suited to the errors that matter.
Logistic Regression
Log-odds, probability estimates, and regularization.
k-NN
Neighbour-based predictions and the choice of k.
Naive Bayes
Conditional probabilities and the independence assumption.
Decision Trees
Splits, impurity measures, and pruning.
Random Forest
Bagging, feature sampling, and out-of-bag assessment.
Boosting
Sequential learners and loss minimization.
SVM
Margins, kernels, and regularization.