Machine learning

Define a learning problem, establish a baseline and evaluate it on data not used for fitting.

1. ML1.1Features, labels and examples
2. ML1.2Learning settings
3. ML1.3Models and fitting
4. ML1.4Splits and generalization
5. ML1.5Loss and metrics
6. ML1.6Baselines and overfitting

Fit a linear predictor by hand, implement its loss and gradient, then compare with a library solution.

1. ML2.1A line as a model
2. ML2.2Residuals and squared loss
3. ML2.3Gradient descent from scratch
4. ML2.4Closed-form fitting
5. ML2.5Several features
6. ML2.6Scaling and evaluation

Compute logistic probabilities and evaluate the tradeoff introduced by a classification threshold.

1. ML3.1Logistic regression
2. ML3.2Binary cross-entropy
3. ML3.3Thresholds and decisions
4. ML3.4Confusion matrices
5. ML3.5Precision, recall and F1
6. ML3.6ROC, PR and calibration

Compare model flexibility, regularization and validation procedures without using the test set for tuning.

1. ML4.1Underfitting and overfitting
2. ML4.2Bias and variance
3. ML4.3L2 regularization
4. ML4.4L1 and feature selection
5. ML4.5Cross-validation
6. ML4.6Feature engineering

Choose a split by impurity reduction and distinguish bagging, random forests and boosting.

1. ML5.1Partitions and leaves
2. ML5.2Impurity
3. ML5.3Split gain
4. ML5.4Bagging
5. ML5.5Random forests
6. ML5.6Boosting

Compare nearest-neighbor, naive Bayes and margin-based predictions using their actual assumptions.

1. ML6.1Nearest neighbors
2. ML6.2Scaling and metrics
3. ML6.3Naive Bayes
4. ML6.4Margins
5. ML6.5Kernels
6. ML6.6Model comparison

Perform a clustering update and explain dimensionality reduction, mixtures and anomaly detection.

1. ML7.1k-means assignments
2. ML7.2Centroid updates
3. ML7.3Hierarchical clustering
4. ML7.4Gaussian mixtures and EM
5. ML7.5PCA and visual embeddings
6. ML7.6Anomalies and evaluation

Update a prior, compare posterior means with MAP estimates and interpret predictive uncertainty.

1. ML8.1Prior and likelihood
2. ML8.2Posterior inference
3. ML8.3MAP and posterior summaries
4. ML8.4Posterior prediction
5. ML8.5Gaussian processes
6. ML8.6Graphical models

Construct a leakage-resistant pipeline and select an evaluation procedure that reflects deployment.

1. ML9.1Pipelines
2. ML9.2Leakage
3. ML9.3Imbalanced classes
4. ML9.4Search and selection
5. ML9.5Experiments and reproducibility
6. ML9.6Monitoring and drift

Distinguish sequential decision-making, temporal prediction, recommendation and causal estimation.

1. ML10.1Bandits and exploration
2. ML10.2MDPs and returns
3. ML10.3Q-learning
4. ML10.4Policy gradients
5. ML10.5Time series and recommendation
6. ML10.6Causal reasoning