Machine Learning

Lecturers

Yohann de Castro, Aurélien Garivier

Course description

The aim of this Master 1 Informatique Fondamentale course is to introduce the basic theory and algorithms of Machine Learning. Topics to be taught (may be modified) ~20h of lectures + 20h of lab sessions.

• General introduction to Machine Learning: learning settings, curse of dimensionality, overfitting/underfitting, etc.
• Overview of Supervised Learning Theory: True risk versus empirical risk, loss functions, regularization, bias/variance trade-off, complexity measures, generalization bounds.
• Linear/Logistic/Polynomial Regression: batch/stochastic gradient descent, closed-form solution.
• Sparsity in Convex Optimization.
• Support Vector Machines: large margin, primal problem, dual problem, kernelization, etc.
• Neural Networks, Deep Learning.
• Theory of boosting: Ensemble methods, Adaboost, theoretical guarantees.
• Non-parametric Methods (K-Nearest-Neighbors)
• Domain Adaptation
• Metric Learning
• Optimal Transport

Class Notes and Exercises

• Slides: Introduction, Clustering, Dimensionality reduction, Introduction to supervised learning (version 2021), CART algorithm, ERM, perceptron, convexification of risk, Robert Schapire's slides on Boosting, Structural Risk Minimization and SVM, deep learning, reinforcement learning
• Some videos of the lectures: Clustering, Introduction to supervised learning, ERM, perceptron, convexification of risk, Structural Risk Minimization, SVM and kernels
• Class notes (written by the students after each lecture).
• Exercises for Hands-on Sessions: correction of TD 1, correction of TP1, correction of TD 2, correction of TD 3, correction of TD 4, correction of TD 5, correction of TD 6, correction of TD 7 (corrections by Clément Lalanne)
• Notebooks for the hands-on sessions: text clustering, dimensionality reduction for MNIST classification (correction), Bias-variance tradeoff for k-nearest neighbors (correction), Fairness and classification on the adult data set, Kernel SVM for time series, Deep Learning, Reinforcemnet Learning: retail store management (julia version with a few more things)
• Projects on challengedata, Data for the Windmill Detection Project

Prerequisite

Basic knowledge of probability theory, linear algebra and analysis over the reals

Evaluation

50% final exam, 50% project and in-class exercises.

References

• Statistical Learning Theory, V. Vapnik, Wiley, 1998
• Machine Learning, Tom Mitchell, MacGraw Hill, 1997
• Pattern Recognition and Machine Learning, M. Bishop, 2013
• Convex Optimization, Stephen Boyd & Lieven Vandenberghe, Cambridge University Press, 2012.
• On-line Machine Learning courses: https://www.coursera.org/