Moodle: Most communication for this class will take place on Moodle. Please sign up here .
Lecture: Wednesdays, 13:30 - 16:00 o'clock, ZOOM for now, S103/226 when again possible
Exercises: Wednesdays, 16:15 - 17:00 o'clock, ZOOM for now, S103/226 when again possible
Language: English

Lectures and exercises take place via Zoom. Please visit the Moodle page to find the link.

Motivation Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it. Many researchers also think it is the best way to make progress towards human-level AI. In this class, you will learn about statistical machine learning techniques, and gain some practice implementing them and getting them to work for yourself.

Lectures

This course consists of a mix of online lectures and flipped classroom sessions. That is, aside from our online lectures, you may also want to watch the aditional videos provided, even before the lecture. Please prepare a list of 5 questions as extra homework for those videos, since we (all of you) may also try to answer your questions in class. We will also present our own work. We may even ask you to read papers, let's see. Aside from the “5-questions” homework, there are also four written assignments involving both theory and programming. Handing in those assignments regularly will earn you a bonus for the final exam.

This is the syllabus for the Summer 2020 iteration of the course.

Lecture #	Date	Description	Our Slides	Videos
1 Introduction	April 22, 2020	Course Introduction, Basic ML Intoduction	.pdf	Intro, T. Mitchell, CMU Intro, A. Ihler, UCI
2 Linear Algebra	April 29, 2020	Vector, matrix, (Pseudo) Inverse, Eigenvectors, ...	.pdf	Matrix Algebra for Engineers, J. Chasnov, Udacity/HKUST Linear Algera Basics, C. Pryby, Udacity Eigenvalues and Eigenvectors, MIT 18.06C Linear Algebra
3 Statistics	May 6, 2020	Random Variables, Distributions, Moments, ...	.pdf	Probabilitiy distributions and their Properties, B. Carlson Random Var. & prob. dist., M. Rooduijn, E. van Loon, UoA Random Variables, Expection, Covariance, ..., IIT, India
4 Optimization	May 13, 2020	Convex, (Un)constraint Optimization, Lagragian Multiplier, Duality, Gradient descent, Newton, CG, ...	.pdf	Convex Probelems, T. Balch, A. Chakraborty, Georgia Tech Convexity, S. Dasgupta, UCSD Convexity and Optimization, A. Smola, CMU Duality, S. Dasgupta, UCSD Lagrangian Multipliers, G.J. Gordon, CMU
5 Bayesian Decision Theory	May 20, 2020	Class Conditional Probs, Class Prios, Bayesian Probs, Decision Boundary, Risk Minimization , ...	.pdf	Empirical and Expected Risk, Smola, UC Berkeley Risk and Loss Functions, Dini, Pravahan, AT&T Risk Minimization, Andy Park, Purdue
6 Probabilisty Density Estimation	May 27, 2020	Maximum Likelihood Estimation Bayesian Estimation, Nonparametric Density, Curse of Dimensionality, Expectation Maximization, ...	.pdf	Estimating Probabilities from Data, Weinberger, Cornell MLE Gaussian, Littman, Isbell, Kolhe, GeorgiaTech Curse of Dimesionality, Littman, Isbell, Kolhe, GeorgiaTech Mixture of Gaussians, Fox, U. Washington Expectation Maximization, Ng, Stanford kNN and Parzen Window, Lavrenko, U. Edingburgh Hyper Cube, Dini, Pravahan, AT&T
7a Expectation Maximization	June 10, 2020	Extra Slides on Expectation Maximization (EM), ...	.pdf	k-Means Clustering, Ihler, UCI Gaussian Mixtures, EM, Ihler, UCI Gaussian Mixtures, EM, Lavrenko, U. Edinburgh
7b Clustering and Evaluation	June 10, 2020	K-Means, Agglomerative Clustering, Mean Shift, Bias and Variance, ...	.pdf	Mean Shift, Bobick, Essa, Chakraborty, GeorgiaTech Overfitting, Ihler, UCI Bias and Variance , Ihler, UCI Bias and Variance, Ng, deeplearning.ai Cross-Validation, Thrun, Udacity
8 Regression	June 17, 2020	Linear Regression, ML Linear Regression, Bayesian Linear Regression ...	.pdf	Lineare Regression, Ihler, UCI ML and Linear Regression, de Freitas, UBC Regularization and Regression, de Freitas, UBC Bayesian Regression, Lawrence, Sheffield U.
9 Classification	June 24, 2020	Discriminant, Fisher's Linear Discriminant, Perceptron, Logistic Regression ...	.pdf	Lineare Classifiers: Basics, Ihler, UCI Lineare Classifiers: Learning Parameters, Ihler, UCI Intro to Linear Classificaiton, Jensen, TUM Linear Discriminant Models, Jensen, TUM Linear Discriminant Analyis, Starmer, StatQuest Linear Discriminant Analysis, Lavrenko, U. Edinburgh
10 Dim. Reduction & ERM	July 1, 2020	Projection, PCA, Empirical Risk Minimization (ERM), CV Dimension, ...	.pdf	PCA and SVD, Ihler, UCI VC Dimension, Ihler, UCI Generalization Bound, VC Dimension, Huang, Virginia Tech Approx./Estimation & ERM, Avati, Stanford
11 SVM	July 8, 2020	Hyperplane, Max Margin, Support Vector Machines, Kernels, Slack, ...	.pdf	Support Vector Classification, Smola, CMU SVM - Linear SVM/primal form, Ihler, UCI SVM - Lagrangian and Dual, Ihler, UCI SVM - Kernels, Ihler, UCI
12 Neural Networks	July 8, 2020	Activation Functions, Neural Networks (NN), Stochastic Gradient, Backpropagation, Convolutional NN, Drop Out, Adam ...	.pdf	Neural Networks, Ihler, UCI Backpropagation, Ihler, UCI Backpropagation, Lavrenko, U. Edinburgh Convolutional NNs, Soleimany, MIT Backpropagation, Yeung, Stanford Convolutional NNs, Yeung, Stanford
13 Gaussian Processes	July 15, 2020	Kernels Revisited, Dual of RBF Networks, Gaussian Processes, Bayesian learning ...	.pdf	Gaussian Process Regression, Smola, CMU Introduction to Gaussian Processes, Lawrence, U. Sheffield Gaussian Processes for Machine Learning, Cunningham, Cambridge Gaussian Prcoesses, Poupart, U. Waterloo Introduction to Gaussian Processes, de Freitas, UBC

Assignments and Exercises

Aside from the “5-questions” homework, there are also four written assignments involving both theory and programming. Handing in those assignments regularly will earn you a bonus for the final exam. Pleses visit Moodle for further information, and for submitting them.

Further Material and Links

• Online Mathematics for Ml book at https://mml-book.github.io/
• Model-based Machine Learning by John Winn an others