- English
- Italiano

**Lecture September 24, 2019**- (by Marco Boresta) Questionnaire for checking background (here the results). Description of the exams and phyton background.**Lecture September 25, 2019**- Introduction to the course. The learning process. Supervised and unsupervised paradigms. Data (trainig testing and validation) (Ref. slide 1_lecture, Bishop – Pattern Recognition and Machine Learning, Springer, 2006 (Chap 1), V. Cherlassky, F. Mulier - Learning from Data, John Wiley and Sons, 2007 (chap.1 and 2) , Bottou, Curtis, and Nocedal - Optimization Methods for LargeScale Machine Learning (section 1 and 2), Teaching Notes chapt. 1)Minimization of the expected risk. Over and under fitting. Hint on Vapnik Chervonenkis theory and Structural risk minimization principle. VC dimension of oriented hyperplanes (The theorem - NO proof) . Hyperplanes with margin - (Ref. slide_2_lecture, Teaching Notes chapt. 1, P. Domingos - A few useful things to know about Machine Learning)**Lecture October 1, 2019 -**The formal neuron and the percepron algorithm (theorem on finite convergence with proof). The voting and the average perceptron Example: the logical OR. The voting and average perceptron. (Ref. slide_3_lecture, Teaching Notes chap. 2)**Lecture October 2, 2019 -****Lecture October 8, 2019**- Beyond perceptron. Feedforward Neural Network: Multiplayer perceptron (Ref. Slide 4th Lecture)**Lecture October 9, 2019**- Hyperparameters and weights problems (Ref. Slide 6th Lecture, [L. Palagi, 2019 - Global optimization issues in deep network regression: an overview] -Section 4) . Python (Numpy) by Marco Boresta (Ref. Homework 1)**Lecture October 15, 2019**- Short review of optimization: basic definitions and line search algorithms. (Ref. D. Bertsekas, Nonlinear Programming- 2nd ed., L. Grippo and M.- Sciandrone, Metodi di ottimizzazione non vincolata)**Lecture October 16, 2019**- Feedforward Neural Network: Multiplayer perceptron: shallow netowrk (Ref. Slide 8th Lecture, , Teaching notes chapt. 3). Python (Numpy) by Marco Boresta (Ref. Homework 2 - For advanced topic: question 3 of Perceptron Project or the Rosembrock's function)**Lecture October 22, 2019**- The output of a Deep Betwrk: forward propagation (Ref. Slide 9th Lecture, Teaching notes chapt. 3)**Lecture October 23, 2019**- Evaluation test in the class. Basics of gradient batch methods (direction, line searches) (Ref. Slide 10th lecture)**Lecture October 29, 2019**- Back propagation procedure. Convergence of the batch gradient method with constant stepsize (learning rule) (Ref. Slide 11th lecture)**Lecture October 30, 2019**- Seminar by Ruggiero Seccia on IBM Watson. Short review of Phyton routines for optimization (Ref. Slide 12th lecture - Phyton). Beyond Batch gradient: samplewise decomposition (Ref. Slide 12th lecture - Methods. In depth reading: Optimization methods for large-scale machine learning by L. Bottou, FE Curtis, J Nocedal). Early stopping rules (Ref. Slide 12th lecture - Early stopping- In depth reading: Early stopping-but when? by Prechelt, Lutz)**Lecture November 5, 2019**- Beyond Batch gradient: - Decomposition methods for MLP: Extreme learning (Ref. Slide of the 13th lecture, teaching notes chapt. 3)- Lecture November 6, 2019 - Decomposition methods for MLP (Ref. Slide of the 14th lecture)
- Lecture November 12, 2019 - Radial Basis Function Networks: regularized and generalized RBF network. The XOR example (Ref: Slide of the 15th lecture; Girosi, F. and Poggio, T., 1990. Networks and the best approximation property. Biological cybernetics, 63(3), pp.169-176; Teaching notes chapt. 4)
- Lecture November 13, 2019 - Unsupervised selection of centers. Supervised selection of the centers: Full optimization. Two block decompostion methods, exact and inexact solution of subproblems, convergence properties. Decomposition methods: block learning of centers (Ref. Slide of the 16th lecture, Teaching notes chapt. 4). Discussion on the project 1 with Marco Boresta.
**MIDTERM November 19, 2019**- Lecture November 20, 2019 - Beyond vanilla gradient: monentum term, averaging itateraion, diagonal scaling (Ref. Slide of the 18th lecture). Seminar on Reinforcment Learning by Tommaso Colombo
- Lecture November 26, 2019 - Hard SVM: generalities, defintion of margin, defintion of the max margin problem (Ref. Slide 19th lecture). The primal hard SVM problem
- Lecture November 27, 2019 - Soft SVM: generalities, the primal soft SVM problem. Convex optimization: KKT conditions for linearly constrained porblems. Feasible and descent directions. Frank-wolfe conditional gradient
- Lecture November 27, 2019 - S
- Lecture December 3, 2019 - Quadratic constrained optimzation in Phyton (Marco Boresta). Multiclass classification problems: One against one and one against all
- Lecture December 4, 2019 - Duality in convex quadratic programming. The weak duality theorem (with proof). The Wolfe dual. Construction of the dual Hard and Soft SVM problem.
- Lecture December 10, 2019 - The KKT conditions for the dual SVM problem (eliminating the multipliers).
- Lecture December 11, 2019 - Decomposition algorithms for SVM: the case q=2 (Analytic solution of QP in two dimension) and q>2. Defintion of kernels and examples on the LIBSVM (REF: Slide of the lectures on kernels and LIBSVM graphical interface).
- Lecture December 12, 2019 - (room B2 -11:00-12:30) Exercise on SVM
- Lecture December 17, 2019 - Final term
- Lecture December 18, 2019 - Debriefing on the project