Machine Learning for Safety Systems

Course Description and Motivation

The course objective is to provide a general overview of the modern techniques of Machine Learning and their applicability to safety systems. In addition to the description of the foundations of Machine Learning, the course provides the necessary background in order to understand and apply Machine Learning approaches to classification, regression and clustering techniques to solve practical problems in different applicative scenarios by mean of neural networks and other learning techniques. During the course, it will also describe the use of specific software packages, such as WEKA, for the implementation, use and validation of the modern Machine Learning techniques. At the end of the course students will be able to handle different Machine Learning models, to tune them to specific applications, and to design approaches that may scale with large amount of data.

• Knowledge and understanding: to know the problems, methodologies and applications of Machine Learning techniques.

• Applying knowledge and understanding: to implement different classification, regression and clustering algorithms to solve problems in different applicative scenarios.

• Making judgements: to develop adequate critical skills through practical activities in implementing peculiar simulative algorithms and interpreting the obtained results.

• Communication skills: to improve ability to critically expose the matters learned during the course.

• Learning skills: to improve autonomous and independent study capacity.

Scientific Sector (SSD)

ING-IND/31 (Electrical Engineering)

Overview and Credits

MSZR; First year; Second semester. 6 CFUs.

Prerequisites

Students are expected to have the following background:

• Basic linear algebra

• Basic optimization theory

• Basic probability theory

• (Optional but preferred) Knowledge of basic computer programming, at a level sufficient to write a reasonably non-trivial computer program

Grading

Project. Students are requested to request a dataset via email and prepare the project according to the template provided with the dataset. The discussion of the project can be carried out at students’ early convenience. The formal exam registration on Infostud system will take place on the dates indicated below.

Syllabus

1. Course overview

2. Introduction to Machine Learning: motivation to Machine Learning, Data Mining, challenges and application examples.

3. Signal and sequence representation.

4. Basics on probability theory.

5. Basics on estimation theory. Maximum likelihood, MAP, MMSE and MAE estimation.

6. Basics on optimization theory. Unconstrained and constrained optimization: Lagrangian function and KKT conditions. The regularization.

7. Meta-heuristics: genetic algorithms, Random Search, Simulated Annealing and Tabu Search.

8. Data preparation. Pre-processing: mean removal, de-trending and dimensionality reduction. SVD, PCA, LDA e CCA. Outlier removal.

9. Probability density estimation: parametric and non-parametric methods. Parzen windowing.

10. Instance-based learning. The k-NN algorithm. Bayesian classification and Naive Bayes classifier. Associacion rules.

11. Linear learning: optimal estimate and LMS algorithm. Logistic regression.

12. Testing Machine Learning algorithms. Indicators: confusion matrix, True Positive Rate, False Positive Rate, Precision, Recall, F-Measure, AUC, ROC analysis. K-fold Cross Validation, One Leave Out. Grid search.

13. Decision trees: ID3, CART and C4.5. Rule extraction.

14. Neural Networks: the neuron, the perceptron, the MLP, deep networks. The back propagation algorithm.

15. Kernel Machines: the RBF and the Support Vector Machine (SVM).

16. Combining several algorithms: voting, bagging, boosting, mixture of experts. Random Forest

17. Clustering techniques: K-means, EM algorithm, hierarchical clustering.

18. Ethics in Machine Learning.

19. WEKA: description and usage of the software. The arff file format. Examples of applications.

20. Hints of Python programming for Machine Learning algorithm development. Examples of applications.

21. Python for Machine Learning. Advanced topics on Python to work with ML applications.

22. Reinforcement Learning.

23. Monte Carlo methods. Sampling.

24. Graphical models: directed and undirected models. Inference and diagnosis.

25. Hidden Markov Models (HMMs).

26. Fuzzy logic and fuzzy reasoning. Fuzzy vs. crisp. Fuzzyfication and defuzzyfication, membership function. Mamdani and Sugeno rules.

Detailed contents.

Suggested Books

• Chirag Shah, A Hands-On Introduction to Machine Learning, Cambridge University Press, 2022.

• Stephen Marsland, Machine Learning: An Algorithmic Perspective, 2nd edition, Chapman and Hall/CRC, 2014.

• Sebastian Raschka, Vahid Mirjalili, Python Machine Learning - Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow, Second Edition, Packt Publishing, 2017. (Practical)

• Sebastian Raschka, Vahid Mirjalili, Machine Learning con Python - Costruire algoritmi per generare conoscenza, Seconda edizione, Apogeo, 2020. (Practical in Italian)

• Lecture notes.
  
  

Further reading

• Ethem Alpaydin, Introduction to Machine Learning, 3rd edition, MIT Press, 2014.

• Ian H. Witten, Eibe Frank, Mark A. Hall, Christopher J. Pal, Data Mining: Practical Machine Learning Tools and Techniques, 4th edition, Morgan Kaufmann, 2016.

• Andriy Burkov, The Hundred-Page Machine Learning Book, 2019.

• Jeremy Watt, Reza Borhani, Aggelos K. Katsaggelos, Machine Learning Refined: Foundations, Algorithms, and Applications, Cambridge University Press, 2016.

• Sergios Theodoridis, Machine Learning: A Bayesian and Optimization Perspective, Academic Press, 2015.

• Kevin P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012.

• Jiawei Han, Micheline Kamber, Jian Pei, Data Mining: Concepts and Techniques, 3rd edition, Morgan Kauffmann, 2012.

• Naomi Ceder, The Quick Python Book,  3rd Edition, ‎ Manning, 2018.

• Naomi Ceder, Python - Guida alla sintassi, alle funzionalità avanzate e all'analisi dei dati, Apogeo, 2019. (in Italian)

• Andreas C. Müller, Sarah Guido, Introduction to Machine Learning with Python, O'Reilly Media, 2016. (Practical)

• Aurélien Géron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, O'Reilly Media, 2nd edition, 2019. (Practical)

Time and Location

The course will start on February 28 2024, with the following schedule:

• Wednesday, 15:00 - 18:00, Room 34

• Thursday, 12:00 - 14:00, Room 20

Office Hours

Every day by appointment.

Classroom

News, updates, and communications about the course will be available on Google Classroom, with the code: 6fpeh7p.

Course Materials

Lecture Slides:

1. Overview

2. Introduction

3. Signal and sequence representation

4. Basics on probability theory

5. Basics on estimation theory

6. Basics on optimization theory

7. Meta-heuristics

8. Data preparation

9. Probability density estimation

10. Instance-based algorithms

11. Linear models

12. Testing ML algorithms

13. Decision trees

14. Neural networks

15. Kernel machines and SVM

16. Ensemble learning

17. Clustering algorithms

18. Ethics in Machine Learning

19. WEKA

20. Python

21. Python for Machine Learning

22. Reinforcement learning

23. Monte Carlo methods

24. Graphical models

25. Hidden Markov models

26. Fuzzy logic

Python Source Codes:

• Example source code

Lecture Notes:

• Note 1

• Note 2

Exams Session

The discussion of the project can be held at any time, by appointment. The formal verbalization takes place on the dates indicated below:

• I round: 13 June 2024 at 14:00 (Exam reservation date ends on 12 June 2024).

• II round: 18 July 2024 at 14:00 (Exam reservation date ends on 17 July 2024).

• III round: 16 September 2024, at 14:00 (Exam reservation date ends on 15 September 2024).

• Special round: 24 October 2024, at 14:00 (Exam reservation date ends on 23 October 2024). Please note that this round is reserved only for part-time, off-course, repeating and graduating students.

• IV round: 23 January 2025 at 14:00 (Exam reservation date ends on 22 January 2025).

• V round: 13 February 2025 at 14:00 (Exam reservation date ends on 12 February 2025).

• Special round: 03 April 2025 at 14:00 (Exam reservation date ends on 02 April 2025). Please note that this round is reserved only for part-time, off-course, repeating and graduating students.

-------------------------------------------------------------------------------------------------------------------------------------------------

• I round: 12 June 2025 at 14:00 (Exam reservation date ends on 11 June 2025).

• II round: 17 July 2025 at 14:00 (Exam reservation date ends on 16 July 2025).

• III round: 17 September 2025, at 14:00 (Exam reservation date ends on 16 September 2025).

• Special round: 29 October 2025, at 14:00 (Exam reservation date ends on 28 October 2025). Please note that this round is reserved only for part-time, off-course, repeating and graduating students.

• IV round: 22 January 2026 at 14:00 (Exam reservation date ends on 21 January 2026).

• V round: 12 February 2026 at 14:00 (Exam reservation date ends on 11 February 2026).

• Special round: 09 April 2026 at 14:00 (Exam reservation date ends on 08 April 2026). Please note that this round is reserved only for part-time, off-course, repeating and graduating students.

 Please, note that the exam is reserved electronically via the INFOSTUD system.