Machine Learning for Signal Processing

A.Y. 2021/2022

General Information
Teacher: Danilo Comminiello
Credits: 6 CFU
Scientific sector: ING-IND/31
Course language: English
Offered programs: Laurea Magistrale in Ingegneria Elettronica (LMIE) and Master Degree in Electronics Engineering (MDEE). The course is also commuted for the Laurea Magistrale in Ingegneria delle Comunicazioni (MCOR). However, the course can be also freely chosen and attended by interested students from other degree programs of Sapienza University, upon approval of the study plan.
Calendar: February 24 - May 20, 2022
Class timing: Thursday 8:30-10:00, Friday 8:30-10:45
In-person attendance: Classroom 6, Building RM031, Via Eudossiana 18
Remote attendance: Connect with your Sapienza account at this Zoom link (meeting ID: 871 4141 7046, passcode: 273830).
Classroom page: Download slides, videolectures, Python notebooks, homeworks and additional material. Access code: yi2f4cq.
Office hours: by appointment

Course Description
Information conveyed in real-world signals may be often affected by noise, partially corrupted or even unavailable. Thus, extracting desired information from real-world signals can be even very complicated. Machine learning for signal processing (MLSP) is the science that deals with the development of efficient algorithms and models that are able to detect and unveil a possible hidden structure in signals, thus recovering a desired information. This process is autonomously and automatically performed by MLSP algorithms, by simply learning from the available data, which is the basis of any science related to artificial intelligence.

This course aims at presenting the main machine learning and deep learning paradigms and applying them for the processing of a variety of signals, including electrical and mechanical signalss, sensor signals, audio and speech, images, movies, music, biological and medical signals, among many others. The course is based on regular classroom lessons, which also include regular exercises in Python on practical problems.

Prerequisites
Most of the prerequisites will be briefly recalled in classes. However, basic knowledge of linear algebra, signal processing theory and stochastic processes are warmly recommended, as well as basic programming skills.

Exam Assessment
Exam grades (in thirties) will be based on two homework assignments (40%) and final written test (60%). A project can be request as an alternative to the homeworks. The theoretical skills acquired by the student will be evaluated as well as the ability to apply and implement a specific methodology in a practical problem. Lode is given only to students presenting an outstanding project with the maximum score at the written test.

Program

INTRODUCTION TO MACHINE LEARNING FOR SIGNAL PROCESSING. Course overview. Introduction to signals, time series and possible representations. Elements of stochastic processes and signal processing. Introduction to machine learning computation. Examples of practical MLSP data and scenarios.
MACHINE LEARNING FUNDAMENTALS. Regression and classification problems. Optimization algorithms for deep learning. Propagation, computational graphs and automatic differentiation. Multilayer perceptrons and backpropagation. Layers, blocks and activations. Training networks.
MLSP FOR IMAGES AND STRUCTURED SIGNALS. Convolutional neural networks (CNNs). Deep learning architectures based on blocks (VGG). Residual and dense networks. Advanced CNN architectures. Applicative examples.
MLSP FOR SEQUENTIAL AND TIME-VARYING SIGNALS. Recurrent layers. Encoder-decoder architectures. Attention mechanism. Transformer. Applicative examples.
FAST AND EFFICIENT MLSP. Low-complexity approaches and sparsification techniques. Model compression and acceleration. Quantized and efficient architectures. Applicative examples.
APPLICATION EXAMPLES. Applicative examples using sensor signals, images, biomedical signals, audio, speech and music, edge-AI and IoT data.

Detailed program

Textbook and Material

Main textbooks:

Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola, "Dive Into Deep Learning", 2020 (available online).
Sergios Theodoridis, "Machine Learning: A Bayesian and Optimization Perspective", 2nd ed., Elsevier, 2020.

Further references:

Kevin P. Murphy, "Probabilistic Machine Learning: An Introduction". The MIT Press, 2022 (available online).
Sebastian Raschka and Vahid Mirjalili, "Python Machine Learning". Packt Publishing, (3rd. ed.), 2019.
Christopher M. Bishop, "Pattern Recognition and Machine Learning". Springer, 2006 (available online).
Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”. The MIT Press, 2016 (available online).
Danilo Comminiello and José C. Príncipe (Eds.), "Adaptive Learning Methods for Nonlinear System Modeling". Elsevier, 2018.

Supplementary material (e.g., course slides, papers) will be provided by the instructor.

Exam dates

Students are reminded that exams are booked electronically via the INFOSTUD portal. Booking for extraordinary exam sessions is allowed only to part-time students, students enrolled in supplementary years or non-conventional paths, and graduating students.

Scheduled exam sessions for the year 2021/2022: