Machine Learning for Signal Processing
A.Y. 2021/2022
General Information
Teacher: Danilo Comminiello
Credits: 6 CFU
Scientific sector: ING-IND/31
Course language: English
Offered programs: Laurea Magistrale in Ingegneria Elettronica (LMIE) and Master Degree in Electronics Engineering (MDEE). The course is also commuted for the Laurea Magistrale in Ingegneria delle Comunicazioni (MCOR). However, the course can be also freely chosen and attended by interested students from other degree programs of Sapienza University, upon approval of the study plan.
Calendar: February 24 - May 20, 2022
Class timing: Thursday 8:30-10:00, Friday 8:30-10:45
In-person attendance: Classroom 6, Building RM031, Via Eudossiana 18
Remote attendance: Connect with your Sapienza account at this Zoom link (meeting ID: 871 4141 7046, passcode: 273830).
Classroom page: Download slides, videolectures, Python notebooks, homeworks and additional material. Access code: yi2f4cq.
Office hours: by appointment
Course Description
Information conveyed in real-world signals may be often affected by noise, partially corrupted or even unavailable. Thus, extracting desired information from real-world signals can be even very complicated. Machine learning for signal processing (MLSP) is the science that deals with the development of efficient algorithms and models that are able to detect and unveil a possible hidden structure in signals, thus recovering a desired information. This process is autonomously and automatically performed by MLSP algorithms, by simply learning from the available data, which is the basis of any science related to artificial intelligence.
This course aims at presenting the main machine learning and deep learning paradigms and applying them for the processing of a variety of signals, including electrical and mechanical signalss, sensor signals, audio and speech, images, movies, music, biological and medical signals, among many others. The course is based on regular classroom lessons, which also include regular exercises in Python on practical problems.
Prerequisites
Most of the prerequisites will be briefly recalled in classes. However, basic knowledge of linear algebra, signal processing theory and stochastic processes are warmly recommended, as well as basic programming skills.
Exam Assessment
Exam grades (in thirties) will be based on two homework assignments (40%) and final written test (60%). A project can be request as an alternative to the homeworks. The theoretical skills acquired by the student will be evaluated as well as the ability to apply and implement a specific methodology in a practical problem. Lode is given only to students presenting an outstanding project with the maximum score at the written test.
Program
INTRODUCTION TO MACHINE LEARNING FOR SIGNAL PROCESSING. Course overview. Introduction to signals, time series and possible representations. Elements of stochastic processes and signal processing. Introduction to machine learning computation. Examples of practical MLSP data and scenarios.
MACHINE LEARNING FUNDAMENTALS. Regression and classification problems. Optimization algorithms for deep learning. Propagation, computational graphs and automatic differentiation. Multilayer perceptrons and backpropagation. Layers, blocks and activations. Training networks.
MLSP FOR IMAGES AND STRUCTURED SIGNALS. Convolutional neural networks (CNNs). Deep learning architectures based on blocks (VGG). Residual and dense networks. Advanced CNN architectures. Applicative examples.
MLSP FOR SEQUENTIAL AND TIME-VARYING SIGNALS. Recurrent layers. Encoder-decoder architectures. Attention mechanism. Transformer. Applicative examples.
FAST AND EFFICIENT MLSP. Low-complexity approaches and sparsification techniques. Model compression and acceleration. Quantized and efficient architectures. Applicative examples.
APPLICATION EXAMPLES. Applicative examples using sensor signals, images, biomedical signals, audio, speech and music, edge-AI and IoT data.
Textbook and Material
Main textbooks:
Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola, "Dive Into Deep Learning", 2020 (available online).
Sergios Theodoridis, "Machine Learning: A Bayesian and Optimization Perspective", 2nd ed., Elsevier, 2020.
Further references:
Kevin P. Murphy, "Probabilistic Machine Learning: An Introduction". The MIT Press, 2022 (available online).
Sebastian Raschka and Vahid Mirjalili, "Python Machine Learning". Packt Publishing, (3rd. ed.), 2019.
Christopher M. Bishop, "Pattern Recognition and Machine Learning". Springer, 2006 (available online).
Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”. The MIT Press, 2016 (available online).
Danilo Comminiello and José C. Príncipe (Eds.), "Adaptive Learning Methods for Nonlinear System Modeling". Elsevier, 2018.
Supplementary material (e.g., course slides, papers) will be provided by the instructor.
Exam dates
Students are reminded that exams are booked electronically via the INFOSTUD portal. Booking for extraordinary exam sessions is allowed only to part-time students, students enrolled in supplementary years or non-conventional paths, and graduating students.
Scheduled exam sessions for the year 2021/2022:
Session I: June 22, 2022 - DIET Reading Room, 10:00
Session II: July 14, 2022 - Room 16, 10:00
Session III: September 12, 2022 - Room 13, 10:00
Extraordinary Session I: October 20, 2022
Session IV: January 24, 2023 - 15:00
Session V: February 14, 2023 - 15:00
Extraordinary Session II: March 20, 2023 - 12:00
Past academic years
MLSP 2017/2018
About Machine Learning for Signal Processing
Here is some video by the IEEE Signal Processing Society about typical MLSP applications.
In the next video Prof. Bernard Widrow introduces ADALINE and talks about a "machine that learns from its own experience".
...and some fun
Enjoy the Machine Listening playlist on Spotify (...for well-trained ears only!)