Machine Learning for Signal Processing
A.Y. 2023/2024
News
The course will start on Thursday, March 7, 2024.
General Information
Credits: 6 CFU
Scientific sector: ING-IND/31
Course language: English
Offered programs: Laurea Magistrale in Ingegneria Elettronica (LMIE) and Master Degree in Electronics Engineering (MDEE). The course is also commuted for the Laurea Magistrale in Ingegneria delle Comunicazioni (MCOR). However, the course can be also freely chosen and attended by interested students from other degree programs of Sapienza University, upon approval of the study plan.
Classroom page: Download slides, videolectures, Python notebooks, homework, and additional material. Access code: cyqsj25.
Office hours: by appointment
Calendar: February 29 - May 31, 2024
Class timing:
- Thursday 8:30-10:00, Aula 32, Via Eudossiana 18
- Friday 8:30-10:45, Aula 23, Via Eudossiana 18
Course Description
Information conveyed in real-world signals may be often affected by noise, partially corrupted or even unavailable. Thus, extracting desired information from real-world signals can be even very complicated. Machine learning for signal processing (MLSP) is the science that deals with the development of efficient algorithms and models that are able to detect and unveil a possible hidden structure in signals, thus recovering a desired information. This process is autonomously and automatically performed by MLSP algorithms, by simply learning from the available data, which is the basis of any science related to artificial intelligence.
This course aims at presenting the main machine learning and deep learning paradigms and applying them for the processing of a variety of signals, including electrical and mechanical signalss, sensor signals, audio and speech, images, movies, music, biological and medical signals, among many others. The course is based on regular classroom lessons, which also include regular exercises in Python on practical problems.
Prerequisites
Most of the prerequisites will be briefly recalled in classes. However, basic knowledge of linear algebra, signal processing theory and stochastic processes are warmly recommended, as well as basic programming skills.
Exam Assessment
Exam grades (in thirties) will be based on a project and oral questions. The theoretical skills acquired by the student will be evaluated as well as the ability to apply and implement a specific methodology in a practical problem. Lode is given only to students presenting an outstanding project and theoretical preparation.
Program
INTRODUCTION TO MACHINE LEARNING FOR SIGNAL PROCESSING. Course overview. Introduction to signals, time series, and possible representations. Elements of stochastic processes and signal processing. Introduction to machine learning computation. Examples of practical MLSP data and scenarios.
MACHINE LEARNING FUNDAMENTALS. Regression and classification problems. Optimization algorithms for deep learning. Propagation, computational graphs, and automatic differentiation. Multilayer perceptrons and backpropagation. Layers, blocks, and activations. Training networks.
MLSP FOR IMAGES AND STRUCTURED SIGNALS. Convolutional neural networks (CNNs). Deep learning architectures based on blocks (VGG). Residual and dense networks. Advanced CNN architectures. Applicative examples.
MLSP FOR SEQUENTIAL AND TIME-VARYING SIGNALS. Recurrent layers. Encoder-decoder architectures. Attention mechanism. Transformer. Applicative examples.
FAST AND EFFICIENT MLSP. Low-complexity approaches and sparsification techniques. Model compression and acceleration. Quantized and efficient architectures. Applicative examples.
APPLICATION EXAMPLES. Applicative examples involving sensor signals, images, smart grids, biomedical signals, audio, speech and music, edge-AI, and IoT data.
Textbook and Material
Main textbooks:
Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola, "Dive Into Deep Learning", 2020 (available online).
Sergios Theodoridis, "Machine Learning: A Bayesian and Optimization Perspective", 2nd ed., Elsevier, 2020.
Further references:
Kevin P. Murphy, "Probabilistic Machine Learning: An Introduction". The MIT Press, 2022 (available online).
Sebastian Raschka and Vahid Mirjalili, "Python Machine Learning". Packt Publishing, (3rd. ed.), 2019.
Christopher M. Bishop, "Pattern Recognition and Machine Learning". Springer, 2006 (available online).
Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”. The MIT Press, 2016 (available online).
Danilo Comminiello and José C. Príncipe (Eds.), "Adaptive Learning Methods for Nonlinear System Modeling". Elsevier, 2018.
Supplementary material (e.g., course slides, papers) will be provided by the instructor.
Exam dates
Students are reminded that exams are booked electronically via the INFOSTUD portal. Booking for extraordinary exam sessions is allowed only to part-time students, students enrolled in supplementary years or non-conventional paths, and graduating students.
Scheduled exam sessions for the year 2022/2023:
Session I: 17 June 2024 (time and classroom To Be Confirmed)
Session II: 11 July 2024 (TBC)
Session III: 12 September 2024 (TBC)
Extraordinary Session I: 23 October 2024 (TBC)
Session IV: 16 January 2025 (TBC)
Session V: 10 February 2025 (TBC)
Extraordinary Session II: 26 March 2025 (TBC)
Past academic years
About Machine Learning for Signal Processing
Here is some video by the IEEE Signal Processing Society about typical MLSP applications.
In the next video Prof. Bernard Widrow introduces ADALINE and talks about a "machine that learns from its own experience".