Machine Learning for Signal Processing

News
The beginning of this course has been postponed by one week. So the first lesson will be held on the 2nd of March.

General Information
Credits: 6 CFU
Scientific sector: ING-IND/31
Course language: English
Offered programs: Laurea Magistrale in Ingegneria Elettronica (LMIE) and Master Degree in Electronics Engineering (MDEE). The course is also commuted for the Laurea Magistrale in Ingegneria delle Comunicazioni (MCOR). However, the course can be also freely chosen and attended by interested students from other degree programs of Sapienza University, upon approval of the study plan.
Classroom page: Download slides, videolectures, Python notebooks, homework, and additional material. Access code: sikidky.
Office hours: by appointment
Calendar: February 23 - May 19, 2023
Class timing:
  - Thursday 8:30-10:00, Aula 6, Building RM031, Via Eudossiana 18
  - Friday 8:30-10:45, Aula 6, Building RM031, Via Eudossiana 18

Course Description
Information conveyed in real-world signals may be often affected by noise, partially corrupted or even unavailable. Thus, extracting desired information from real-world signals can be even very complicated. Machine learning for signal processing (MLSP) is the science that deals with the development of efficient algorithms and models that are able to detect and unveil a possible hidden structure in signals, thus recovering a desired information. This process is autonomously and automatically performed by MLSP algorithms, by simply learning from the available data, which is the basis of any science related to artificial intelligence.

This course aims at presenting the main machine learning and deep learning paradigms and applying them for the processing of a variety of signals, including electrical and mechanical signalss, sensor signals, audio and speech, images, movies, music, biological and medical signals, among many others. The course is based on regular classroom lessons, which also include regular exercises in Python on practical problems.

Prerequisites
Most of the prerequisites will be briefly recalled in classes. However, basic knowledge of linear algebra, signal processing theory and stochastic processes are warmly recommended, as well as basic programming skills.

Exam Assessment
Exam grades (in thirties) will be based on a project and oral questions. The theoretical skills acquired by the student will be evaluated as well as the ability to apply and implement a specific methodology in a practical problem. Lode is given only to students presenting an outstanding project and theoretical preparation.

Program

• INTRODUCTION TO MACHINE LEARNING FOR SIGNAL PROCESSING. Course overview. Introduction to signals, time series, and possible representations. Elements of stochastic processes and signal processing. Introduction to machine learning computation. Examples of practical MLSP data and scenarios. 

• MACHINE LEARNING FUNDAMENTALS. Regression and classification problems. Optimization algorithms for deep learning. Propagation, computational graphs, and automatic differentiation. Multilayer perceptrons and backpropagation. Layers, blocks, and activations. Training networks.

• MLSP FOR IMAGES AND STRUCTURED SIGNALS. Convolutional neural networks (CNNs). Deep learning architectures based on blocks (VGG). Residual and dense networks. Advanced CNN architectures. Applicative examples.

• MLSP FOR SEQUENTIAL AND TIME-VARYING SIGNALS. Recurrent layers. Encoder-decoder architectures. Attention mechanism. Transformer. Applicative examples.

• FAST AND EFFICIENT MLSP. Low-complexity approaches and sparsification techniques. Model compression and acceleration. Quantized and efficient architectures. Applicative examples.

• APPLICATION EXAMPLES. Applicative examples using sensor signals, images, smart grids, biomedical signals, audio, speech and music, edge-AI, and IoT data.

Detailed program

Textbook and Material

Main textbooks:

• Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola, "Dive Into Deep Learning", 2020 (available online).

• Sergios Theodoridis, "Machine Learning: A Bayesian and Optimization Perspective", 2nd ed., Elsevier, 2020.

Further references:

• Kevin P. Murphy, "Probabilistic Machine Learning: An Introduction". The MIT Press, 2022 (available online).

• Sebastian Raschka and Vahid Mirjalili, "Python Machine Learning". Packt Publishing, (3rd. ed.), 2019.

• Christopher M. Bishop, "Pattern Recognition and Machine Learning". Springer, 2006 (available online).

• Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”. The MIT Press, 2016 (available online).

• Danilo Comminiello and José C. Príncipe (Eds.), "Adaptive Learning Methods for Nonlinear System Modeling". Elsevier, 2018.

Supplementary material (e.g., course slides, papers) will be provided by the instructor.

Exam dates

Students are reminded that exams are booked electronically via the INFOSTUD portal. Booking for extraordinary exam sessions is allowed only to part-time students, students enrolled in supplementary years or non-conventional paths, and graduating students.

Scheduled exam sessions for the year 2022/2023:

• Session I: June 19, 2023 - TBD

• Session II: July 12, 2023 - TBD

• Session III: September 11, 2023 - TBD

• Extraordinary Session I: October 20, 2023 - TBD

• Session IV: Mid January, 2024 - TBD

• Session V: Mid February, 2024 - TBD

• Extraordinary Session II: End March, 2024 - TBD

Past academic years

• MLSP 2021/2022

• MLSP 2020/2021

• MLSP 2019/2020

• MLSP 2018/2019

• MLSP 2017/2018

About Machine Learning for Signal Processing

Here is some video by the IEEE Signal Processing Society about typical MLSP applications.

In the next video Prof. Bernard Widrow introduces ADALINE and talks about a "machine that learns from its own experience".