Machine Learning for Signal Processing
A.Y. 2019/2020
A.Y. 2019/2020
General Information
Credits: 6 CFU
Scientific sector: ING-IND/31
Course language: English
Offered programs: Laurea Magistrale in Ingegneria Elettronica (LMIE) and Master Degree in Electronics Engineering (MDEE). The course is also commuted for the Laurea Magistrale in Ingegneria delle Comunicazioni (MCOR). However, the course can be also freely chosen and attended by interested students from other degree programs of Sapienza University.
Calendar: February 27 - May 29, 2020
Class timing: Thursday 8:30-10:00, Friday 8:30-10:45 - Room 6
Office hours: by appointment
News and updates: Interested students are advised to send an email to subscribe to the MLSP course mailing list (or directly join the list by Google Groups) and receive any kind of communication related to the course.
Moodle page: download lectures slides, lab session notebooks, homeworks and videos on Moodle.
Google Meet: tfm-hvsi-zvs
Course Description
Information conveyed in real-world signals may be often affected by noise, partially corrupted or even unavailable. Thus, extracting desired information from real-world signals can be even very complicated. Machine learning for signal processing (MLSP) is the science that deals with the development of efficient algorithms and models that are able to detect and unveil a possible hidden structure in signals, thus recovering a desired information. This process is autonomously and automatically performed by MLSP algorithms, by simply learning from the available data, which is the basis of any science related to artificial intelligence.
This course aims at presenting the main machine learning paradigms and applying them for the processing of a variety of signals, including audio and speech, images, movies, music, biological, electrical and mechanical, among many others. The course is based on regular classroom lessons, which also include regular exercises in Python on practical problems.
Prerequisites
Most of the prerequisites will be briefly recalled in classes. However, basic knowledge of linear algebra, signal theory and stochastic processes are warmly recommended, as well as basic programming skills.
Exam assessment and grade evaluation
Final projects will be assigned to small teams of students. The theoretical skills acquired by the student will be evaluated as well as the ability to apply and implement a specific methodology in a practical problem. Exam grades (in thirties) will be based on homework assignments (30%) and final project (70%).
Program
Introduction to MLSP. Using machine learning methods signal processing problems. Introduction to signals, time series and possible representations. Introduction to regression and classification problems. MLSP taxonomy. Typical applicative examples in MLSP.
MLSP basics. Brief recall of the main concepts of linear algebra, probability and random variables, distributions, stochastic processes, information theory, related to MLSP. Practical examples using typical MLSP data and scenarios.
Learning in parametric modeling. Parametric estimation. Bias-variance dilemma. Deterministic and stochastic estimation. Cost functions and optimization for machine learning. Normal equations. Least-square optimal estimation. Linear and logistic regression. Applicative examples.
Nonlinear learning algorithms. Nonlinear modeling and estimation. Nonlinear online learning algorithms. Learning with kernels. Regression and classification using kernels. Kernel adaptive filters. Applicative examples.
Neural networks and deep learning for signal processing. Introduction to neural networks, universal approximation, backpropagation, feed-forward neural networks. Recurrent neural networks for signal processing. Long short-term memory models. Deep learning basics. Convolutional neural networks. Advanced deep neural networks. Examples with application to audio/image/music/biomedical signals.
Textbook and Material
Main textbook:
Sergios Theodoridis, "Machine Learning: A Bayesian and Optimization Perspective". Elsevier, 2015.
Further references:
Kevin P. Murphy, "Machine Learning: A Probabilistic Perspective". The MIT Press, 2012.
Aston Zhang and Zachary C. Lipton and Mu Li and Alexander J. Smola, "Dive Into Deep Learning", 2020 (free pdf available online).
Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”. The MIT Press, 2016 (free pdf available online).
Christopher M. Bishop, "Pattern Recognition and Machine Learning". Springer, 2006 (free pdf available online).
Aurelio Uncini, "Introduction to Adaptive Algorithms and Machine Learning", 2018 (printed copies available at AZ Centro Stampa).
Danilo Comminiello and JosĂ© C. PrĂncipe (Eds.), "Adaptive Learning Methods for Nonlinear System Modeling". Elsevier, 2018.
Supplementary material (e.g., course slides, papers) will be provided by the instructor.
Lectures 2019/2020
[L01] Machine Learning for Signal Processing: A Course Introduction. [February 27, 2020] (pdf)
[L02] Taxonomy of Learning Approaches: From Human to Machine Learning. [February 28, 2020] (pdf)
[L03] Learning Tasks in Machine Learning for Signal Processing. [March 12, 2020] (pdf)
[L04] A Python Introduction to Machine Learning for Signal Processing. [March 13, 2020] (pdf)
[L05] Elements of Linear Algebra for MLSP. [March 19/20, 2020] (pdf)
[L06] Probability and Random Variables. [March 26, 2020] (pdf)
[L07] Learning in Parametric Modeling. [March 27, 2020] (pdf)
[L08] Linear Regression. [April 2, 2020] (pdf)
[L09] Applications of Linear Regression. [April 16, 2020] (pdf)
[L10] Stochastic Gradient Descent. [April 17, 2020] (pdf)
[L11] Introduction to Classification: Naive Classifiers. [April 23/24, 2020] (pdf)
[L12] Logistic Regression. [April 30, 2020] (pdf)
[L13] A Tour of Classic Machine Learning Classification Algorithms. [May 7, 2020] (pdf)
[L14] Regularization and Sparsity-Aware Learning. [May 14, 2020] (pdf)
[L15] Nonlinear Modeling and Learning with Kernels. [May 15, 2020] (pdf)
[L16] From Shallow to Deep Learning. [May 21/22, 2020] (pdf)
[L17] Exam Projects for MLSP 2019/2020. [May 28, 2020] (pdf)
[L18] Convolutional Neural Networks. [June 4, 2020] (pdf)
[L19] Long Short-Term Memory Networks. [June 5, 2020] (pdf)
[L20] Dimensionality Reduction. [Additional material] (pdf)
Lab Sessions 2019/2020
*Lab notebooks are available on the Moodle page of the course on the Sapienza e-Learning platform.
[EX01] Introduction to Python. [March 13, 2019]
[EX02] Elements of Linear Algebra and its Application to Signals. [March 19, 2020]
[EX03] Linear Algebra Operations in Signal Processing Problems. [March 20, 2020]
[EX04] Probability and Statistical Averages. [March 27, 2020]
[EX05] Least-Square Linear Regression. [April 3, 2020]
[EX06] Noise Cancellation. [April 17, 2020]
[EX07] Naive Classifiers. [April 24, 2020]
[EX08] Logistic Regression in NumPy. [April 30, 2020]
[EX09] Machine Learning Classifiers Using Scikit-learn. [May 8, 2020]
[EX10] Playlist Classification on Spotify. [May 8, 2020]
[EX11] Nonlinear Learning with Kernels. [May 15, 2020]
[EX12] Implementation of Multilayer Perceptron from Scratch. [May 22, 2020]
[EX13] Building Neural Networks with TensorFlow. [May 29, 2020]
[EX14] Deep Convolutional Neural Networks. [June 5, 2020]
[EX15] Blind Source Separation. [Additional material]
[EX16] Generative Adversarial Networks. [Additional material by Eleonora Grassucci]
Exam dates
Students are reminded that exams are booked electronically via the INFOSTUD portal. Booking for extraordinary exam sessions is allowed only to part-time students, students enrolled in supplementary years or non-conventional paths, and graduating students.
Scheduled exam sessions for the year 2019/2020:
Session I: June 30, 2020
Session II: July 17, 2020
Session III: September 10, 2020
Extraordinary Session I: October 20, 2020
Session IV: January 14, 2021
Session V: February 15, 2021
Extraordinary Session II: April 8, 2021
Past academic years
MLSP 2017/2018