Machine Learning for Signal Processing
A.Y. 2018/2019
A.Y. 2018/2019
General Information
Credits: 6 CFU
Scientific sector: ING-IND/31
Course language: English
Offered programs: Laurea Magistrale in Ingegneria Elettronica (LMIE) and Master Degree in Electronics Engineering (MDEE). The class can be also freely chosen and attended by interested students from other degree programs of Sapienza University.
Calendar: February 28 - May 31, 2019
Class timing: Thursday 8:30-10:00, Room 6 - Friday 8:30-10:45, Room 32
Course Description
Information conveyed in real-world signals may be often affected by noise, partially corrupted or even unavailable. Thus, extracting desired information from real-world signals can be even very complicated. Machine learning for signal processing (MLSP) is the science that deals with the development of efficient algorithms and models that are able to detect and unveil a possible hidden structure in signals, thus recovering a desired information. This process is autonomously and automatically performed by MLSP algorithms, by simply learning from the available data, which is the basis of any science related to artificial intelligence.
This course aims at presenting the main machine learning paradigms and applying them for the processing of a variety of signals, including audio and speech, images, movies, music, biological, electrical and mechanical, among many others. The course is based on regular classroom lessons, which also include regular exercises in Python on practical problems.
Prerequisites
Most of the prerequisites will be briefly recalled in classes. However, basic knowledge of linear algebra, signal theory and stochastic processes are warmly recommended, as well as basic programming skills.
Exam assessment and grade evaluation
Final projects will be assigned to small teams of students. The theoretical skills acquired by the student will be evaluated as well as the ability to apply and implement a specific methodology in a practical problem. Exam grades (in thirties) will be based on homework assignments (30%) and final project (70%).
Program
Introduction to MLSP. Using machine learning methods signal processing problems. Introduction to signals, time series and possible representations. Introduction to regression and classification problems. MLSP taxonomy. Typical applicative examples in MLSP.
MLSP basics. Brief recall of the main concepts of linear algebra, probability and random variables, distributions, stochastic processes, information theory, related to MLSP. Practical examples using typical MLSP data and scenarios.
Learning in parametric modeling. Parametric estimation. Bias-variance dilemma. Deterministic and stochastic estimation. Cost functions and optimization for machine learning. Normal equations. Least-square optimal estimation. Linear and logistic regression. Applicative examples.
Nonlinear learning algorithms. Nonlinear modeling and estimation. Nonlinear online learning algorithms. Learning with kernels. Regression and classification using kernels. Kernel adaptive filters. Applicative examples.
Neural networks and deep learning for signal processing. Introduction to neural networks, universal approximation, backpropagation, feed-forward neural networks. Recurrent neural networks for signal processing. Long short-term memory models. Deep learning basics. Convolutional neural networks. Advanced deep neural networks. Examples with application to audio/image/music/biomedical signals.
Textbook and Material
Main textbook:
Sergios Theodoridis, "Machine Learning: A Bayesian and Optimization Perspective". Elsevier, 2015.
Further references:
Aurelio Uncini, "Introduction to Adaptive Algorithms and Machine Learning", 2018 (printed copies available at AZ Centro Stampa).
Kevin P. Murphy, "Machine Learning: A Probabilistic Perspective". The MIT Press, 2012.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”. The MIT Press, 2016 (free pdf available online).
Christopher M. Bishop, "Pattern Recognition and Machine Learning". Springer, 2006 (free pdf available online).
Danilo Comminiello and José C. Príncipe (Eds.), "Adaptive Learning Methods for Nonlinear System Modeling". Elsevier, 2018.
Supplementary material (e.g., course slides, papers) will be provided by the instructor.
Lectures 2018/2019
[L01/L02] Machine Learning for Signal Processing: A Course Introduction. [February 28/March 1, 2019] (pdf)
[L03] Taxonomy of Learning Approaches: From Human to Machine Learning. [March 7, 2019] (pdf)
[L04] Learning Tasks in Machine Learning for Signal Processing. [March 8, 2019] (pdf)
[L05/L06] Elements of Linear Algebra for MLSP. [March 14/15, 2019] (pdf)
[L07] Probability, Random Variables and Stochastic Processes. [March 21/22, 2019] (pdf)
[L08/L09] Learning in Parametric Modeling. [March 22/28, 2019] (pdf)
[L10] Elements of Parametric Estimation Theory. [March 29, 2019] (pdf)
[L11] Optimal Linear Filtering. [March 29, 2019] (pdf)
[L12] MSE Filtering Applications. [April 4, 2019] (pdf)
[L13] Stochastic Gradient Descent. [April 5, 2019] (pdf)
[L14] Stochastic-Gradient Adaptive Algorithms. [April 11, 2019] (pdf)
[L15] Hessian-based Adaptive Algorithms. [April 12, 2019] (pdf)
[L16] Dimensionality Reduction. [May 2, 2019] (pdf)
[L17] Classification: A Tour of the Classics. [May 3, 2019] (pdf)
[L18] Nonlinear Modeling and Learning with Kernels. [May 9, 2019] (pdf)
[L19] Regularization and Ridge Regression. [May 9, 2019] (pdf)
[L20] Kernel Methods for Classification and Online Learning. [May 10, 2019] (pdf)
[L21] Neural Networks. (Guest Lecturer: Simone Scardapane) [May 17, 2019] (pdf)
[L22] Introduction to Deep Learning. [May 24, 2019] (pdf)
[L23] Long Short-Term Memory Networks. [May 30, 2019] (pdf)
[L24] MLSP Projects. [May 31, 2019] (pdf)
Lab Sessions 2018/2019
[EX01] Lab Session 1: Introduction to Python. [March 8, 2019]
[EX02] Lab Session 2: Linear Algebra and its Application to Signals. [March 15, 2019]
[EX03] Lab Session 3: Empirical Estimation of Statistic Averages. [March 22, 2019]
[EX04] Lab Session 4: Least-Square Parametric Modeling. [March 28, 2019]
[EX05] Lab Session 5: Noise Cancellation by Optimal MSE Linear Filtering. [April 5, 2019]
[EX06] Lab Session 6: Adaptive System Identification. [April 12, 2019]
[EX07] Lab Session 7: Blind Source Separation. [May 2, 2019]
[EX08] Lab Session 8: Classification by Logistic Regression. [May 10, 2019]
[EX09] Lab Session 9: Introduction to TensorFlow. (Guest Lecturer: Simone Scardapane) [May 17, 2019]
[EX10] Lab Session 10: Neural Networks with TensorFlow. (Guest Lecturer: Simone Scardapane) [May 23, 2019]
[EX11] Lab Session 11: Convolutional Neural Networks. [May 24, 2019]
*Lab notebooks can be required by emal.
Exam dates
Students are reminded that exams are booked electronically via the INFOSTUD portal. Booking for extraordinary exam sessions is allowed only to part-time students, students enrolled in supplementary years or non-conventional paths, and graduating students.
Scheduled exam sessions for the year 2018/2019:
Session I (2018/2019): June 14, 2019
Session II (2018/2019): July 19, 2019
Session III (2018/2019): September 10, 2019
Extraordinary Session I (2018/2019): October 22, 2019
Session IV (2018/2019): January 22, 2020
Session V (2018/2019): February 12, 2020
Extraordinary Session II (2018/2019): April 8, 2020