Lost in Translation: Generative Models for Unpaired Data

News
The course will start on June 3, 2026.

General Information
Teachers: Danilo Comminiello and Eleonora Grassucci
Credits: 3 CFU
Course language: English
Offered programs: PhD Course in Information and Communication Technology (ICT) and National PhD in AI
Calendar: June 3, 4, 10, 11,  17, 18
Class timing: 10:00-13:00
Lecture modality: Hybrid: Aula 2 SPV for the days 3, 4, 17, and 18 June,  Aula 37 for the days 10 and 11 June/ Online via Zoom
Classroom page: Download slides, videolectures, notebooks, and additional material. Access code: 44gdfx3w. Participants are invited to register here.
Office hours: by appointment

Course Description
This PhD course investigates the principles and applications of unpaired data-to-data translation within the framework of modern generative deep learning. Unlike paired data settings, where inputs and outputs are explicitly aligned, unpaired data refers to collections of examples from different domains without one-to-one correspondence. This scenario arises naturally in practice: from dealing with missing or incomplete data, to enabling translation between modalities (e.g., medical scans, satellite vectors, multimedia content) without requiring expensive labeling or additional supervised training.

The course begins with the foundations of generative modeling, i.e., VAEs, GANs, diffusion models, and flow-matching approaches, then focuses on architectures and cost functions specifically designed for unpaired translation. Applications span a wide range of domains: from classical image-to-image translation and image modality translation for medical imaging (e.g., MRI–CT), to vector-to-image translation for satellite data, and cross-modal translation tasks such as video-to-audio generation or multimedia synthesis. By the end of the course, participants will acquire both theoretical understanding and practical insights, illustrated through state-of-the-art research examples and real-world case studies.

Application examples can be agreed upon with the teacher based on their own PhD research topics. Please inform the teacher in advance.

TensorFlow Faculty Award
The course of Generative Deep Learning has been awarded the TensorFlow Faculty Award 2021 from Google in support of the development of new teaching courses on emerging machine learning topics that also promote diversity initiatives aimed at widening access to machine learning education.

Prerequisites
Knowledge of machine learning is warmly recommended.

Program

• INTRODUCTION TO GENERATIVE MODELS AND UNPAIRED TRANSLATION. Generative modeling and probabilistic generative models. Learning latent representations. Introduction to Unpaired Data Translation.

• MAIN GENERATIVE MODELS. Autoregressive models. Generative Adversarial Networks. Variational Autoencoders. Diffusion Models. Flow-Matching Models. 

• GENERATIVE ARCHITECTURES FOR UNPAIRED DATA TRANSLATION. Typical problems in unpaired data translation. From CycleGAN to probabilistic architectures. Latent generative approaches.

• TRAINING GENERATIVE MODELS FOR UDT. Specific cost functions for unpaired translation (cycle-consistency loss, adversarial losses, style-based losses). Loss functions for diffusion and flow models (noise prediction loss, KL divergence, optimal transport, contrastive loss).

• APPLICATIONS AND CASE STUDIES. Examples of generative models for unpaired data translation, including: natural image-to-image translation, image style and domain translation, satellite image translation, music generation, voice conversion, image-to-text generation, video-to-audio generation, medical image modality translation, and brain activity translation, among others.

Class Schedule
The course will be held in June 2026 with the following schedule:

• Tuesday, June 3 10:00-13:00        - Aula 2 SPV

• Thursday, June 4            10:00-13:00        - Aula 2 SPV

• Tuesday, June 10            10:00-13:00        - Aula 37 SPV

• Thursday, June 11          10:00-13:00        - Aula 37 SPV

• Tuesday, June 17           10:00-13:00 - Aula 2 SPV

• Thursday, June 18          10:00-13:00 - Aula 2 SPV

The course will be held in a hybrid modality:

• In-person: Aula 2 SPV for the days 3, 4, 17, and 18 June,  Aula 37 for the days 10 and 11 June, SPV, Via Eudossiana 18

• Online: via Zoom at the following link:
  https://uniroma1.zoom.us/j/86596351314?pwd=T0pja1RKLzFTd2ZabWN0TS9yN2tMQT09

Final Examination
Small project assignment on one of the course topics. The topic can also be agreed based on the students' PhD research program.

Research ideas can be developed by group of students.

Textbooks and Material

Course slides, lecture notes, and lab notebooks by the instructor.

All the material can be found on the Classroom page of the course. Participants are invited to register.

Textbooks:

• Jakub M. Tomczak, "Deep Generative Modeling", 2nd Ed., Springer, 2024.

• Kevin P. Murphy, "Probabilistic Machine Learning: Advanced Topics", The MIT Press, 2022.

• David Foster, "Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play", O'Reilly Media, Inc., June 2019.

• TBD