Our guest speaker is Arash Vahdat from NVIDIA Research and you are all cordially invited to the CVG Seminar on Feb 10th at 5:30 p.m. on Zoom (passcode is 908626).
Abstract
A wide variety of deep generative models has been developed in the past decade. Yet, these models often struggle with simultaneously addressing three key requirements including high sample quality, mode coverage, and fast sampling. We call the challenge imposed by these requirements the generative learning trilemma, as the existing models often trade between them. Particularly, denoising diffusion models (DDMs) have shown impressive sample quality and diversity, but their expensive sampling does not yet allow them to be applied in many real-world applications.
In this talk, I will cover our two recent works on reformulating DDMs specifically for fast sampling. In the first part, I will present LSGM, a framework that allows training DDMs in a latent space. In this work, we show that by mapping data to a latent space, we can learn smoother generative processes in a smaller space, resulting in fewer network evaluations and faster sampling. In the second part, I will present denoising diffusion GANs that model the denoising distribution in DDMs using conditional GANs. In this work, we show that our multi-modal denoising distributions, in contrast to unimodal Gaussian distributions, can reduce the number of denoising steps in DDMs to as few as two steps.
Project Pages
https://nvlabs.github.io/LSGM/
https://nvlabs.github.io/denoising-diffusion-gan/
Bio
Arash Vahdat is a principal research scientist at NVIDIA research specializing in machine learning and computer vision. Before joining NVIDIA, he was a research scientist at D-Wave Systems where he worked on deep generative learning and weakly supervised learning. Prior to D-Wave, Arash was a research faculty member at Simon Fraser University (SFU), where he led research on deep video analysis and taught graduate-level courses on big data analysis. Arash obtained his Ph.D. and MSc from SFU under Greg Mori’s supervision working on latent variable frameworks for visual analysis. His current areas of research include deep generative learning, weakly supervised learning, efficient neural networks, and probabilistic deep learning.