Machine Learning at the Flatiron Institute Seminar: Michael Albergo

Title: Frontiers of dynamical control of generative models

Abstract: Flow and diffusion models have become cornerstones of both scientific and industrial generative AI research. These methods work by construction of a dynamics that maps samples from a reference distribution to samples from a target distribution known empirically through data. An open question is how to best control and modify these dynamics so as to satisfy specified target sampling constraints, often specified by a reward or tilting function. I will provide an overview of the mathematics underlying this construction and discuss some of our recent efforts to make this sort of reward alignment highly scalable, both during training via fine-tuning and during inference via steering. Along the way, I will argue that the flow map — the solution operator of these dynamics — is essential to performing efficient and scalable alignment, and, time permitting, I will discuss recent efforts to bring these advances to language modeling.