Abstract
The impact of AI on science is illustrated this year by two Nobel Prizes: one awarded to Hinton in Physics for deep learning, and another awarded to Demis Hassabis and John Jumper in Chemistry for the AlphaFold2 algorithm, which predicts the three-dimensional structure of proteins.
This year’s course focuses on data generation, which is an unsupervised problem where one must estimate a probability distribution based on n data points x of dimension d, which are considered independent samples. We must then sample from this distribution to generate new data. For high-dimensional data (d), in the absence of strong assumptions, the curse of high dimension predicts that the number of data points n must grow exponentially with d in order to accurately estimate the probability distribution. The central question is how to avoid this curse when possible, and with which algorithms.
This introduction provides an overview of several possible approaches that will be studied. Energy models use exponential distributions of maximum entropy, whose Gibbs energy is characterized by a finite number of moments. A completely different approach has been introduced in AI with “Generative Adversarial Networks” (GANs), which pit a generative neural network against a discriminative network. The course also examines “Normalizing Flows,” which compute the generation via a transport process of a simpler distribution—such as a Gaussian—using a sequence of parameterized, invertible operators. The principles of distribution transport are reviewed using high-dimensional ordinary differential equations (ODEs) and stochastic differential equations (SDEs). The state of the art is currently achieved using “score diffusion” algorithms via denoising and stochastic interpolators, whose generalization properties will be studied.
