Lunch available beginning at 12:15 PM
Speaker to begin at 12:30 PM

Abstract: Diffusion models represent a significant breakthrough in generative AI, operating by progressively transforming random noise distributions into structured outputs, with adaptability for specific tasks through guidance or fine-tuning. In this presentation, we delve into the statistical and optimization aspects of diffusion models and establish their connection to theoretical optimization frameworks. In the first part, we explore how unconditioned diffusion models efficiently capture complex high-dimensional data, particularly when low-dimensional structures are present. We present the first efficient sample complexity bound for diffusion models that depend on the small intrinsic dimension. Moving to the second part, we leverage our understanding of diffusion models to introduce a pioneering optimization method termed "generative optimization." Here, we harness diffusion models as data-driven solution generators to maximize any user-specified reward function. We introduce gradient-based guidance to guide the sampling process of a diffusion model while preserving the learnt low-dim data structure.  We show that adapting a pre-trained diffusion model with guidance is essentially equivalent to solving a regularized optimization problem. Further we consider adaptively fine-tuning the score network using new samples together with gradient guidance. This process mimics a first-order optimization iteration, for which we established an O(1/T) converge rate to the global optimum. Moreover, these solutions maintain fidelity to the intrinsic structures within the training data, suggesting a promising avenue for optimization in complex, structured spaces through generative AI.

Contributions to and/or sponsorship of any event does not constitute departmental or institutional endorsement of the specific program, speakers or views presented.