Generative Models

We revisit the classic signal denoising problem through the lens of optimal transport. We introduce a hierarchy of denoisers that are agnostic to the signal distribution, depending only on higher-order score functions of the noisy observations. Each denoiser is progressively refined using higher-order score functions, achieving better denoising quality measured by the Wasserstein metric. The limiting denoiser identifies the optimal transport map for signal denoising. Our results connect information geometry, optimal transport, and advanced combinatorics.

Empirical Bayes tends to produce overly aggressive shrinkage as a denoiser. We introduce new denoisers that optimally shrink the distribution toward the true signal distribution with order-of-magnitude improvements. Unlike empirical Bayes denoiser, our denoisers are universal and agnostic to the signal and noise distributions. One immediate application of our distributional shrinkage theory is to enhance generative modeling: we can replace the stochastic backward diffusion process with optimal deterministic denoisers to achieve higher-order accuracy.

We introduce a novel generative modeling framework called parabolic Monge-Ampère PDE sampler. We establish theoretical guarantees for generative modeling through the lens of no-regret analysis, demonstrating that the iterates converge to the optimal Brenier map under a variety of step-size schedules. We derive a new Evolution Variational Inequality connecting geometry, transportation cost, and regret.

Adding noise is easy; what about denoising? Diffusion is easy; what about reverting a diffusion? We provide a fine-grained analysis of the diffuse-then-denoise process. We discover a notion of multi-scale curvature complexity that collectively determines the success or failure mode of probabilistic diffusion models.

Motivated by the seminal work on distance and isomorphism between metric measure spaces, we propose a new notion called the Reversible Gromov-Monge (RGM) distance and study how RGM can be used to design new transform samplers to perform simulation-based inference.

Motivated by the pursuit of a systematic computational and algorithmic understanding of Generative Adversarial Networks (GANs), we present a simple yet unified non-asymptotic local convergence theory for smooth two-player games, which subsumes several discrete-time gradient-based saddle point dynamics. The analysis reveals the surprising nature of the off-diagonal interaction term as both a blessing and a curse.

This paper studies the rates of convergence for learning distributions implicitly with the adversarial framework and Generative Adversarial Networks (GANs), which subsume Wasserstein, Sobolev, MMD GAN, and Generalized/Simulated Method of Moments (GMM/SMM) as special cases. We study a wide range of parametric and nonparametric target distributions under a host of objective evaluation metrics. We investigate how to obtain valid statistical guarantees for GANs through the lens of regularization.