Distributional Shrinkage

We revisit the classic signal denoising problem through the lens of optimal transport. We introduce a hierarchy of denoisers that are agnostic to the signal distribution, depending only on higher-order score functions of the noisy observations. Each denoiser is progressively refined using higher-order score functions, achieving better denoising quality measured by the Wasserstein metric. The limiting denoiser identifies the optimal transport map for signal denoising. Our results connect information geometry, optimal transport, and advanced combinatorics.

Empirical Bayes tends to produce overly aggressive shrinkage as a denoiser. We introduce new denoisers that optimally shrink the distribution toward the true signal distribution with order-of-magnitude improvements. Unlike empirical Bayes denoiser, our denoisers are universal and agnostic to the signal and noise distributions. One immediate application of our distributional shrinkage theory is to enhance generative modeling: we can replace the stochastic backward diffusion process with optimal deterministic denoisers to achieve higher-order accuracy.