Download:
Abstract:

We revisit the problem of denoising from noisy measurements where only the noise level is known, not the noise distribution. In multi-dimensions, independent noise $Z$ corrupts the signal $X$, resulting in the noisy measurement $Y = X + \sigma Z$, where $\sigma \in (0, 1)$ is a known noise level. Our goal is to recover the underlying signal distribution $P_X$ from denoising $P_Y$. We propose and analyze universal denoisers that are agnostic to a wide range of signal and noise distributions. Our distributional denoisers offer order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie’s formula, if the focus is on the entire distribution $P_X$ rather than on individual realizations of $X$. Our denoisers shrink $P_Y$ toward $P_X$ optimally, achieving $O(\sigma^4)$ and $O(\sigma^6)$ accuracy in matching generalized moments and density functions. Inspired by optimal transport theory, the proposed denoisers are optimal in approximating the Monge-Ampère equation with higher-order accuracy, and can be implemented efficiently via score matching.

Let $q$ represent the density of $P_Y$; for optimal distributional denoising, we recommend replacing the Bayes-optimal denoiser, $$ \mathbf{T}^*(y) = y + \sigma^2 \nabla \log q(y), $$ with denoisers exhibiting less aggressive distributional shrinkage, $$ \mathbf{T}_1(y) = y + \frac{\sigma^2}{2} \nabla \log q(y), $$ $$ \mathbf{T}_2(y) = y + \frac{\sigma^2}{2} \nabla \log q(y) - \frac{\sigma^4}{8} \nabla \left( \frac{1}{2} | \nabla \log q(y) |^2 + \nabla \cdot \nabla \log q(y) \right) . $$

Citation

Tengyuan Liang. 2025. “Distributional Shrinkage I: Universal Denoiser Beyond Tweedie’s Formula.” arXiv:2511.09500.

@misc{Liang2025DS1,
  title = {Distributional Shrinkage {I}: Universal Denoiser Beyond Tweedie's Formula},
  author = {Liang, Tengyuan},
  year = {2025},
  eprint = {2511.09500},
  archivePrefix = {arXiv},
  primaryClass = {stat.ML},
  url = {https://arxiv.org/abs/2511.09500},
}

This work is part I of a two-part series on distributional shrinkage; part II can be found here .

Previously circulated as “Distributional Shrinkage I: Universal Denoisers in Multi-Dimensions” (arXiv:2511.09500v1 ).