Distributional Shrinkage I: Universal Denoisers in Multi-Dimensions

Download:

• Paper
• arXiv

Abstract:

We revisit the problem of denoising from noisy measurements where only the noise level is known, not the noise distribution. In multi-dimensions, independent noise $Z$ corrupts the signal $X$, resulting in the noisy measurement $Y = X + \sigma Z$, where $\sigma \in (0, 1)$ is a known noise level. Our goal is to recover the underlying signal distribution $P_X$ from denoising $P_Y$. We propose and analyze universal denoisers that are agnostic to a wide range of signal and noise distributions. Our distributional denoisers offer order-of-magnitude improvements over the Bayes-optimal denoiser derived from Tweedie’s formula, if the focus is on the entire distribution $P_X$ rather than on individual realizations of $X$. Our denoisers shrink $P_Y$ toward $P_X$ optimally, achieving $O(\sigma^4)$ and $O(\sigma^6)$ accuracy in matching generalized moments and density functions. Inspired by optimal transport theory, the proposed denoisers are optimal in approximating the Monge-Ampère equation with higher-order accuracy, and can be implemented efficiently via score matching.

Let $q$ represent the density of $P_Y$; for optimal distributional denoising, we recommend replacing the Bayes-optimal denoiser, $$ \mathbf{T}^*(y) = y + \sigma^2 \nabla \log q(y), $$ with denoisers exhibiting less aggressive distributional shrinkage, $$ \mathbf{T}_1(y) = y + \frac{\sigma^2}{2} \nabla \log q(y), $$ $$ \mathbf{T}_2(y) = y + \frac{\sigma^2}{2} \nabla \log q(y) - \frac{\sigma^4}{8} \nabla \left( \frac{1}{2} | \nabla \log q(y) |^2 + \nabla \cdot \nabla \log q(y) \right) . $$

Citation

Tengyuan Liang. 2025. “Distributional Shrinkage I: Universal Denoisers in Multi-Dimensions.” arXiv:2511.09500.

@misc{liang2025distributional,
      title={Distributional Shrinkage I: Universal Denoisers in Multi-Dimensions}, 
      author={Tengyuan Liang},
      year={2025},
      eprint={2511.09500},
      archivePrefix={arXiv},
      primaryClass={stat.ML},
      url={https://arxiv.org/abs/2511.09500}, 
}

Related material