Abstract
The Wiener estimator is relatively simple to analyze because it is a linear operator. The optimization of nonlinear estimators is much more complex. However, quasi-optimal nonlinear estimators can be characterized when the data have a sparse decomposition in an orthonormal basis. In this case, we show that a quasi-optimal estimator can be obtained using an algorithm that thresholds the decomposition coefficients in the basis. The squared error of the estimate is related to an error obtained when approximating a signal using its largest decomposition coefficients in the orthonormal basis. The challenge, then, is to find an orthonormal basis that minimizes this nonlinear approximation error, which amounts to maximizing the rate of decay of the signal’s decomposition coefficients in the basis. This question was at the center of research in harmonic analysis from the 1990s to the 2010s.
Beyond the Fourier basis, orthogonal wavelet bases have made it possible to obtain quasi-optimal, sparse approximations for piecewise-smooth signals and images. This led to the first nonlinear thresholding denoising algorithms in wavelet bases. However, these thresholding algorithms perform significantly worse than deep neural networks.
