figure 7. Denoising experience on a natural image. from left to right and from top to bottom: noisy image (standard deviation s = 20), Gaussian convolution (h = 1. 8), anisotropic filter (h = 2. 4), total variation (l = 0.04), the Yaroslavsky neighborhood filter (r = 7, h = 28), and the nL-means algorithm. Parameters have been set for each algorithm so that the removed quadratic energy is equal to the energy of the added noise.

figure 8. Denoising experience on a natural image. from left to right and from top to bottom: noisy image (standard deviation 35), total variation minimization, neighborhood filter, translation invariant wavelet thresholding, DCt sliding window Wiener filter, and nL-means.

to be the best way to characterize artifact-free algorithms. The transformation of a white noise into any correlated signal creates structure and artifacts. Only white noise is perceptually devoid of visual structure, a principle first stated by Attneave. 4 Figure 9 shows how denoising methods transform a white noise.

The convolution with a Gauss kernel is equivalent to the product in the Fourier domain with a Gauss kernel of inverse standard deviation. Therefore, convolving the noise with a kernel reinforces the low frequencies and cancels the high ones. Thus, the filtered noise actually shows big grains due to its prominent low frequencies.

Noise filtered by a wavelet thresholding is no more a white noise. The few coefficients with a magnitude larger than the threshold are spread all over the image. The pixels that do not belong to the support of one of these coefficients are set to zero. The visual result is a constant image with superposed wavelets as displayed in Figure 9. It is easy to prove that the denoised noise is spatially highly correlated.

Given a noise realization, the filtered value by the neighborhood filter at a pixel i only depends on its value n(i) and the parameters h and r. The neighborhood filter averages noise values at a distance from n(i) less or equal than h. Thus, when the size r of the neighborhood increases, by the law of large numbers the filtered value tends to the expectation of the Gauss distribution restricted to the interval (n(i)− h, n(i)+ h). This filtered value is therefore a deterministic function of n(i) and h. Independent random variables are mapped by a deterministic function on independent variables. Thus the noise-to-noise requirement is asymptotically satisfied by the neighborhood filter. NL-means satisfies the noise-to-noise principle in a similar manner. However, a mathematical statement and proof of this property are more intricate and we shall skip them.

numerical Comparison: Table 1 displays the mean square error for the denoising experiments given in the paper. This numerical measurement is the most objective one, since it does not rely on any visual interpretation. However, this error is not computable in a real problem and a small mean square error does not assure a high visual quality. So all above-discussed criteria seem necessary to compare the performance of denoising algorithms.

6. extensIons sInCe 2005

Our conclusions on the better denoising performance of nonlocal methods with respect to former state-of-the-art algorithms such as the total variation or the wavelet thresholding have been widely accepted. The “method noise” methodology to compare the denoising performance has been adopted ever since.

The NL-means algorithm is easily extended to the denoising of image sequences and video. The denoising algorithm involves indiscriminately pixels not belonging only to same frame but also to all frames in the image.