Learning Deep CNN Denoiser Prior for Image Restoration

Image restoration seeks x from y=Hx+v. The MAP formulation combines data fidelity, 1/2||y-Hx||², with a prior λΦ(x). BM3D, WNNM, NCSR, and related optimization methods handle many operators H but rely on costly iterations. MLP, SRCNN, and TNRD infer quickly yet are usually trained for a particular task. Plug-and-play work showed that variable splitting can turn a denoising routine into a reusable prior, motivating a learned CNN alternative.

The challenge is to obtain a prior that is powerful, fast, color-aware, and reusable across inverse problems. TV, GMM, K-SVD, non-local means, and BM3D can create artifacts, incur substantial computation, or oversmooth structures lacking self-similarity. Color methods often depend on hand-designed transforms. The paper asks whether a discriminative CNN denoiser bank can be inserted into an optimization framework and solve deblurring or super-resolution without task-specific end-to-end retraining.

First, IRCNN uses dilated convolutions to obtain a 33×33 receptive field with only seven layers; a same-depth standard-convolution network provides 15×15. Second, batch normalization, ReLU, and residual learning accelerate and stabilize training. Third, 25 models cover σ=0–50 at interval 2, making iterative denoising approximately matched to the current penalty. Fourth, the CNN is used as a modular HQS prior rather than being trained separately for every degradation operator.

• �� Modeling: formulate y=Hx+v and minimize 1/2||y-Hx||²+λΦ(x).
• �� Splitting: introduce z=x and optimize Lμ=1/2||y-Hx||²+λΦ(z)+μ/2||z-x||².
• �� x-update: solve ||y-Hx||²+μ||x-z||²; the direct solution is (HTH+μI)⁻¹(HTy+μz), with FFT for convolutional blur.
• �� z-update: call Denoiser(x,√(λ/μ)); the explicit prior is replaced implicitly by IRCNN.
• �� Architecture: seven 3×3 dilated-convolution layers with rates 1,2,3,4,3,2,1; 64 feature maps in middle layers; batch normalization and ReLU.
• �� Training: 35×35 patches, N=256×4000 samples, Adam, batch size 256, and residual loss 1/(2N)Σ||f(yi;Θ)−(yi−xi)||². Adjacent noise models use transfer initialization.

Training data contain 400 BSD images, 400 ImageNet validation images, and 4,744 Waterloo Exploration Database images. BSD68 grayscale and color denoising are evaluated with BM3D, WNNM, TNRD, MLP, and CBM3D using PSNR and runtime. Deblurring uses a Gaussian kernel with standard deviation 1.6 and two real kernels from [38], with varying Gaussian noise; comparisons include MLP, IDDBM3D, NCSR, and EPLL. HQS runs for 30 iterations, with denoiser levels decaying from 49 to 1–15.

IRCNN scores 31.63, 29.15, and 26.19 dB on BSD68 at σ=15, 25, and 50, respectively. On color BSD68 at σ=25 it reaches 31.16 dB, 0.45 dB above CBM3D. In the receptive-field ablation, seven standard-convolution layers obtain 28.94 dB, compared with 29.15 for IRCNN and 29.20 for a 16-layer standard network. At 1024×1024, GPU runtime is 0.059 seconds for 24 channels and 0.146 seconds for 64 channels.

For deblurring, the HQS x-step enforces the blur kernel and observation, while the CNN step restores natural structures; FFT makes convolutional updates efficient under circular boundaries. Color IRCNN jointly processes RGB, whereas IDDBM3D and MLP handle channels separately. The same design also applies to super-resolution, inpainting, and other low-level problems that admit a data-consistency subproblem plus denoising.

The method assumes additive Gaussian noise and a known or tractable degradation operator, with a manually selected HQS schedule. Although 25 models are fewer than training networks for every degradation, they still require storage and roughly three days of training. Experiments do not extensively cover real noise, blind degradation, extreme structures, or large-scale deployment. Because the CNN need not correspond to an explicit Φ, convergence, fixed-point behavior, and stability under denoiser mismatch warrant further analysis.