Residual Non-local Attention Networks for Image Restoration

Image restoration is a central low-level vision problem. SRCNN and ARCNN introduced CNNs to super-resolution and compression-artifact reduction; DnCNN used residual learning; IRCNN incorporated denoiser priors; MemNet and encoder–decoder designs explored deeper memory and skip connections. These methods demonstrated the power of CNNs, but most rely on local convolution and do not explicitly model distant similarity, channel dependencies, or unequal restoration difficulty across image regions.

The degradation process is irreversible, making restoration an ill-posed inverse problem. A local receptive field cannot efficiently exploit repeated structures far apart in an image, especially under severe noise. Flat regions are easier to restore than textures, yet conventional models often process them similarly. Channels may also encode useful high-frequency details or corruption, so treating all channels equally can erase structure or preserve artifacts.

• ��A Trunk–Mask design separates hierarchical feature extraction from adaptive weighting.
• ��The local Mask uses large-stride convolution and deconvolution, enlarging context without the detail loss associated with repeated max pooling.
• ��The non-local Mask inserts an Embedded Gaussian Non-local Block, allowing every position to compare with all positions.
• ��Residual attention learning uses HtrunkHmask+x, rather than the classification-oriented Htrunk(Hmask+1), better preserving low-level restoration information.

• ��The input IL passes through shallow convolution; the output reconstructs IR, with IR=HRNAN(IL) and global residual learning focused on degradation.
• ��Each block has q=2 surrounding Residual Blocks; its Trunk contains t=2 simplified RBs, each using two 3×3 convolutions and one ReLU.
• ��The Mask uses m=1 RB, strided convolution, 2m RBs, transposed convolution, more RB processing, a 1×1 convolution, and Sigmoid.
• ��The NLB computes Softmax-weighted global affinity, with g(xj)=Wgxj and residual output Wzyi+xi.
• ��The model contains 10 attention blocks, including 2 RNABs; RB width is 64, while NLB channel dimension C is 32.

The models are trained on 800 DIV2K images using 16 random 48×48 LQ patches per batch. ADAM uses β1=0.9, β2=0.999, ε=10^-8, and an initial learning rate of 10^-4 halved every 200,000 back-propagation iterations; the objective is L2. Denoising uses Kodak24, BSD68, and Urban100 with AWGN σ=10/30/50/70. Demosaicing uses McMaster18, Kodak24, BSD68, and Urban100. Compression reduction uses LIVE1 and Classic5 with JPEG quality 10/20/30/40. PSNR and SSIM are reported.

RNAN obtains the best reported color and grayscale denoising results across the listed datasets and noise levels. At grayscale σ=70, it reaches 25.89 dB on Urban100, versus 27.45 dB for color. Demosaicing reaches 43.09/0.9902 on Kodak24 and 42.50/0.9929 on BSD68. For JPEG artifact reduction, RNAN is best on LIVE1 and Classic5 at all reported qualities. Ablations show independent gains from the Mask Branch, Non-local Block, and additional Residual Attention Blocks.

RNAN can support camera denoising, smartphone enhancement, Bayer demosaicing, JPEG post-processing, and super-resolution. Deployment requires training or fine-tuning on degradation statistics matching the target device. The architecture is a useful controlled-setting baseline because it handles several restoration tasks with one design, but practical products must measure latency, memory, and robustness to unknown corruption.

Dense global affinity is expensive in memory and computation, explaining the use of only two RNABs. The training distribution is dominated by DIV2K and synthetic AWGN/JPEG, so real sensor noise, motion blur, and mixed corruption may cause domain shift. L2 optimization favors average pixel fidelity and does not fully capture perceptual texture quality. Efficient attention, real-degradation modeling, blind restoration, and standardized quality–speed–energy evaluation are natural next steps.