Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement

Low-light image enhancement has evolved from simple histogram equalization and gamma correction to sophisticated deep learning models. Early methods lacked semantic understanding, often causing artifacts. Retinex-based approaches introduced decomposition into reflectance and illumination, improving perceptual quality but still suffering from multi-stage training and noise amplification. Recent CNN models like RetinexNet and KinD improved performance but struggled with long-range dependencies and noise suppression. Transformer models, known for capturing global context, have been under-explored due to computational costs. This work aims to bridge this gap by integrating Transformer mechanisms into a Retinex framework, addressing both efficiency and robustness.

Existing low-light enhancement methods face challenges such as complex multi-stage training, limited ability to model long-distance dependencies, and poor noise suppression, especially in severely corrupted scenes. CNN-based models cannot effectively capture global context, leading to artifacts and detail loss. Traditional Retinex methods assume noise-free illumination maps, which is unrealistic. Therefore, designing a unified, end-to-end model that efficiently models long-range interactions, suppresses noise, and handles diverse lighting conditions remains an open problem.

The paper introduces a one-stage Retinex framework (ORF) that models corruptions via perturbation terms, enabling joint training of illumination and noise suppression. The core innovation is the Illumination-Guided Transformer (IGT), which employs IG-MSA to guide self-attention with illumination features, capturing long-range dependencies efficiently. This approach reduces computational complexity from quadratic to linear relative to spatial size, allowing integration at multiple scales. The architecture simplifies training, improves detail recovery, and effectively suppresses noise and artifacts, setting a new benchmark in low-light enhancement.

• �� Use convolutional layers to estimate illumination maps and extract light-up features from input images.
• �� Introduce perturbation terms to model noise and color distortions, enabling joint end-to-end training.
• �� Design a U-shaped IGT with multiple IGAB units, each containing IG-MSA guided by illumination features.
• �� Implement IG-MSA by reshaping features into tokens, computing scaled dot-product attention with illumination guidance, and concatenating multi-head outputs.
• �� Employ skip connections and multi-scale fusion to refine restored images.
• �� Optimize the entire network with MAE loss, balancing detail preservation and noise suppression.
• �� Validate on multiple datasets, including LOL, SID, SMID, and SDSD, with ablation studies confirming each component’s contribution.

Experiments involve training on diverse datasets with varied lighting conditions, using data augmentation and hyperparameter tuning. The evaluation metrics include PSNR and SSIM, with comparisons against CNN-based and Transformer-based SOTA models. Ablation studies demonstrate the effectiveness of IG-MSA and illumination guidance. The model’s robustness is tested in noisy, overexposed, and underexposed scenarios, showing consistent superiority. Computational efficiency is analyzed through FLOPS and parameter count, confirming the lightweight design. Cross-dataset validation confirms generalization capabilities.

Retinexformer achieves an average PSNR increase of over 4dB across 13 datasets, with improvements exceeding 6dB in noisy scenarios like SID and SDSD. It outperforms DeepUPE, RetinexNet, and Restormer by significant margins, with fewer parameters and FLOPS. Visual results show clearer, more natural images with less noise and color distortion. User studies indicate higher perceptual quality scores, and low-light object detection accuracy improves notably, demonstrating practical utility. Ablation results confirm the importance of illumination-guided attention and multi-scale fusion.

The method can be directly applied to night-time surveillance, autonomous driving, and smartphone photography, where robust low-light enhancement is critical. It requires minimal training data and can be deployed on edge devices for real-time processing. The improved image quality benefits downstream tasks like recognition, tracking, and scene understanding. Long-term, the framework can be extended to multi-modal inputs, such as infrared or depth data, further enhancing performance in challenging environments.

Despite its strengths, the model struggles in extreme overexposure or underexposure scenarios, where details are heavily lost. Its performance under rapid lighting changes or dynamic scenes needs further validation. The reliance on large annotated datasets may limit applicability in data-scarce domains. Computational costs, though reduced, still pose challenges for real-time deployment on very resource-constrained devices. Future work should focus on adaptive mechanisms and unsupervised training to address these issues.