LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models
LightenDiffusion combines Retinex theory with diffusion models for unsupervised low-light image enhancement, outperforming existing methods.
Key Findings
Methodology
LightenDiffusion performs Retinex decomposition in latent space, splitting low-light and normal-light image features into content-rich reflectance maps and content-free illumination maps. It uses diffusion models for unsupervised restoration and introduces self-constrained consistency loss to enhance visual quality.
Key Results
- On the LOL dataset, LightenDiffusion achieves a PSNR of 20.453 and SSIM of 0.803, surpassing all unsupervised methods.
- On the LSRW dataset, LightenDiffusion's PSNR and SSIM outperform supervised methods.
- In real-world scenarios, LightenDiffusion performs best on DICM, NPE, and VV datasets.
Significance
This study addresses visual quality issues in low-light image enhancement by integrating physically interpretable Retinex theory with diffusion models, improving algorithm generalization across various scenes.
Technical Contribution
Introduces a novel content-transfer decomposition network for latent space decomposition, avoiding content information loss in traditional methods, and generates high-quality images using diffusion models.
Novelty
First to perform Retinex decomposition in latent space combined with diffusion models for unsupervised image enhancement, significantly improving visual quality.
Limitations
- In complex scenes, illumination maps may still contain content information, affecting restoration results.
- Requires extensive unpaired data for training.
Future Work
Future work could explore further reducing content information in illumination maps and improving model training efficiency.
AI Executive Summary
Low-light images often suffer from poor visibility and noise, impacting subsequent vision tasks. Traditional methods struggle to adapt to diverse lighting conditions, while deep learning methods, despite progress, face overfitting issues. LightenDiffusion combines Retinex theory with diffusion models, performing decomposition in latent space to address these challenges. Experimental results show that this method surpasses existing unsupervised methods on multiple public benchmarks and, in some cases, outperforms supervised methods. Nonetheless, the method still faces challenges in complex scenes, and future research can further optimize model performance.
Deep Analysis
Background
Low-light image enhancement is a crucial problem in computer vision. Traditional methods like histogram equalization and Retinex theory have some effectiveness but struggle with diverse lighting conditions. Recently, deep learning methods have made significant progress by learning mappings from low-light to normal-light images, but overfitting limits their generalization.
Core Problem
The core problem in low-light image enhancement is improving visual quality without relying on paired data. Existing methods struggle to maintain high-quality restoration in complex scenes.
Innovation
LightenDiffusion performs Retinex decomposition in latent space, splitting image features into content-rich reflectance maps and content-free illumination maps, combined with diffusion models for unsupervised restoration, significantly improving visual quality.
Methodology
- �� Perform Retinex decomposition in latent space to generate reflectance and illumination maps
- �� Use diffusion models for unsupervised restoration
- �� Introduce self-constrained consistency loss to enhance visual quality
Experiments
Experiments are conducted on LOL and LSRW datasets using PSNR and SSIM for performance evaluation, and generalization tests on DICM, NPE, VV datasets. Adam optimizer is used for training.
Results
LightenDiffusion achieves a PSNR of 20.453 and SSIM of 0.803 on the LOL dataset, surpassing all unsupervised methods. On the LSRW dataset, performance exceeds most supervised methods.
Applications
This method can be used for image enhancement in low-light environments, such as night photography and surveillance camera image processing, improving visual quality.
Limitations & Outlook
In complex scenes, illumination maps may still contain content information, affecting restoration results. Future research could explore further reducing content information in illumination maps.
Plain Language Accessible to non-experts
Imagine taking a photo in a dimly lit room, where the image appears blurry and colors are distorted. LightenDiffusion acts like a smart light bulb, analyzing the room's lighting conditions and automatically adjusting the photo's brightness and contrast to make it look as if it was taken in a brightly lit room. It analyzes the photo's details and lighting information, removing unnecessary noise and distortion to make the image more vivid.
ELI14 Explained like you're 14
Imagine taking a photo in a dark room, and it looks all blurry. LightenDiffusion is like a super filter that can automatically analyze the lighting in the photo and adjust the brightness and contrast to make it look as if it was taken in a bright room! It's like a smart assistant that helps remove noise and distortion from the photo, making it more lively.
Glossary
Diffusion Model
A generative model that creates data by gradually adding noise.
Used for generating high-quality images.
Retinex Theory
An image processing theory that decomposes images into reflectance and illumination maps.
Used for image enhancement.
Self-constrained Consistency Loss
A loss function ensuring restored images remain consistent with input images.
Used to enhance visual quality.
Latent Space
A low-dimensional representation space of image features.
Used for Retinex decomposition.
Reflectance Map
Content information in images that remains unchanged under different lighting.
Used for preserving image details.
Open Questions Unanswered questions from this research
- 1 How to further reduce content information in illumination maps to improve restoration quality?
- 2 How to enhance model training efficiency for larger-scale data?
Applications
Immediate Applications
Night Photography
Enhance the quality of photos taken at night, making them clearer and more vivid.
Long-term Vision
Surveillance Camera Image Processing
Improve image quality from surveillance cameras in low-light environments, enhancing security.
Abstract
In this paper, we propose a diffusion-based unsupervised framework that incorporates physically explainable Retinex theory with diffusion models for low-light image enhancement, named LightenDiffusion. Specifically, we present a content-transfer decomposition network that performs Retinex decomposition within the latent space instead of image space as in previous approaches, enabling the encoded features of unpaired low-light and normal-light images to be decomposed into content-rich reflectance maps and content-free illumination maps. Subsequently, the reflectance map of the low-light image and the illumination map of the normal-light image are taken as input to the diffusion model for unsupervised restoration with the guidance of the low-light feature, where a self-constrained consistency loss is further proposed to eliminate the interference of normal-light content on the restored results to improve overall visual quality. Extensive experiments on publicly available real-world benchmarks show that the proposed LightenDiffusion outperforms state-of-the-art unsupervised competitors and is comparable to supervised methods while being more generalizable to various scenes. Our code is available at https://github.com/JianghaiSCU/LightenDiffusion.