FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information
FourLLIE enhances low-light images using Fourier frequency information, surpassing SOTA on four datasets with only 0.31% parameters.
Key Findings
Methodology
FourLLIE employs a two-stage framework: the frequency stage enhances brightness via amplitude transform maps in Fourier space, while the spatial stage integrates global frequency and local spatial information using SNR maps to restore details.
Key Results
- On the LOL-Real dataset, FourLLIE achieves a PSNR of 27.31, outperforming SNR-Aware by 1.2dB with only 0.31% of its parameters.
- On the LSRW-Huawei dataset, FourLLIE achieves an SSIM of 0.5910, significantly better than Transformer-based methods.
- Ablation studies show the amplitude transform map and SNR map significantly improve brightness and detail restoration.
Significance
This study innovatively applies Fourier frequency information to address brightness and detail restoration challenges in low-light image enhancement, offering a lightweight solution suitable for real-world deployment.
Technical Contribution
Introduced amplitude transform maps in Fourier space for brightness enhancement and combined SNR maps for spatial-frequency interaction, significantly reducing model complexity while improving performance.
Novelty
First to combine Fourier frequency information with SNR maps for low-light enhancement, introducing a novel theory linking amplitude and brightness.
Limitations
- Performance may degrade under extremely low-light conditions where frequency information is insufficient.
- SNR map computation introduces preprocessing overhead, potentially increasing inference time.
- Scalability to high-resolution images remains unverified.
Future Work
Future work could explore more efficient SNR map computation methods and incorporate multimodal data (e.g., depth information) to further enhance performance.
AI Executive Summary
Low-light image enhancement (LLIE) is a critical task in computer vision, but existing methods struggle to balance brightness improvement and detail restoration. FourLLIE introduces a novel two-stage framework leveraging Fourier frequency information and SNR maps. The first stage enhances brightness using amplitude transform maps in Fourier space, while the second stage restores details by integrating global frequency and local spatial information guided by SNR maps.
Experiments demonstrate that FourLLIE outperforms SOTA methods on four datasets, including LOL-Real, achieving a PSNR of 27.31 with only 0.31% of SNR-Aware's parameters. Ablation studies confirm the critical roles of amplitude transform maps and SNR maps. Additionally, FourLLIE's lightweight design makes it suitable for practical deployment.
However, FourLLIE has limitations in extreme low-light scenarios and high-resolution images. Future research could focus on optimizing SNR map computation and exploring multimodal approaches to further improve its robustness and applicability.
Deep Analysis
Background
LLIE aims to enhance visibility and quality of images captured under low-light conditions, a key challenge in computer vision. Traditional methods like histogram equalization and Retinex-based approaches struggle with noise and color distortion. Recent deep learning methods, such as SNR-Aware and FECNet, have achieved significant progress but suffer from high model complexity, limiting real-world deployment.
Core Problem
Existing methods face a trade-off between brightness enhancement and detail restoration, while their high complexity hinders practical use. Efficiently leveraging global information to improve LLIE performance remains an open challenge.
Innovation
Key innovations of FourLLIE include:
- �� Introducing amplitude transform maps in Fourier space to enhance brightness by leveraging the positive correlation between amplitude and brightness.
- �� Incorporating SNR maps to integrate global frequency and local spatial information for detailed restoration.
- �� Reducing model parameters to only 0.31% of SNR-Aware, making it lightweight and efficient.
Methodology
FourLLIE employs a two-stage design:
- �� Frequency Stage: Extracts amplitude and phase components via Fourier transform. Six FP blocks estimate the amplitude transform map to enhance brightness.
- �� Spatial Stage: Computes SNR maps to guide the integration of global features (via FP blocks) and local features (via SP blocks), restoring image details.
Experiments
Experiments were conducted on LOL-Real, LOL-Synthetic, LSRW-Huawei, and LSRW-Nikon datasets, with baselines including SNR-Aware and FECNet. Metrics include PSNR and SSIM, and ablation studies validate the effectiveness of amplitude transform maps and SNR maps.
Results
FourLLIE achieves a PSNR of 27.31 and SSIM of 0.871 on LOL-Real, outperforming SNR-Aware by 1.2dB. Ablation studies show removing the amplitude transform map reduces PSNR by 2.1dB, highlighting its importance.
Applications
FourLLIE is applicable to low-light photography, video surveillance, and autonomous driving, particularly on resource-constrained devices.
Limitations & Outlook
FourLLIE struggles in extreme low-light conditions, SNR map computation adds preprocessing overhead, and its scalability to high-resolution images needs further validation.
Plain Language Accessible to non-experts
Imagine taking a photo in a dimly lit room. The picture is too dark to see clearly. FourLLIE acts like a smart lighting assistant. First, it uses Fourier transform to analyze the overall lighting and identifies areas needing 'extra light.' Then, it uses an SNR map to decide which parts need detailed adjustments. The result? A brighter, clearer photo, just like a professional photo editor would create.
ELI14 Explained like you're 14
Imagine you're gaming in a dark room and take a screenshot, but it's too dark to see anything. FourLLIE is like a magic filter! It uses a cool trick called Fourier transform to figure out where the light is missing, then uses another tool called an SNR map to fix blurry and noisy parts. Boom! Your screenshot now looks bright and sharp, like it was taken in daylight!
Glossary
Fourier Transform
A mathematical tool that converts images from spatial to frequency domain, capturing global information.
Used to extract amplitude and phase components of an image.
Amplitude
The part of Fourier transform representing light intensity, correlated with image brightness.
Used to adjust brightness in low-light images.
Signal-to-Noise Ratio (SNR)
The ratio of signal strength to noise, indicating image quality.
Guides the integration of global and local features.
FP Block
Fourier Processing Block that extracts global features in the frequency domain.
Used to estimate amplitude transform maps.
SP Block
Spatial Processing Block that extracts local features in the spatial domain.
Used for detail restoration.
Open Questions Unanswered questions from this research
- 1 How can detail restoration be further improved under extreme low-light conditions?
- 2 Can SNR map computation be optimized to reduce preprocessing time?
Applications
Immediate Applications
Low-light Photography
Enhances image quality in night photography, reducing noise and improving clarity.
Video Surveillance
Improves clarity of surveillance footage captured in low-light environments, enhancing security.
Long-term Vision
Autonomous Driving
Enhances sensor image visibility in low-light conditions, improving reliability of autonomous systems.
Abstract
Recently, Fourier frequency information has attracted much attention in Low-Light Image Enhancement (LLIE). Some researchers noticed that, in the Fourier space, the lightness degradation mainly exists in the amplitude component and the rest exists in the phase component. By incorporating both the Fourier frequency and the spatial information, these researchers proposed remarkable solutions for LLIE. In this work, we further explore the positive correlation between the magnitude of amplitude and the magnitude of lightness, which can be effectively leveraged to improve the lightness of low-light images in the Fourier space. Moreover, we find that the Fourier transform can extract the global information of the image, and does not introduce massive neural network parameters like Multi-Layer Perceptrons (MLPs) or Transformer. To this end, a two-stage Fourier-based LLIE network (FourLLIE) is proposed. In the first stage, we improve the lightness of low-light images by estimating the amplitude transform map in the Fourier space. In the second stage, we introduce the Signal-to-Noise-Ratio (SNR) map to provide the prior for integrating the global Fourier frequency and the local spatial information, which recovers image details in the spatial space. With this ingenious design, FourLLIE outperforms the existing state-of-the-art (SOTA) LLIE methods on four representative datasets while maintaining good model efficiency.