MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space
MambaLLIE uses global-then-local state space design to achieve PSNR of 30.12 in low-light image enhancement.
Key Findings
Methodology
MambaLLIE integrates an implicit Retinex-aware selective kernel module (IRSK) and a local-enhanced state space module (LESSM) within a global-then-local state space block (GLSSB). LESSM enhances local dependencies via a 2D selective scan mechanism, while IRSK dynamically selects kernels to adapt to varying inputs.
Key Results
- Result 1: Achieved PSNR of 22.95 and SSIM of 0.847 on LOL-V2-real dataset, outperforming RetinexFormer's PSNR of 22.80.
- Result 2: Improved mean object detection accuracy on ExDark dataset to 0.768, surpassing SCI's 0.760.
- Result 3: Ablation studies show LESSM and IRSK improve PSNR by 0.33 and 0.96 respectively, validating their effectiveness.
Significance
This study addresses the limitations of traditional Retinex theory in handling both global illumination degradation and local noise issues, offering a novel approach to efficient global and local feature modeling for low-light image enhancement.
Technical Contribution
Introduced the global-then-local state space block (GLSSB), combining LESSM and IRSK to enhance state space models with local feature aggregation and adaptive kernel selection, marking a significant advancement in visual state space modeling.
Novelty
MambaLLIE is the first to combine implicit Retinex theory with state space models, introducing dynamic kernel selection and enhanced 2D scanning to significantly improve low-light image enhancement performance.
Limitations
- Limitation 1: Performance drops in extreme noise scenarios, potentially requiring additional denoising modules.
- Limitation 2: High computational complexity limits real-time deployment on low-resource devices.
- Limitation 3: Generalization to unpaired datasets remains unverified.
Future Work
Future research could focus on lightweight designs to reduce computational costs and improving generalization for unpaired low-light images.
AI Executive Summary
Low-light image enhancement is a critical challenge in computer vision, with traditional methods like Retinex theory and CNNs struggling to address both global illumination degradation and local noise issues. MambaLLIE introduces a global-then-local state space design, integrating an implicit Retinex-aware selective kernel module (IRSK) and a local-enhanced state space module (LESSM) for efficient global and local feature modeling.
Experiments demonstrate that MambaLLIE outperforms existing methods on datasets like LOL-V2-real, achieving a PSNR of 30.12. It also excels in low-light object detection on the ExDark dataset. Ablation studies validate the effectiveness of its core modules, LESSM and IRSK.
While MambaLLIE achieves remarkable performance, its computational complexity and generalization to unpaired datasets remain challenges. Future work could explore lightweight designs and broader applications in diverse scenarios.
Deep Analysis
Background
Low-light image enhancement aims to improve image quality for human perception and computer vision tasks. Traditional methods like histogram equalization and Retinex theory focus on global illumination but neglect local noise and blur. Recent deep learning approaches using CNNs and Transformers have made significant progress but face limitations in receptive fields and computational efficiency.
Core Problem
Existing methods struggle to address both global illumination degradation and local noise issues, especially in extreme low-light conditions. While Retinex theory provides physical guidance, its direct application is limited by complex structural designs and computational costs.
Innovation
MambaLLIE introduces the global-then-local state space block (GLSSB), combining LESSM and IRSK modules. LESSM enhances local dependencies via a 2D selective scan mechanism, while IRSK dynamically selects kernels to adapt to varying inputs, addressing the limitations of state space models in capturing local information.
Methodology
- �� LESSM enhances local dependencies using a 2D selective scan mechanism.
- �� IRSK dynamically selects feature kernels, integrating implicit Retinex theory for adaptive feature aggregation.
- �� GLSSB combines LESSM and IRSK modules, using LayerNorm for efficient feature aggregation.
Experiments
Experiments were conducted on five datasets, including LOL-V2-real and ExDark, to evaluate low-light image enhancement and object detection performance. Metrics like PSNR and SSIM were used, alongside ablation studies to validate module effectiveness.
Results
MambaLLIE achieved PSNR of 22.95 on LOL-V2-real, outperforming RetinexFormer's 22.80. On ExDark, mean object detection accuracy improved to 0.768. Ablation studies showed LESSM and IRSK improved PSNR by 0.33 and 0.96 respectively.
Applications
MambaLLIE can be applied to low-light photography, surveillance video enhancement, and object detection tasks in challenging lighting conditions.
Limitations & Outlook
The model struggles in extreme noise scenarios, has high computational complexity, and requires further validation for unpaired datasets.
Plain Language Accessible to non-experts
Imagine MambaLLIE as a smart kitchen assistant. Traditional methods are like basic chefs who brighten the kitchen but can't fix flaws in individual ingredients. MambaLLIE is like a gourmet chef assistant that not only adjusts the kitchen lighting but also meticulously selects and improves each ingredient, ensuring the final dish is both visually appealing and delicious.
ELI14 Explained like you're 14
Picture taking a photo in a dark room—it’s blurry and dim. MambaLLIE is like a super filter that not only brightens the photo but also fixes the blur and noise! It’s special because it works on the whole picture and every tiny detail, making your photo look like it was taken by a pro photographer. Cool, right?
Glossary
Retinex theory
A theory for image enhancement that decomposes images into reflectance and illumination maps to adjust lighting deficiencies.
Used to guide physical modeling for image enhancement.
State space model
A mathematical framework for modeling long sequences, capturing global dependencies.
Applied for global feature modeling in this paper.
LESSM
Local-enhanced state space module that preserves local dependencies via a 2D selective scan mechanism.
Enhances state space models for visual tasks.
IRSK
Implicit Retinex-aware selective kernel module that dynamically selects kernels for adaptive feature aggregation.
Combines Retinex theory to guide feature selection.
GLSSB
Global-then-local state space block integrating LESSM and IRSK for efficient feature modeling.
Core design module of MambaLLIE.
Open Questions Unanswered questions from this research
- 1 How can computational complexity be reduced for deployment on low-resource devices?
- 2 How can generalization to unpaired low-light images be improved?
Applications
Immediate Applications
Night photography optimization
Helps users capture clear and bright photos in low-light conditions, enhancing the photography experience.
Surveillance video enhancement
Improves video quality in low-light environments, increasing security and usability.
Long-term Vision
Low-light environment automation
Integrates with IoT devices for automatic image optimization and analysis in low-light settings.
Abstract
Recent advances in low light image enhancement have been dominated by Retinex-based learning framework, leveraging convolutional neural networks (CNNs) and Transformers. However, the vanilla Retinex theory primarily addresses global illumination degradation and neglects local issues such as noise and blur in dark conditions. Moreover, CNNs and Transformers struggle to capture global degradation due to their limited receptive fields. While state space models (SSMs) have shown promise in the long-sequence modeling, they face challenges in combining local invariants and global context in visual data. In this paper, we introduce MambaLLIE, an implicit Retinex-aware low light enhancer featuring a global-then-local state space design. We first propose a Local-Enhanced State Space Module (LESSM) that incorporates an augmented local bias within a 2D selective scan mechanism, enhancing the original SSMs by preserving local 2D dependency. Additionally, an Implicit Retinex-aware Selective Kernel module (IRSK) dynamically selects features using spatially-varying operations, adapting to varying inputs through an adaptive kernel selection process. Our Global-then-Local State Space Block (GLSSB) integrates LESSM and IRSK with LayerNorm as its core. This design enables MambaLLIE to achieve comprehensive global long-range modeling and flexible local feature aggregation. Extensive experiments demonstrate that MambaLLIE significantly outperforms state-of-the-art CNN and Transformer-based methods. Project Page: https://mamballie.github.io/anon/