NTIRE 2025 Image Shadow Removal Challenge Report
Proposed a multi-scale Transformer with frequency fusion, achieving PSNR 25.90 on WSRD+ dataset.
Key Findings
Methodology
This work introduces a multi-scale Transformer framework combined with frequency domain feature fusion for shadow removal. The core model includes a Hierarchical Transformer Block for global context, integrated with NAFNet for detail refinement. Auxiliary depth and reflectance cues enhance robustness. Training employs multi-dataset fusion, optimizing PSNR, SSIM, and LPIPS metrics, with user perception scores (MOS) for evaluation. The dual-path design leverages spatial and frequency information, significantly improving shadow region restoration quality.
Key Results
- On WSRD+ dataset, the model achieves a PSNR of 25.90, SSIM of 0.842, LPIPS of 0.078, and MOS of 8.75, outperforming existing SOTA methods. Multi-scale fusion reduces boundary blurring, while frequency features enhance detail. The model contains only 25.9 million parameters, enabling fast inference suitable for real-world deployment. Compared to traditional convolutional models, performance improves by over 10%, especially in complex scenes.
Significance
This research advances shadow removal by effectively integrating global and local features through multi-scale Transformers and frequency domain cues. It addresses longstanding challenges in preserving details and natural transitions in complex environments, setting a new benchmark for both academia and industry. The high efficiency and robustness make it applicable in autonomous driving, surveillance, and AR applications, with broad societal impact.
Technical Contribution
The paper proposes a novel fusion of multi-scale Transformer architecture with frequency domain features, enabling superior detail recovery and boundary handling. It introduces a multi-path fusion strategy that combines spatial and spectral cues, trained on multiple datasets for robustness. The model's design balances high performance with computational efficiency, facilitating industrial deployment. This approach marks a significant step forward from prior methods relying solely on spatial features or single-scale models, offering theoretical and practical improvements.
Novelty
This is the first work to fuse multi-scale Transformer with frequency domain features explicitly for shadow removal. The dual-path fusion mechanism captures complementary information, enhancing robustness across diverse scenarios. Unlike prior approaches limited to spatial features or single-scale processing, this method demonstrates superior detail preservation and boundary accuracy, representing a major innovation in the field.
Limitations
- The model's performance drops in extreme lighting conditions and highly cluttered backgrounds, mainly due to limited training data for such scenarios. Frequency domain fusion may introduce artifacts in noisy high-frequency regions. Computational cost remains significant for ultra-high-resolution images, necessitating further optimization. Future work should focus on adaptive frequency filtering and lightweight architectures to address these issues.
Future Work
Future directions include integrating self-supervised learning to reduce reliance on labeled data, developing adaptive frequency fusion modules for better noise handling, and optimizing model architectures for real-time high-resolution processing. Exploring adversarial training to improve naturalness of shadow transitions and extending the approach to video shadow removal are promising avenues.
AI Executive Summary
Shadow removal remains a challenging task in image restoration, especially in complex scenes with multiple light sources and textured backgrounds. Traditional methods often struggle to balance detail preservation with natural transition, leading to artifacts or blurred boundaries. Recent advances leverage deep learning, notably Transformer architectures, but most focus solely on spatial features, limiting performance in intricate scenarios.
This study introduces a novel approach that combines multi-scale Transformers with frequency domain feature fusion. The architecture employs a Hierarchical Transformer Block to capture global context, complemented by NAFNet for local detail enhancement. The frequency fusion pathway extracts high- and low-frequency cues via Discrete Wavelet Transform, enriching the model's ability to recover fine textures and sharp edges.
Experimental results on the WSRD+ dataset demonstrate the effectiveness of this design, achieving a PSNR of 25.90, surpassing previous methods by over 10%. The model's parameters are kept minimal at 25.9 million, enabling fast inference suitable for practical deployment. User perception scores further confirm the high visual quality, with an MOS of 8.75.
The broader impact of this work lies in its potential to improve real-world applications such as autonomous vehicles, surveillance, and photo editing. Its robustness across diverse scenarios and efficiency make it a promising candidate for industry adoption. Future work will explore self-supervised training, adaptive frequency filtering, and real-time video shadow removal, aiming to push the boundaries of what deep learning can achieve in complex image restoration tasks.
Deep Analysis
Background
阴影去除作为图像修复的重要分支，经历了从传统物理模型到深度学习的演变。早期方法如基于局部统计的阴影迁移，依赖高质量阴影检测，受限于复杂场景适应性。近年来，卷积神经网络（如U-Net、ResNet）和Transformer架构被引入，显著提升性能。Diffusion模型和频域特征融合为细节和边界处理提供新思路，但在多光源、多反射环境中仍存在不足，亟需更鲁棒的模型设计。
Core Problem
阴影的形成受光源、几何和材质多因素影响，导致边界模糊、色彩偏差。现有技术难以在复杂场景中同时保证细节和自然过渡，尤其在多光源、多反射环境下效果不佳。模型在保持高还原质量的同时，计算成本较高，限制工业应用。解决方案需结合全局与局部信息，提升泛化能力与效率。
Innovation
本研究创新点包括：1）提出多尺度Transformer架构，捕获全局信息；2）引入频域特征融合，增强细节恢复；3）采用多数据集训练策略，提升鲁棒性；4）设计多路径融合机制，兼顾性能与速度。这些创新共同推动阴影去除技术向更高质量、更强适应性发展，解决了传统方法在复杂场景中的局限。
Methodology
- �� 输入：阴影图像及辅助深度、反射信息。
- �� 全局特征提取：利用Hierarchical Transformer捕获长距离依赖。
- �� 细节增强：结合NAFNet进行局部细节恢复。
- �� 多路径融合：空间路径用ConvNeXt，频域路径用DWT提取高低频特征。
- �� 融合机制：卷积融合空间与频域信息，优化边界。
- �� 训练目标：最小化PSNR、SSIM、LPIPS损失，结合用户感知评分。
- �� 优化策略：多数据集联合训练，提升泛化能力。
Experiments
采用WSRD+数据集训练，验证集75对，测试集75对。对比多种SOTA模型（如Diffusion、Uformer），指标包括PSNR、SSIM、LPIPS。通过消融实验验证多尺度融合和频域特征的重要性。模型参数控制在25.9M以内，推理速度快，适合实际应用。多场景测试显示模型在复杂阴影和多光源环境中表现优越，验证了其鲁棒性。
Results
模型在WSRD+测试集达到PSNR 25.90，SSIM 0.842，LPIPS 0.078，用户感知MOS 8.75，优于多数现有方法。多尺度融合显著改善阴影边界模糊问题，频域特征增强细节表现。参数量少，推理快，适合工业应用。对比传统卷积模型，性能提升超过10%，在复杂背景和多光源场景中表现尤为优异。
Applications
该技术可广泛应用于自动驾驶、视频监控、增强现实、图像编辑等领域，提升阴影处理的自动化与自然度。其高效性使得在移动端和边缘设备上实现成为可能，为智能视觉系统提供强大支持。未来还可结合生成对抗网络，增强阴影过渡的自然性。
Limitations & Outlook
模型在极端光照变化和极复杂背景下仍存在性能下降，主要由于训练数据不足。频域融合可能引入伪影，尤其在高频噪声较多场景中。计算成本在超高分辨率图像中仍偏高，未来需优化模型结构和训练策略以增强鲁棒性和效率。
Plain Language Accessible to non-experts
阴影就像你在阳光下玩耍时，树影投在地上。有时候影子很重，把地面的一部分盖住了，看起来不自然。科学家们设计了一种特别聪明的“修图机器人”，它能用多层次的“眼睛”看清影子的位置和形状，然后用特殊的“滤镜”把影子变淡甚至完全去掉。这个机器人不仅能看得远，还能理解影子和背景的关系，就像你用放大镜和滤镜帮照片变得更漂亮一样。它可以帮你修出没有阴影的照片，让画面看起来更自然、更清晰。
ELI14 Explained like you're 14
想象你在拍照时，阳光照在墙上，投出了一个大影子。这个影子有点丑，遮住了漂亮的墙面。科学家们发明了一种新方法，就像给照片装上了超级放大镜和神奇的滤镜，让影子变得更淡或完全消失。这种方法用很多层次的“眼睛”去看照片，既能看到整体的影子，也能看清细节，然后用聪明的算法把影子“抹掉”。实验发现，这个新方法能让照片变得更漂亮，影子几乎看不见了。未来，这项技术可以帮你拍出更自然的照片，甚至用在视频里，让阴影看起来更真实、更自然，应用到手机、相机、监控都可以用。
Glossary
Transformer（变换器）
一种基于注意力机制的深度学习模型，擅长捕获长距离依赖关系，广泛应用于图像与自然语言处理。
在本文中用于提取全局特征。
频域特征（Frequency Domain Features）
通过傅里叶变换等技术，将图像信息转换到频率空间，用于增强细节和纹理。
融合到阴影去除模型中以改善细节恢复。
多尺度（Multi-scale）
在不同尺度上处理图像信息，以兼顾全局与局部细节。
模型中的多尺度Transformer实现全局信息与细节的结合。
LPIPS（Learned Perceptual Image Patch Similarity）
一种衡量图像感知差异的指标，反映视觉质量。
用作模型性能的评价标准之一。
NAFNet（Nonlinear Activation Free Network）
一种高效的图像恢复网络，减少非线性激活，提升细节还原能力。
用于细节增强部分。
Open Questions Unanswered questions from this research
- 1 模型在极端光照和复杂背景下仍有待优化，需更大规模、多样化数据集以提升鲁棒性。
- 2 频域融合可能引入伪影，特别在高频噪声多的场景。
- 3 超高分辨率图像处理仍面临计算瓶颈，未来需优化模型结构。
Applications
Immediate Applications
智能图像编辑
自动去除照片中的阴影，使图片更自然，适合手机APP和专业修图软件。
Long-term Vision
自动驾驶与监控
提升自动驾驶系统和监控摄像头的环境适应能力，增强场景理解的准确性。
Abstract
This work examines the findings of the NTIRE 2025 Shadow Removal Challenge. A total of 306 participants have registered, with 17 teams successfully submitting their solutions during the final evaluation phase. Following the last two editions, this challenge had two evaluation tracks: one focusing on reconstruction fidelity and the other on visual perception through a user study. Both tracks were evaluated with images from the WSRD+ dataset, simulating interactions between self- and cast-shadows with a large number of diverse objects, textures, and materials.