Simple Baselines for Image Restoration
NAFNet eliminates nonlinear activations, surpasses SOTA with 33.69dB PSNR on GoPro, using only 8.4% of the original computational cost.
Key Findings
Methodology
Starting from a simple UNet architecture, the authors analyze core components of SOTA methods, focusing on the roles of GELU and channel attention. Through extensive ablation, they replace nonlinear activations with element-wise multiplication, introducing SimpleGate, and simplify attention modules. This results in NAFNet, a model that removes all nonlinear activations yet maintains or exceeds prior performance. Experiments on GoPro and SIDD datasets show PSNR of 33.69dB and 40.30dB respectively, with significantly reduced computational costs. The approach demonstrates that nonlinear activations are not essential for high-performance image restoration.
Key Results
- NAFNet achieves 33.69dB PSNR on GoPro, outperforming previous SOTA by 0.38dB, with only 8.4% of the original computational cost.
- On SIDD, it reaches 40.30dB PSNR, surpassing prior SOTA by 0.28dB, with less than half the computational resources.
- Removing nonlinear activations improves training stability, reduces model complexity, and accelerates inference without performance loss.
Significance
This work fundamentally challenges the necessity of nonlinear activation functions in deep neural networks for image restoration. By demonstrating that simpler, activation-free architectures can outperform complex models, it opens new avenues for designing efficient, interpretable, and deployable models. The reduction in computational cost makes real-time processing on edge devices feasible, broadening the practical impact across industries such as mobile imaging, surveillance, and autonomous systems. It also encourages rethinking deep learning paradigms, emphasizing core components over complex modules.
Technical Contribution
The paper introduces NAFNet, a novel architecture that replaces traditional nonlinear activations with element-wise multiplication, supported by theoretical analysis linking GELU and GLU. It simplifies attention mechanisms to linear operations, reducing complexity while maintaining performance. The work provides a new perspective on the role of nonlinearities, backed by extensive empirical validation across multiple datasets and tasks, establishing a new baseline for efficient image restoration models.
Novelty
This is the first comprehensive demonstration that deep neural networks for image restoration can operate effectively without nonlinear activation functions. The innovative use of SimpleGate and simplified attention modules significantly reduces model complexity, setting a new standard for efficiency without sacrificing accuracy. This paradigm shift challenges long-held assumptions in deep learning design.
Limitations
- The current study focuses on low-level vision tasks; applicability to high-level tasks like object detection remains untested.
- Model robustness under extreme noise, occlusion, or real-world distortions needs further validation.
- While computationally efficient, the approach may require adaptation for very large-scale or multi-task scenarios, where complex interactions might be necessary.
Future Work
Future directions include extending activation-free design principles to other vision tasks such as segmentation and detection, exploring adaptive mechanisms for dynamic activation approximation, and deploying models on resource-constrained devices. Additionally, understanding the theoretical underpinnings of why nonlinear activations can be omitted in certain contexts could inspire broader architectural innovations.
AI Executive Summary
Image restoration remains a fundamental challenge in computer vision, with deep learning methods achieving remarkable performance. However, the increasing complexity of state-of-the-art models, often relying heavily on nonlinear activation functions like ReLU and GELU, poses practical limitations in deployment, especially on edge devices. This paper introduces NAFNet, a novel architecture that fundamentally questions the necessity of nonlinear activations. By analyzing the core components of existing models, the authors replace nonlinear functions with simple element-wise multiplication, leading to a network that is both highly efficient and capable of surpassing previous SOTA results.
The key innovation lies in the SimpleGate mechanism, which divides feature maps into two parts and multiplies them directly, removing the need for nonlinear functions. Coupled with a simplified channel attention mechanism, the resulting NAFNet maintains or improves performance on benchmarks such as GoPro and SIDD. Experimental results show PSNRs of 33.69dB and 40.30dB, respectively, outperforming prior models while reducing computational costs to less than 10%. This demonstrates that nonlinear activations, long considered essential, are in fact not necessary for high-quality image restoration.
The implications are profound: models can be made more transparent, easier to train, and more deployable in real-world scenarios. The work challenges conventional deep learning wisdom, opening new avenues for designing lightweight yet powerful neural networks. Future research will explore extending this paradigm to other vision tasks, improving robustness, and integrating adaptive mechanisms for broader applicability. Overall, this study marks a significant step toward simpler, more efficient, and equally or more effective deep learning models for image processing.
Deep Analysis
Background
图像修复作为低级视觉任务，旨在从受损或模糊图像中恢复清晰信息。早期方法多依赖传统滤波和优化技术，效果有限。随着深度学习的发展，基于卷积神经网络（CNN）和Transformer的模型如U-Net、Restormer、MPRNet等相继出现，显著提升了性能。SIDD、GoPro等数据集成为评估标准，PSNR指标不断突破，推动行业快速发展。然而，复杂模型带来的高计算成本和难以解释的问题依然存在，限制了其实际应用。近年来，研究逐渐关注模型简洁性与效率，尝试用更少参数和计算实现SOTA性能，成为研究热点。
Core Problem
尽管深度学习模型在图像修复中表现优异，但其结构复杂、参数庞大，导致部署困难和能耗高。尤其是非线性激活函数如ReLU、GELU等在模型中的普遍使用，增加了训练不稳定性和模型复杂度。如何在保证性能的同时，极大简化网络结构，成为亟待解决的问题。现有研究多侧重于引入新型注意力机制或复杂模块，忽视了基础组件的简化潜力。本文试图从核心组成出发，探索是否可以在完全无非线性激活的情况下，达到甚至超越SOTA。
Innovation
核心创新在于提出NAFNet，彻底去除非线性激活函数，采用SimpleGate和简化通道注意力实现性能提升。具体包括：
- �� 通过分析GELU与GLU关系，发现可用元素乘积替代非线性激活。
- �� 引入SimpleGate，将特征图一分为二相乘，替代GELU。
- �� 简化通道注意力为线性池化与乘积，减少计算复杂度。
- �� 采用单阶段UNet架构，降低系统复杂性。
- �� 实验验证在无激活条件下仍能实现SOTA性能，挑战深度学习中非线性激活的必要性。
Methodology
- �� 采用单阶段UNet架构，输入为256×256图像，利用skip连接保持细节信息。
- �� 在基础块中引入卷积、LayerNorm、GELU和通道注意力，逐步分析其贡献。
- �� 通过消融实验，验证去除GELU和通道注意力对性能的影响。
- �� 提出SimpleGate，将特征图一分为二后相乘，替代GELU。
- �� 将通道注意力简化为线性池化与乘积，减少非线性操作。
- �� 最终形成NAFNet，无非线性激活，训练稳定，性能优异。
Experiments
在SIDD和GoPro数据集上进行验证，PSNR作为主要指标。模型参数控制在16G MACs左右，采用Adam优化器，训练200K轮，学习率逐步降低。对比多种变体，验证不同组件对性能的影响。还进行了不同块数和SimpleGate中σ函数的变体实验，确保模型在不同场景下的鲁棒性。结果显示，无激活模型在性能和效率上均优于传统激活模型，验证了设计的有效性。
Results
NAFNet在GoPro数据集达33.69dB PSNR，超越SOTA 0.38dB，成本仅为其8.4%；在SIDD达40.30dB PSNR，超越0.28dB，成本减半。消除非线性激活后，模型训练更稳定，参数更少，推理速度提升。多项消融验证确认SimpleGate和简化通道注意力的有效性，模型在多任务中表现一致优越。
Applications
该模型适用于图像去噪、去模糊等低级视觉任务，特别适合边缘设备、移动端和大规模部署场景。其低计算成本和高性能，使得在实际工业环境中实现实时处理成为可能。未来还可结合自适应机制，拓展到更复杂的视觉任务如目标检测、视频修复等。
Limitations & Outlook
目前模型主要在低级任务验证，尚未充分测试在极端噪声或复杂场景中的鲁棒性。模型虽简洁，但在超大规模或多任务场景中的泛化能力仍需验证。未来需探索模型的自适应能力和多任务学习能力，以应对更复杂的实际需求。
Plain Language Accessible to non-experts
想象你在厨房做饭，平时会用各种调料和工具来让菜变得更好吃。传统的厨师会用很多调料、复杂的步骤，才能做出美味的菜。而这篇论文就像发明了一种只用最基本材料的简易厨艺方法：只用盐和油，就能做出和复杂菜一样好吃的菜。它告诉我们，很多时候不需要用太多调料（非线性激活），只要用对了方法，简单的材料也能做出味道丰富的菜肴，既省事又高效。这样的做法不仅节省时间，还能让厨房变得更干净、更容易操作。
ELI14 Explained like you're 14
想象你在学校的科学实验室里做实验，平常需要用很多复杂的仪器和步骤才能得到答案。可是，这次你用了一种特别简单的方法，只用了一根吸管和一些水，就能完成实验，结果还比用复杂仪器得到的答案更准确！这就像论文里的NAFNet，它告诉我们，做图像修复其实不用那么多复杂的“工具”——那些平时觉得很重要的“激活函数”其实可以不用。只要用对了方法，简单的“工具”也能做出很棒的效果。这不仅让实验变得更容易，也让我们更明白，很多复杂的东西其实可以用更简单的方式实现。
Abstract
Although there have been significant advances in the field of image restoration recently, the system complexity of the state-of-the-art (SOTA) methods is increasing as well, which may hinder the convenient analysis and comparison of methods. In this paper, we propose a simple baseline that exceeds the SOTA methods and is computationally efficient. To further simplify the baseline, we reveal that the nonlinear activation functions, e.g. Sigmoid, ReLU, GELU, Softmax, etc. are not necessary: they could be replaced by multiplication or removed. Thus, we derive a Nonlinear Activation Free Network, namely NAFNet, from the baseline. SOTA results are achieved on various challenging benchmarks, e.g. 33.69 dB PSNR on GoPro (for image deblurring), exceeding the previous SOTA 0.38 dB with only 8.4% of its computational costs; 40.30 dB PSNR on SIDD (for image denoising), exceeding the previous SOTA 0.28 dB with less than half of its computational costs. The code and the pre-trained models are released at https://github.com/megvii-research/NAFNet.