UnSCAR: Universal, Scalable, Controllable, and Adaptable Image Restoration
UnSCAR employs a multi-expert MoE architecture, supporting over 16 degradation types, enhancing scalability and controllability.
Key Findings
Methodology
UnSCAR introduces a multi-branch Mixture-of-Experts framework, leveraging low-level visual priors, task-aware degradation-content disentangled embeddings, and residual attention MoE modules for efficient joint learning across multiple degradations. It employs a unified guidance encoder to integrate diverse visual cues, and degradation-aware embeddings built on CLIP to distinguish tasks. Capacity-efficient ControlNet with bidirectional feedback enhances controllability. Training combines latent diffusion objectives with pixel-level reconstruction and degradation supervision, enabling robust performance on 16+ degradation types.
Key Results
- UnSCAR surpasses existing methods on benchmarks, improving PSNR by 3-5dB and SSIM by 0.05-0.1, especially excelling in complex mixed degradation scenarios, demonstrating superior generalization.
- User-controlled sliders enable fine-grained degradation manipulation, maintaining stability and interpretability. Few-shot fine-tuning with only 50 samples achieves near full-data performance, showing strong transferability.
- In medical imaging tasks like laparoscopic de-smoking and fundus restoration, UnSCAR exhibits zero-shot and few-shot adaptation, outperforming traditional single-task models significantly.
Significance
This work advances scalable, controllable universal image restoration, addressing the limitations of prior models in handling diverse, complex, and mixed degradations. Its multi-expert design and rich visual priors improve robustness and generalization, pushing the field toward practical, real-world applications across industries such as healthcare, remote sensing, and consumer electronics.
Technical Contribution
UnSCAR's core innovations include a multi-expert MoE architecture with residual attention, a unified low-level guidance encoder, and degradation-aware embeddings, enabling scalable, interpretable, and controllable multi-degradation restoration. The capacity-efficient ControlNet and bidirectional feedback further enhance model flexibility and performance. Few-shot adaptation strategies allow rapid transfer to new tasks, making the framework highly versatile compared to existing single-task or limited multi-task models.
Novelty
This is the first comprehensive system integrating multi-expert MoE with rich visual priors and degradation-aware embeddings for over 16 degradation types. Its capacity-efficient control pathway and bidirectional feedback mechanism set new standards for scalable, interpretable, and user-controllable universal image restoration, surpassing prior approaches that either lack scalability or fine control.
Limitations
- The model's performance drops on extremely complex or unseen degradation combinations, partly due to limited generalization of degradation embeddings. Computational costs remain high, challenging deployment on edge devices.
- Few-shot transfer, while effective, still requires careful tuning and more data in some cases. Handling ultra-rare or novel degradations needs further research.
- The architecture's complexity may hinder real-time applications without further optimization.
Future Work
Future directions include model compression for deployment efficiency, integrating automatic degradation recognition for fully autonomous restoration, and extending to video and 3D data. Exploring unsupervised or weakly supervised training could further improve generalization to unseen degradations, while optimizing the architecture for real-time applications remains a key goal.
AI Executive Summary
The challenge of restoring high-quality images from complex, real-world degradations has long hindered progress in computer vision. Existing methods often excel at single degradation types but struggle with the diverse, mixed, and unseen corruptions encountered in practical scenarios. Traditional models tend to be limited in scalability, often requiring separate training for each task, which leads to large, inefficient architectures and poor generalization.
UnSCAR addresses these issues by proposing a novel multi-expert Mixture-of-Experts framework that can handle over 16 degradation types simultaneously. Its architecture is built upon rich low-level visual priors, which are encoded into a unified guidance representation, enabling the model to distinguish and adapt to various degradations effectively. To ensure user control, the model incorporates capacity-efficient ControlNet modules with sliders, allowing fine-grained manipulation of restoration behavior.
The core technical innovation lies in the integration of residual attention MoE modules, degradation-aware embeddings based on CLIP, and a bidirectional feedback mechanism that exchanges information between control and generation pathways. This design significantly improves scalability, interpretability, and controllability. The training leverages latent diffusion models combined with pixel-level reconstruction and degradation supervision, resulting in high-fidelity restorations.
Extensive experiments on benchmark datasets demonstrate that UnSCAR outperforms state-of-the-art methods, especially in complex mixed degradation scenarios and few-shot transfer tasks. Its ability to adapt quickly to unseen domains, such as medical imaging, highlights its practical potential. Despite its high performance, the model's computational complexity remains a challenge, and future work will focus on efficiency and broader application.
Overall, UnSCAR establishes a new paradigm for scalable, controllable, and robust universal image restoration, bridging the gap between academic innovation and industrial needs, and paving the way for versatile, real-world deployment.
Deep Dive
Abstract
Universal image restoration aims to recover clean images from arbitrary real-world degradations using a single inference model. Despite significant progress, existing all-in-one restoration networks do not scale to multiple degradations. As the number of degradations increases, training becomes unstable, models grow excessively large, and performance drops across both seen and unseen domains. In this work, we show that scaling universal restoration is fundamentally limited by interference across degradations during joint learning, leading to catastrophic task forgetting. To address this challenge, we introduce a unified inference pipeline with a multi-branch mixture-of-experts architecture that decomposes restoration knowledge across specialized task-adaptable experts. Our approach enables scalable learning (over sixteen degradations), adapts and generalizes robustly to unseen domains, and supports user-controllable restoration across degradations. Beyond achieving superior performance across benchmarks, this work establishes a new design paradigm for scalable and controllable universal image restoration.