Bridging Degradation Discrimination and Generation for Universal Image Restoration
Proposes BDG framework combining MAS-GLCM and diffusion models for multi-task universal image restoration, significantly improving fidelity.
Key Findings
Methodology
This work introduces MAS-GLCM, a multi-angle multi-scale gray-level co-occurrence matrix, for fine-grained degradation discrimination. The diffusion training is divided into three stages: generation pre-training, degradation bridging, and fine-tuning, where residual information and MAS-GLCM features are aligned via feature matching and contrastive learning. Loss functions include Kullback-Leibler divergence, classification, and contrastive losses, ensuring the model learns both to recognize degradations and generate high-fidelity images. The approach maintains the diffusion architecture, enabling multi-task, multi-degradation restoration with significant performance gains.
Key Results
- In all-in-one restoration, BDG outperforms previous methods like DiffUIR by +3.72dB PSNR, achieving state-of-the-art results across tasks such as deraining, low-light enhancement, and dehazing. On real-world super-resolution, it surpasses competitors in PIQE and BRISQUE metrics, demonstrating robustness. MAS-GLCM achieves 97.13% classification accuracy for degradation type and level, outperforming gradient and frequency features. Ablation studies confirm the effectiveness of multi-angle multi-scale design and feature alignment.
- In mixed degradation scenarios, BDG shows strong robustness, accurately identifying complex degradations and restoring details. Quantitative metrics indicate superior fidelity and perceptual quality, with notable improvements in real-world datasets. The feature alignment and contrastive learning modules are validated as key components for balancing recognition and generation tasks.
- Experimental results validate that the proposed framework effectively combines degradation discrimination and generation priors, leading to versatile, high-fidelity restoration across diverse scenarios, including challenging real-world images.
Significance
This research advances universal image restoration by integrating detailed degradation recognition with powerful generative priors, enabling models to handle complex, real-world scenarios. It addresses the longstanding challenge of balancing fidelity and perceptual quality in multi-task settings. The approach has broad implications for practical applications such as medical imaging, surveillance, and remote sensing, where diverse degradation types are common. By achieving high accuracy in degradation discrimination and superior restoration quality without architecture modifications, BDG paves the way for more robust, adaptable low-level vision systems, fostering progress toward truly universal image enhancement solutions.
Technical Contribution
The key technical innovation lies in embedding MAS-GLCM features into the diffusion model via a three-stage training process, including feature alignment and contrastive learning. This enables the model to simultaneously recognize fine-grained degradation patterns and generate detailed textures, a significant departure from existing methods that focus solely on either generation or classification. The feature alignment mechanism ensures balanced multi-task learning without architecture changes, while the contrastive loss enhances discrimination of degradation types. These contributions collectively establish a new paradigm for multi-task, multi-degradation image restoration.
Novelty
This is the first work to integrate MAS-GLCM-based fine-grained degradation discrimination with diffusion-based generative models in a unified framework. The three-stage training strategy and feature alignment mechanism enable the model to excel in both recognition and synthesis tasks simultaneously, surpassing prior approaches that treat these tasks separately. The method effectively bridges the gap between degradation identification and high-fidelity image generation, setting a new benchmark for universal image restoration.
Limitations
- Despite its robustness, the model struggles under extremely severe or non-linear degradations, such as complex motion blur combined with noise, where recognition and restoration degrade significantly.
- Training complexity and computational cost are high due to multi-stage procedures and feature alignment, limiting deployment in resource-constrained environments.
- Parameter sensitivity, especially in MAS-GLCM configuration, requires careful tuning for different scenarios, affecting generalization across diverse datasets.
Future Work
Future research will focus on adaptive parameter tuning for MAS-GLCM, reducing computational overhead, and extending the framework to video and 3D data. Exploring unsupervised or weakly supervised degradation recognition methods could further enhance robustness. Additionally, integrating multi-modal data (e.g., depth, semantic maps) may improve performance in specialized applications like medical imaging and autonomous driving.
AI Executive Summary
Image restoration remains a fundamental challenge in low-level vision, especially when faced with diverse and complex degradations encountered in real-world scenarios. Traditional approaches often excel in specific tasks but lack the flexibility to handle multiple degradation types simultaneously, limiting their practical utility. Recent advances in deep learning, particularly diffusion models, have demonstrated remarkable generative capabilities, yet integrating detailed degradation recognition into these models remains an open problem.
This paper introduces BDG (Bridging Degradation discrimination and Generation), a novel framework that combines the fine-grained recognition ability of MAS-GLCM with the powerful generative capacity of diffusion models. The core idea is to enable the model to understand the specific degradation in an image and then leverage this understanding to produce high-fidelity restorations across multiple tasks. The training process is divided into three stages: initial generation pre-training, a bridging phase where MAS-GLCM features are aligned with diffusion features, and a fine-tuning stage emphasizing accurate restoration.
The innovative aspect lies in the multi-angle, multi-scale evaluation of pixel relationships via MAS-GLCM, which surpasses traditional gradient or frequency-based features in discriminating degradation types and levels. By integrating these features into the diffusion process through feature alignment and contrastive learning, the model maintains its generative strengths while gaining detailed degradation awareness. Experimental results on datasets like DIV2K and RealSR show that BDG achieves PSNR improvements of over 3.7dB compared to prior state-of-the-art methods, with superior performance in real-world super-resolution and multi-task restoration.
This work significantly impacts both academia and industry by providing a versatile, high-fidelity restoration framework capable of handling diverse, real-world degradations. Its ability to balance perceptual quality and fidelity without architectural modifications sets a new standard in universal image restoration. Future directions include optimizing training efficiency, extending to video and 3D data, and exploring multi-modal integration, promising broader applications in medical imaging, autonomous systems, and remote sensing. Overall, BDG marks a pivotal step toward truly universal, robust low-level vision systems.
Deep Dive
Abstract
Universal image restoration is a critical task in low-level vision, requiring the model to remove various degradations from low-quality images to produce clean images with rich detail. The challenges lie in sampling the distribution of high-quality images and adjusting the outputs on the basis of the degradation. This paper presents a novel approach, Bridging Degradation discrimination and Generation (BDG), which aims to address these challenges concurrently. First, we propose the Multi-Angle and multi-Scale Gray Level Co-occurrence Matrix (MAS-GLCM) and demonstrate its effectiveness in performing fine-grained discrimination of degradation types and levels. Subsequently, we divide the diffusion training process into three distinct stages: generation, bridging, and restoration. The objective is to preserve the diffusion model's capability of restoring rich textures while simultaneously integrating the discriminative information from the MAS-GLCM into the restoration process. This enhances its proficiency in addressing multi-task and multi-degraded scenarios. Without changing the architecture, BDG achieves significant performance gains in all-in-one restoration and real-world super-resolution tasks, primarily evidenced by substantial improvements in fidelity without compromising perceptual quality. The code and pretrained models are provided in https://github.com/MILab-PKU/BDG.