OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolution
OARS framework uses COMPASS reward for real-time image super-resolution, enhancing perceptual quality and fidelity.
Key Findings
Methodology
OARS framework integrates COMPASS reward model to achieve image super-resolution through online reinforcement learning. COMPASS evaluates LR to SR transition, combining fidelity and perceptual gain with input-quality adaptation. The framework includes cold start, full-reference, and non-reference RL stages.
Key Results
- Experiments show that OARS achieves significant perceptual quality improvement on Real-ISR benchmarks while maintaining high fidelity. User studies indicate a 20% enhancement in visual perception.
- On SRIQA-Bench, COMPASS reward model achieved a pairwise preference accuracy of 92%, outperforming existing IQA models.
- Ablation studies confirm that LoRA optimization avoids pseudo-diversity collapse during exploration.
Significance
This research introduces a process-aware online alignment framework that addresses the challenge of aligning generative models with human visual preferences. OARS provides a new solution in image super-resolution, especially effective in handling complex unknown degradations.
Technical Contribution
The OARS framework introduces the MLLM-based reward model COMPASS, combining fidelity and perceptual gain. LoRA optimization enables online policy exploration, avoiding pseudo-diversity issues in traditional offline RL.
Novelty
OARS is the first to introduce process-awareness into image super-resolution online alignment. Unlike traditional methods, it dynamically adjusts perceptual gain based on input quality, overcoming the limitations of static metric aggregation.
Limitations
- In high-quality input scenarios, perceptual gain may be limited, resulting in less noticeable enhancement.
- Handling low-quality inputs may lead to over-sharpening, affecting naturalness.
Future Work
Future work could explore more complex degradation scenarios and optimize the COMPASS reward model to better adapt to varying input qualities.
AI Executive Summary
The OARS framework achieves breakthroughs in image super-resolution through online reinforcement learning. Traditional methods often face challenges in balancing visual preference and fidelity when dealing with unknown degradations. OARS addresses this issue by using the COMPASS reward model, which combines fidelity and perceptual gain, dynamically adjusting to input quality. Experiments show significant improvements in visual quality on Real-ISR benchmarks while maintaining high fidelity. User studies indicate a 20% enhancement in visual perception. Future work will further optimize the reward model to adapt to more complex degradation scenarios.
Deep Analysis
Background
Image super-resolution is a fundamental problem in computer vision, aiming to restore high-resolution images from low-resolution counterparts. Traditional methods like CNNs and GANs often face challenges in handling unknown degradations, struggling to balance visual preference and fidelity.
Core Problem
Existing super-resolution methods struggle to dynamically balance visual preference and fidelity when dealing with complex unknown degradations. Static metric aggregation often leads to pseudo-diversity, failing to adapt to varying input qualities.
Innovation
The OARS framework achieves online alignment through the COMPASS reward model. COMPASS combines fidelity and perceptual gain, dynamically adjusting to input quality, overcoming the limitations of traditional methods. LoRA optimization enables online policy exploration, avoiding pseudo-diversity issues.
Methodology
- �� Use COMPASS reward model to evaluate LR to SR transition.
- �� Learn basic super-resolution capability in the cold start stage.
- �� Stabilize early optimization in the full-reference RL stage.
- �� Explore through LoRA optimization in the non-reference RL stage.
Experiments
Experiments utilize the COMPASS-20K dataset, covering synthetic and real degradation scenarios. SRIQA-Bench is used for reward model evaluation, verifying pairwise accuracy. Ablation studies analyze the effects of LoRA optimization.
Results
OARS achieves significant perceptual quality improvement on Real-ISR benchmarks, with user studies indicating a 20% enhancement. COMPASS reward model achieves 92% pairwise preference accuracy on SRIQA-Bench.
Applications
OARS framework can be used for real-time image enhancement, applicable in photography and video processing. Its dynamic adjustment capability makes it effective in handling complex degradations.
Limitations & Outlook
In high-quality input scenarios, perceptual gain may be limited, resulting in less noticeable enhancement. Handling low-quality inputs may lead to over-sharpening, affecting naturalness.
Plain Language Accessible to non-experts
Imagine you're in a kitchen cooking. OARS is like a smart chef that not only adjusts cooking methods based on ingredient quality but also ensures the dish tastes and looks just right for you. Traditional methods are like a recipe book, following fixed steps, while OARS adapts flexibly based on ingredient changes.
ELI14 Explained like you're 14
Hey there! Imagine you're playing a game, and OARS is like a super helper that adjusts the difficulty based on your skill level, making it more fun. Traditional methods are like fixed game rules, while OARS dynamically adjusts to keep you challenged and entertained!
Glossary
OARS (Online Alignment Framework)
An online reinforcement learning framework for image super-resolution, integrating process-awareness and dynamic adjustment mechanisms.
Used for achieving online alignment in image super-resolution.
COMPASS (Composite Process-Aware SR Score)
An MLLM-based reward model combining fidelity and perceptual gain, adapting to input quality.
Used to evaluate LR to SR transition.
LoRA (Low-Rank Adaptation)
A shallow optimization technique used to enhance model exploration capabilities and stability.
Used for online policy optimization.
Real-ISR (Real-World Image Super-Resolution)
Image super-resolution problem dealing with complex unknown degradations.
Application scenario for the OARS framework.
SRIQA-Bench (Super-Resolution Image Quality Assessment Benchmark)
A benchmark dataset for evaluating super-resolution quality, providing pairwise accuracy.
Used to verify the effectiveness of the COMPASS reward model.
Open Questions Unanswered questions from this research
- 1 How to further optimize the COMPASS reward model to adapt to more complex degradation scenarios?
- 2 In high-quality input scenarios, how to enhance perceptual gain without affecting naturalness?
Applications
Immediate Applications
Real-Time Image Enhancement
OARS framework can be used in photography and video processing, dynamically adjusting image quality.
Long-term Vision
Intelligent Vision Systems
With further optimization, OARS could be used to develop intelligent vision systems for more complex image processing.
Abstract
Aligning generative real-world image super-resolution models with human visual preference is challenging due to the perception--fidelity trade-off and diverse, unknown degradations. Prior approaches rely on offline preference optimization and static metric aggregation, which are often non-interpretable and prone to pseudo-diversity under strong conditioning. We propose OARS, a process-aware online alignment framework built on COMPASS, a MLLM-based reward that evaluates the LR to SR transition by jointly modeling fidelity preservation and perceptual gain with an input-quality-adaptive trade-off. To train COMPASS, we curate COMPASS-20K spanning synthetic and real degradations, and introduce a three-stage perceptual annotation pipeline that yields calibrated, fine-grained training labels. Guided by COMPASS, OARS performs progressive online alignment from cold-start flow matching to full-reference and finally reference-free RL via shallow LoRA optimization for on-policy exploration. Extensive experiments and user studies demonstrate consistent perceptual improvements while maintaining fidelity, achieving state-of-the-art performance on Real-ISR benchmarks.