GeoEyes: On-Demand Visual Focusing for Evidence-Grounded Understanding of Ultra-High-Resolution Remote Sensing Imagery
GeoEyes enhances UHR remote sensing image understanding with AdaZoom-GRPO, achieving 54.23% accuracy on XLRS-Bench.
Key Findings
Methodology
GeoEyes employs a staged training framework, including a cold-start supervised fine-tuning on the UHR-CoZ dataset and the AdaZoom-GRPO reinforcement learning method. UHR-CoZ covers diverse zooming regimes, and AdaZoom-GRPO enhances evidence acquisition and answer improvement through reward mechanisms.
Key Results
- GeoEyes achieved 54.23% accuracy on XLRS-Bench, outperforming DeepEyes by 4.23%.
- In object classification tasks, GeoEyes reached 66.1% accuracy, significantly higher than Qwen3-VL-235B's 39.0%.
- GeoEyes excelled in overall counting tasks, demonstrating its advantage in fine-grained perception tasks.
Significance
GeoEyes addresses the challenge of sparse evidence in UHR remote sensing images through dynamic zooming strategies, providing new insights for applying multimodal large language models in remote sensing with significant academic and industrial impact.
Technical Contribution
GeoEyes introduces the UHR-CoZ dataset and AdaZoom-GRPO algorithm, addressing tool usage homogenization, offering new theoretical guarantees and engineering possibilities, and significantly enhancing model adaptability in UHR scenarios.
Novelty
GeoEyes is the first to implement task-adaptive zooming strategies in UHR remote sensing, significantly improving evidence acquisition efficiency and answer accuracy compared to existing methods.
Limitations
- GeoEyes may still experience performance degradation in extremely complex scenarios, especially with highly sparse evidence.
- The computational overhead is substantial, potentially limiting its application in resource-constrained environments.
Future Work
Future research could explore more efficient zooming strategies and broader application scenarios, such as real-time monitoring and dynamic environmental analysis.
AI Executive Summary
In ultra-high-resolution remote sensing images, task-relevant cues are often sparse and tiny, making it challenging for existing methods to effectively acquire evidence. GeoEyes addresses this by introducing the UHR-CoZ dataset and the AdaZoom-GRPO algorithm, solving the issue of tool usage homogenization and significantly improving evidence acquisition efficiency and answer accuracy.
GeoEyes achieved 54.23% accuracy on XLRS-Bench, significantly outperforming existing multimodal large language models. Its dynamic zooming strategy allows the model to perform multi-round zooming when needed and stop when unnecessary, avoiding unnecessary computational overhead.
This approach is not only significant academically but also offers new possibilities for practical applications in remote sensing, such as improving the precision and efficiency of satellite image analysis. Future research can further optimize zooming strategies and explore its applications in real-time monitoring and dynamic environmental analysis.
Deep Analysis
Background
Remote sensing technology plays an increasingly important role in Earth sciences, particularly ultra-high-resolution satellite images that capture fine-grained structures of complex surface patterns. However, existing multimodal large language models often struggle to effectively acquire task-relevant fine-grained information from these images.
Core Problem
In ultra-high-resolution remote sensing scenarios, task-relevant cues are often sparse and tiny, making it difficult for existing methods to effectively acquire evidence, leading to limited answer accuracy. How to enhance the model's evidence acquisition capability without increasing computational overhead is a pressing issue.
Innovation
GeoEyes introduces the UHR-CoZ dataset and the AdaZoom-GRPO algorithm to implement task-adaptive zooming strategies. UHR-CoZ covers diverse zooming regimes, and AdaZoom-GRPO enhances evidence acquisition and answer improvement through reward mechanisms, significantly improving model adaptability in UHR scenarios.
Methodology
- �� Use the UHR-CoZ dataset for cold-start supervised fine-tuning, covering diverse zooming regimes.
- �� Apply the AdaZoom-GRPO algorithm to enhance evidence acquisition and answer improvement through reward mechanisms.
- �� Conduct experiments on XLRS-Bench to evaluate model accuracy and efficiency.
Experiments
Experiments were conducted on XLRS-Bench, using the UHR-CoZ dataset for cold-start supervised fine-tuning and applying the AdaZoom-GRPO algorithm for reinforcement learning. The experiments evaluated the model's performance across multiple subtasks, including object classification and overall counting.
Results
GeoEyes achieved 54.23% accuracy on XLRS-Bench, significantly outperforming existing multimodal large language models. In object classification tasks, GeoEyes reached 66.1% accuracy and excelled in overall counting tasks.
Applications
GeoEyes can be used to improve the precision and efficiency of satellite image analysis, especially in tasks requiring fine-grained perception, such as environmental monitoring and urban planning.
Limitations & Outlook
GeoEyes may still experience performance degradation in extremely complex scenarios, especially with highly sparse evidence. Additionally, the computational overhead is substantial, potentially limiting its application in resource-constrained environments.
Plain Language Accessible to non-experts
Imagine you're playing a massive jigsaw puzzle game, where only a few pieces are useful. GeoEyes acts like a smart assistant that can quickly find these useful pieces and put them together. It uses zooming tools to carefully examine each piece, ensuring it finds the most relevant information. This way, even if the puzzle is very complex, it can help you complete it quickly.
ELI14 Explained like you're 14
Imagine you're playing a huge jigsaw puzzle game with thousands of pieces, but only a few are useful. GeoEyes is like a super-smart helper that can quickly find those key pieces! It uses a magnifying glass to check each piece carefully, making sure nothing important is missed. So, even if the puzzle is super complex, it can help you solve it easily!
Glossary
GeoEyes
GeoEyes is a multimodal large language model for analyzing ultra-high-resolution remote sensing images, capable of implementing task-adaptive zooming strategies.
In the paper, GeoEyes is used to enhance the accuracy and efficiency of remote sensing image analysis.
UHR-CoZ
UHR-CoZ is a dataset for cold-start supervised fine-tuning, covering diverse zooming regimes.
In the paper, UHR-CoZ is used to initialize GeoEyes' basic visual planning capabilities.
AdaZoom-GRPO
AdaZoom-GRPO is a reinforcement learning method that enhances evidence acquisition and answer improvement through reward mechanisms.
In the paper, AdaZoom-GRPO is used to optimize GeoEyes' tool usage strategies.
XLRS-Bench
XLRS-Bench is a benchmark dataset for evaluating the performance of remote sensing image analysis models.
In the paper, XLRS-Bench is used to validate GeoEyes' accuracy and efficiency.
Tool Usage Homogenization
Tool usage homogenization refers to the model's tendency to use tools in a consistent pattern, leading to inefficient evidence acquisition.
In the paper, tool usage homogenization is a key issue that GeoEyes aims to address.
Open Questions Unanswered questions from this research
- 1 How to further enhance model performance in extremely complex scenarios without increasing computational overhead.
- 2 How to optimize GeoEyes' zooming strategies for broader application scenarios.
Applications
Immediate Applications
Satellite Image Analysis
GeoEyes can be used to improve the precision and efficiency of satellite image analysis, especially in tasks requiring fine-grained perception.
Long-term Vision
Real-time Environmental Monitoring
GeoEyes has the potential to be used for real-time environmental monitoring, improving monitoring precision through dynamic zooming strategies.
Abstract
The "thinking-with-images" paradigm enables multimodal large language models (MLLMs) to actively explore visual scenes via zoom-in tools. This is essential for ultra-high-resolution (UHR) remote sensing VQA, where task-relevant cues are sparse and tiny. However, we observe a consistent failure mode in existing zoom-enabled MLLMs: Tool Usage Homogenization, where tool calls collapse into task-agnostic patterns, limiting effective evidence acquisition. To address this, we propose GeoEyes, a staged training framework consisting of (1) a cold-start SFT dataset, UHR Chain-of-Zoom (UHR-CoZ), which covers diverse zooming regimes, and (2) an agentic reinforcement learning method, AdaZoom-GRPO, that explicitly rewards evidence gain and answer improvement during zoom interactions. The resulting model learns on-demand zooming with proper stopping behavior and achieves substantial improvements on UHR remote sensing benchmarks, with 54.23% accuracy on XLRS-Bench.