Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training
Mobile-R1 enhances VLM-based mobile agents' interactive capabilities through systematic training, significantly improving exploration and self-correction.
Key Findings
Methodology
Mobile-R1 employs a three-stage hierarchical curriculum: format alignment, on-policy exploration, and multi-turn task training. Initially, format alignment uses high-quality action trajectory samples for cold start. Then, on-policy exploration stabilizes basic execution with verifiable action feedback. Finally, multi-turn task training in realistic environments fosters exploration and self-correction.
Key Results
- Mobile-R1 achieved an accuracy of 78.55%, outperforming the best baseline by 2.7 points, demonstrating significant improvement in task success rate.
- In long-tail application task success, Mobile-R1 reached 37.40%, showing clear advantages over other models.
- Through multi-turn online reinforcement learning, Mobile-R1 demonstrated the ability to self-correct from incorrect states.
Significance
This research significantly enhances the interactive capabilities of VLM-based mobile agents through systematic training, addressing existing methods' shortcomings in exploration and error correction. Notably, it provides a high-quality Chinese mobile dataset, filling the gap in diverse GUI data.
Technical Contribution
Mobile-R1 introduces a systematic training method combining format alignment, on-policy exploration, and multi-turn task training, significantly enhancing agents' exploration and self-correction capabilities. Compared to existing methods, Mobile-R1 excels in applying task-level rewards, addressing training instability caused by sparse rewards.
Novelty
Mobile-R1 is the first to apply a three-stage hierarchical curriculum to VLM-based mobile agent training, combining verifiable action feedback and multi-turn task training, significantly enhancing agents' exploration and self-correction capabilities.
Limitations
- Training in extremely sparse reward environments may still pose challenges for Mobile-R1.
- The model may require further optimization when handling complex multi-task environments.
Future Work
Future research could explore applying Mobile-R1 in more complex multi-task environments and further optimize its performance in extremely sparse reward settings.
AI Executive Summary
Vision-language models (VLMs) are increasingly applied in mobile agents, but existing methods fall short in exploration and error correction. Mobile-R1 significantly enhances VLM-based mobile agents' interactive capabilities through systematic training.
Mobile-R1 employs a three-stage hierarchical curriculum: format alignment, on-policy exploration, and multi-turn task training. Format alignment uses high-quality action trajectory samples for cold start, on-policy exploration stabilizes basic execution with verifiable action feedback, and multi-turn task training in realistic environments fosters exploration and self-correction.
Experimental results show that Mobile-R1 outperforms existing methods in accuracy and task success rate, particularly excelling in long-tail application task success. This research also provides a high-quality Chinese mobile dataset, filling the gap in diverse GUI data, offering valuable resources for research in non-English environments.
Deep Dive
Abstract
Vision-language model-based mobile agents have gained the ability to understand complex instructions and mobile screenshots, benefiting from reinforcement learning paradigms like Group Relative Policy Optimization (GRPO). However, existing approaches centers on offline training or local action-level rewards often trap agents in local optima, hindering effective exploration and error correction with the environment. Crucially, we find that directly applying task-level rewards often leads to convergence difficulties due to the sparse nature of GUI interactions. To address these challenges, we present \textbf{Mobile-R1}, a systematic training recipe that bridges atomic action execution and strategic task completion. We propose a hierarchical curriculum consisting of three stages: (1) format alignment for reasoning structure, (2) on-policy exploration with verifiable action feedback to ground basic execution, and (3) multi-turn task-level training with realistic environment to unlock exploration and self-correction. This hierarchical strategy effectively bootstraps the agent, significantly enhancing its capability for exploration and self-correction (the ``Eureka'' moments). Furthermore, addressing the critical scarcity of diverse GUI data in non-English ecosystems, we contribute a comprehensive Chinese mobile dataset covering 28 applications with 24,521 high-quality manual annotations, and establish a rigorous benchmark with 500 trajectories. We will open source all resources, including the dataset, benchmark, model weight, and codes: https://mobile-r1.github.io/Mobile-R1/.