MobileForge: Annotation-Free Adaptation for Mobile GUI Agents with Hierarchical Feedback-Guided Policy Optimization
MobileForge uses hierarchical feedback-guided policy optimization to adapt mobile GUI agents without annotations, achieving 67.2% Pass@3 on AndroidWorld.
Key Findings
Methodology
MobileForge integrates MobileGym and HiFPO modules. MobileGym conducts real app interaction for task generation and rollout evaluation, employing UI state graph traversal and trajectory mining. HiFPO leverages multi-attempt feedback—outcome, process, and correction hints—to generate hint-contextualized gradient updates (GRPO). The process involves trajectory filtering, hierarchical feedback incorporation, and multi-try rollout strategies, optimizing policies via an adapted PPO framework. This design effectively connects real app exploration with multi-level feedback, enabling annotation-free adaptation.
Key Results
- On AndroidWorld, Qwen3-VL-8B trained with MobileForge achieves 67.2% Pass@3, close to the 69.0% of the GUI-specialized GUI-Owl-1.5-8B. The adapted ForgeOwl-8B reaches 77.6% Pass@3 on AndroidWorld and 41.0% success on out-of-domain MobileWorld, surpassing previous open-data mobile GUI agents.
- Scaling experiments show performance improves with more tasks (200, 400, 900), demonstrating the positive correlation between data volume and transferability. The hierarchical feedback mechanism significantly enhances sample efficiency and generalization compared to isolated rollouts and coarse rewards.
- Ablation studies reveal that incorporating corrective hints and hierarchical feedback boosts success rates from 52% to 77%, and Pass@3 from 49% to 72.5%. Hint-guided multi-try rollout and feedback filtering outperform traditional supervised fine-tuning and reward-based methods, confirming the effectiveness of the proposed approach.
Significance
This work addresses key bottlenecks in mobile GUI agent transfer: the lack of a unified interaction and evaluation platform, and the reliance on sparse rewards. By enabling autonomous exploration, multi-level feedback, and iterative policy refinement without manual annotations, it significantly reduces development costs and accelerates deployment. The framework fosters scalable, adaptable, and robust mobile agents, advancing the field toward fully autonomous, self-improving systems capable of handling diverse, evolving app ecosystems, thus bridging the gap between research and real-world deployment.
Technical Contribution
The core innovations include MobileGym as a real interaction substrate, and HiFPO for feedback-driven policy updates. The approach combines trajectory-based task mining, multi-level feedback filtering, and hint-guided multi-try rollouts, integrated into a reinforcement learning framework with local gradient updates. The algorithm introduces a novel hierarchical feedback mechanism that filters and reuses experience, improving sample efficiency and stability. Engineering-wise, it achieves end-to-end automatic task grounding, multi-layer feedback fusion, and scalable policy training, setting new standards for annotation-free mobile agent adaptation.
Novelty
This is the first work to embed hierarchical, multi-level feedback into a real app interaction platform for annotation-free mobile GUI adaptation. Unlike prior methods relying solely on coarse rewards or isolated exploration, it systematically leverages outcome, process, and correction hints to guide multi-try rollouts and policy updates. The integration of trajectory filtering, task variation generation, and hint-contextualized gradient updates constitutes a novel paradigm, markedly improving transferability and robustness in diverse app environments.
Limitations
- The current framework is primarily validated on Android apps; cross-platform generalization to iOS or multi-modal inputs remains unaddressed.
- Handling long-horizon, multi-application workflows with persistent states and complex dependencies still poses challenges, especially in multi-task scenarios.
- Training involves significant computational resources due to large-scale automatic data generation and multi-try rollouts; real-time deployment efficiency needs further optimization.
Future Work
Future directions include extending the approach to cross-platform environments, integrating multi-modal inputs like vision and speech, and enhancing long-term memory for multi-application workflows. Improving training efficiency, reducing computational costs, and enabling continual learning in dynamic app ecosystems are also key goals. Additionally, exploring multi-agent collaboration and multi-task generalization will further broaden applicability and robustness.
AI Executive Summary
MobileForge introduces a pioneering framework for annotation-free adaptation of mobile GUI agents, addressing longstanding challenges in transferability and scalability. Central to this system are MobileGym and HiFPO modules. MobileGym acts as a real interaction platform, enabling agents to explore target apps, generate executable tasks, and evaluate rollouts using trajectory mining and UI state graph traversal. This grounding in real app behavior ensures that generated tasks are relevant and meaningful. HiFPO then leverages multi-level hierarchical feedback—comprising outcome success, step-wise process insights, and corrective hints—to iteratively refine policies. This feedback is used to filter, select, and optimize local decision steps through a hint-guided multi-try rollout process, employing an adapted PPO algorithm with local gradient updates. The approach effectively bridges the gap between exploration and policy learning, overcoming the limitations of sparse rewards and isolated rollouts common in prior work.
Deep Analysis
Background
随着大规模多模态预训练模型（如GPT、CLIP）在UI理解中的应用，移动GUI智能体逐渐成为研究热点。早期工作如UI-Venus、ShowUI等，依赖大量标注数据，效果有限。近年来，无标注迁移方法如TongUI、MobileA3gent、GUI-explorer等，尝试利用自主探索和自我反馈减少人工干预，但仍面临任务关联薄弱和反馈稀疏的问题。传统方法多依赖孤立rollouts和粗糙奖励，难以实现高效迁移。MobileForge在此基础上，提出端到端的无标注迁移框架，结合真实应用交互和多层次反馈，推动了移动GUI智能体的自主学习能力。
Core Problem
现有移动GUI迁移方法依赖大量人工标注和任务定义，成本高且易过时。应用场景复杂多变，模型难以在新应用中快速适应，尤其在多任务、多应用环境下，策略的泛化和长时序规划成为难点。孤立的rollouts和稀疏奖励机制限制了策略的样本效率和迁移能力，导致模型在实际应用中表现不佳。如何建立一个无需标注、能在真实环境中自主挖掘任务、利用多层次反馈持续优化的迁移体系，成为亟待解决的问题。
Innovation
MobileForge的创新在于：1）提出MobileGym作为真实应用交互的探索平台，实现任务挖掘和轨迹评估；2）引入层级反馈机制，将轨迹结果、过程信息和修正提示融合，构建多层次反馈体系；3）开发HiFPO策略，通过多尝试和反馈过滤，有效利用轨迹信息进行策略优化；4）结合Hint引导的多尝试rollout，提升样本利用率。与传统方法相比，显著改善了任务关联和奖励稀疏的问题，增强模型的迁移能力。
Methodology
- �� MobileGym在真实移动应用中探索状态空间，挖掘任务变体，通过深度优先搜索和UI状态图实现目标导向。• 任务生成基于轨迹的连贯性筛选，生成多样化变体，确保任务的多样性和实用性。• 轨迹评估由MobileGym-Critic完成，提供任务完成状态、步骤合理性和修正提示，形成多层次反馈。• HiFPO利用多次尝试的反馈信息，过滤已掌握任务，筛选困难步骤，结合Hint引导优化策略。• 采用Hint-guided多尝试rollout，重复利用反馈信息，逐步改进策略。• 训练过程中引入基于轨迹的过滤和局部梯度更新，结合PPO变体实现稳定优化。
Experiments
在AndroidWorld和MobileWorld两个数据集上验证，训练Qwen3-VL-8B和GUI-Owl-1.5-8B模型。采用200、400、900任务子集进行迁移训练，评估指标为Pass@1/2/3和成功率。对比传统孤立rollouts、奖励稀疏机制和不同反馈策略，进行消融分析。模型性能在不同任务难度和跨域场景中测试，验证迁移效果和泛化能力。实验还包括不同任务筛选策略、反馈过滤和评价器模型的影响。
Results
在AndroidWorld，Qwen3-VL-8B通过900任务训练后，Pass@3达67.2%，接近专用模型69.0%。ForgeOwl-8B在相同条件下达77.6%，跨域MobileWorld成功率达41.0%。消融实验显示，修正提示和层级反馈显著提升成功率和样本效率。多尝试策略和Hint引导训练优于传统SFT，验证了层级反馈机制的有效性。整体结果表明，无标注迁移在移动GUI任务中具有巨大潜力。
Applications
该方法适用于自动化测试、智能助手和UI设计优化，无需手工标注即可实现模型快速适应新应用。可广泛应用于移动设备管理、个性化推荐和交互式教育等场景，降低开发成本，提高用户体验。未来还可结合多模态信息，扩展到跨平台、多任务和长时序场景，推动智能移动生态系统的发展。
Limitations & Outlook
目前主要在Android环境验证，跨平台迁移和多模态融合仍待突破。长时序、多应用场景下的跨任务记忆和规划能力有限，模型在复杂多任务环境中的表现仍不足。训练成本较高，依赖大量自动生成任务，实时性和能效需优化。未来需解决多模态融合、长远记忆和跨平台迁移的挑战。
Plain Language Accessible to non-experts
想象你在一家大型工厂工作，工厂里有很多不同的机器和流程。以前，你需要手动告诉每台机器该做什么，怎么操作，才能完成一项任务。现在，工厂引入了一台聪明的机器人，它可以自己观察机器的工作状态，学习不同的操作步骤，甚至在遇到问题时自己想办法修正。这个机器人不用提前告诉它所有细节，只需要通过观察和尝试，就能逐渐掌握工厂的全部流程。它会不断试错、总结经验，然后用这些经验去完成更复杂的任务。MobileForge就像这个聪明的机器人，它在真实的手机应用中学习如何操作，而不用依赖人工提前写好的任务或奖励。它通过不断尝试、观察和修正，逐步变得更聪明、更可靠，最终可以自主完成各种复杂的手机操作任务。这种方式大大降低了人工干预的成本，也让智能体变得更加灵活和适应性强。
ELI14 Explained like you're 14
想象你在学校里学习做菜，你没有老师告诉你每一步怎么做，而是自己试一试，遇到问题再想办法修正。比如，你试着做一个三明治，发现面包不够新鲜，你就换个面包或者调整配料。每次尝试后，你会记住哪些步骤可以成功，哪些需要改进。慢慢地，你变得越来越擅长做各种菜。这就像MobileForge在手机应用中学习操作一样，它不依赖老师提前写好的步骤，而是通过自己尝试、观察和修正，不断学习。它会在真实的手机环境中试验不同的操作，得到反馈，然后用这些反馈改进自己的行为。这样，机器人可以自己学会很多复杂的任务，而不用每次都由人来告诉它怎么做。这种学习方式就像你自己在厨房里摸索，最终变成了大厨！
Abstract
MLLM-based mobile GUI agents have made substantial progress in UI understanding and action execution, but adapting them to real target apps remains costly because mobile apps are numerous, frequently updated, and hard to cover with human-written tasks, demonstrations, or reward labels. Existing annotation-free GUI learning reduces manual supervision, yet lacks a unified substrate connecting target-app exploration, curriculum mining, rollout execution, and feedback, while policy optimization often relies on isolated rollouts and coarse rewards that are hard to convert into reliable improvement signals. We present MobileForge, an annotation-free adaptation system for mobile GUI agents. MobileForge consists of MobileGym, which grounds task generation and rollout evaluation in real mobile app interaction, and Hierarchical Feedback-Guided Policy Optimization (HiFPO), which turns trajectory outcomes, step-level process feedback, and corrective hints into hint-contextualized step-level GRPO updates. Using only automatically generated annotation-free adaptation data, MobileForge adapts Qwen3-VL-8B to 67.2% Pass@3 on AndroidWorld, close to the closed-data GUI-specialized GUI-Owl-1.5-8B base model at 69.0%. The MobileForge-adapted ForgeOwl-8B further reaches 77.6% Pass@3 on AndroidWorld and 41.0% success on the out-of-domain MobileWorld GUI-only split, establishing the strongest open-data mobile GUI agent in our evaluation. Code, data, and trained models will be released at https://mobile-forge.github.io/.