Faithful Mobile GUI Agents with Guided Advantage Estimator
Faithful-Agent framework boosts Trap SR from 13.88% to 80.21% using Guided Advantage Estimator.
Key Findings
Methodology
The study introduces the Faithful-Agent framework, enhancing GUI agent faithfulness through a two-stage pipeline. Stage I is a faithfulness-oriented SFT to instill abstainment behaviors under evidence perturbations. Stage II is RFT, utilizing the Guided Advantage Estimator (GuAE) to prevent advantage collapse in low-variance rollout groups, and a thought-action consistency reward to elevate faithfulness.
Key Results
- Experiments show Faithful-Agent (Stage II) elevates Trap SR from 13.88% to 80.21% relative to the baseline, while preserving robust general instruction-following performance.
- Across scenarios, Faithful-Agent achieves an overall success rate of 71.67%.
- GuAE effectively mitigates advantage collapse compared to different GRPO variants.
Significance
This research is significant in the field of vision-language model-based GUI interaction. It addresses long-standing pain points regarding evidence grounding and internal consistency, providing a more reliable automation solution.
Technical Contribution
Technical contributions include the introduction of the Guided Advantage Estimator (GuAE), maintaining non-trivial advantages in low-variance rollout groups, and enhancing faithfulness through thought-action consistency rewards. These designs fundamentally differ from existing SOTA methods, offering new theoretical guarantees and engineering possibilities.
Novelty
This is the first to treat faithfulness as the core problem for GUI agents and solve advantage collapse using Guided Advantage Estimator, fundamentally innovating compared to existing methods.
Limitations
- In extremely sparse reward scenarios, the system may still experience faithfulness degradation.
- Complex user interfaces may lead to model misjudgments.
Future Work
Future directions include exploring more complex user interface scenarios and further optimizing the Guided Advantage Estimator to adapt to different reward structures.
AI Executive Summary
Graphical User Interface (GUI) agents supported by vision-language models have demonstrated strong interaction capabilities. However, these agents often rely on memorized shortcuts rather than grounding actions in displayed screen evidence or user instructions. To address this, the research team proposes Faithful-Agent, a faithfulness-first framework that reformulates GUI interaction to prioritize evidence groundedness and internal consistency. The framework employs a two-stage pipeline: Stage I is a faithfulness-oriented SFT to instill abstainment behaviors under evidence perturbations; Stage II is RFT, utilizing the Guided Advantage Estimator (GuAE) to prevent advantage collapse in low-variance rollout groups, and a thought-action consistency reward to elevate faithfulness. Experiments show Faithful-Agent (Stage II) elevates Trap SR from 13.88% to 80.21% relative to the baseline, while preserving robust general instruction-following performance. This research is significant in the field of vision-language model-based GUI interaction, addressing long-standing pain points regarding evidence grounding and internal consistency, providing a more reliable automation solution.
Deep Analysis
Background
In recent years, vision-language models (VLMs) have made significant advances in graphical user interface (GUI) interaction. These models can process visual observations and textual instructions to generate executable actions. However, existing GUI agents often rely on memorized shortcuts rather than grounding actions in displayed screen evidence or user instructions. This unfaithful behavior leads to interaction failures, affecting the reliability of automation.
Core Problem
The core problem lies in the unfaithful behavior of GUI agents, including failing to condition decisions on interface states and deviating from user instructions. This behavior leads to internal inconsistency and occasional erratic interactions, affecting the reliability of automation.
Innovation
The study proposes the Faithful-Agent framework, enhancing GUI agent faithfulness through a two-stage pipeline. Stage I is a faithfulness-oriented SFT to instill abstainment behaviors under evidence perturbations. Stage II is RFT, utilizing the Guided Advantage Estimator (GuAE) to prevent advantage collapse in low-variance rollout groups, and a thought-action consistency reward to elevate faithfulness.
Methodology
- �� Faithfulness-oriented SFT: Instills abstainment behaviors under evidence perturbations.
- �� RFT with GuAE: Prevents advantage collapse in low-variance rollout groups.
- �� Thought-action consistency reward: Enhances alignment between stated intent and executed actions.
Experiments
The experimental design includes training and testing on multiple datasets, comparing different GRPO variants and reward designs. Key hyperparameters include rollout group size and reward combination weights. Ablation studies assess the contribution of each component.
Results
Experiments show Faithful-Agent (Stage II) elevates Trap SR from 13.88% to 80.21% relative to the baseline, while preserving robust general instruction-following performance. GuAE effectively mitigates advantage collapse compared to different GRPO variants.
Applications
The framework can be applied to automation interactions in mobile, web, and desktop environments, especially in scenarios requiring high faithfulness. It provides a more reliable automation solution for the industry.
Limitations & Outlook
In extremely sparse reward scenarios, the system may still experience faithfulness degradation. Complex user interfaces may lead to model misjudgments. Future directions include exploring more complex user interface scenarios and further optimizing the Guided Advantage Estimator to adapt to different reward structures.
Plain Language Accessible to non-experts
Imagine a factory where workers are on a production line. Each worker has their task, but sometimes they make mistakes due to memorized shortcuts. Faithful-Agent acts like a smart supervisor, ensuring workers make decisions based on actual conditions rather than memory. It improves worker accuracy through two training stages: the first stage teaches them to stop working when information is incomplete, and the second stage helps them make better decisions in complex situations.
ELI14 Explained like you're 14
Imagine you're playing a game where you need to make decisions based on screen prompts. But sometimes, the screen information is incomplete, and you might make wrong decisions. Faithful-Agent is like a smart assistant, helping you make the right choices when information is incomplete. It improves your gaming skills through two training stages: the first stage teaches you to stop when information is incomplete, and the second stage helps you make better decisions in complex situations.
Glossary
Faithful-Agent
A faithfulness-first framework designed to enhance GUI agent faithfulness.
Used to address unfaithful behavior in GUI agents.
Guided Advantage Estimator
A mechanism to prevent advantage collapse in low-variance rollout groups.
Used to enhance faithfulness in the RFT stage.
SFT
A training stage that instills abstainment behaviors under evidence perturbations.
Used in Stage I of Faithful-Agent training.
RFT
A training stage that enhances faithfulness using Guided Advantage Estimator.
Used in Stage II of Faithful-Agent training.
Trap SR
Measures agent success rate in complex scenarios.
Used to evaluate Faithful-Agent performance.
Open Questions Unanswered questions from this research
- 1 How to further improve faithfulness in extremely sparse reward scenarios?
- 2 How to optimize the model to reduce misjudgments in complex user interface scenarios?
Applications
Immediate Applications
Mobile Automation
The framework can be used to enhance the reliability of automation interactions on mobile devices.
Long-term Vision
Cross-Platform Automation
The framework has the potential to achieve high faithfulness automation interactions across multiple platforms.
Abstract
Vision-language model based graphical user interface (GUI) agents have shown strong interaction capabilities. However, they often behave unfaithfully, relying on memorized shortcuts rather than grounding actions in displayed screen evidence or user instructions. To address this, we propose Faithful-Agent, a faithfulness-first framework that reformulates GUI interaction to prioritize evidence groundedness and internal consistency. Faithful-Agent employs a two-stage pipeline: (i) a faithfulness-oriented SFT stage to instill abstainment behaviors under evidence perturbations; (ii) an RFT stage that further amplifies faithfulness by introducing the guided advantage estimator (GuAE), an anchor-based and variance-adaptive advantage tempering mechanism built upon GRPO. GuAE prevents advantage collapse in low-variance rollout groups under sparse GUI rewards, and with a thought-action consistency reward, Faithful-Agent (Stage II) elevates the Trap SR from 13.88\% to 80.21\% relative to the baseline, while preserving robust general instruction-following performance.