LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning
LiteGUI enhances lightweight GUI agents using Guided On-policy Distillation and Multi-solution Dual-level GRPO, achieving SOTA performance.
Key Findings
Methodology
This paper introduces a novel training paradigm that significantly enhances the performance of lightweight GUI agents through Guided On-policy Distillation and a Multi-solution Dual-level GRPO framework. Guided On-policy Distillation integrates oracle reference trajectories with a dynamic retrieval mechanism to reduce hallucinations and mitigate cognitive misalignment in multi-solution tasks. The Multi-solution Dual-level GRPO framework aligns macro-level subtask planning with micro-level execution matching, improving exploration in long-horizon tasks.
Key Results
- On ScreenSpot-Pro, LiteGUI achieves text and icon recognition accuracy of 52.79% and 15.62%, significantly outperforming other lightweight models.
- On OS-World and Lite-Bench, LiteGUI's task success rate is comparable to larger models, demonstrating strong adaptability across various environments.
- Ablation studies show that Guided On-policy Distillation and Multi-solution Dual-level exploration fully unlock the potential of 2B/3B scale agents, surpassing traditional imitation learning limits.
Significance
This research introduces a new training paradigm that significantly improves the performance of lightweight GUI agents, achieving or approaching the level of large-scale models across various benchmarks. This not only provides new possibilities for on-device lightweight models but also offers more efficient solutions for cross-platform automated interaction. By addressing the overfitting and policy rigidity issues of traditional supervised fine-tuning, this method better supports error recovery in dynamic UI environments.
Technical Contribution
The technical contribution of this paper lies in proposing an SFT-free training paradigm that breaks through the performance bottlenecks of traditional methods through Guided On-policy Distillation and a Multi-solution Dual-level GRPO framework. Guided On-policy Distillation reduces hallucinations with a dynamic retrieval mechanism, while the Multi-solution Dual-level GRPO framework enhances exploration capabilities in long-horizon tasks by aligning macro and micro tasks.
Novelty
This is the first attempt to apply Guided On-policy Distillation to the GUI agent domain, significantly reducing cognitive misalignment in multi-solution tasks by integrating oracle reference trajectories and dynamic retrieval mechanisms. Compared to existing methods, this approach offers more flexible exploration capabilities in handling multi-solution tasks.
Limitations
- In some complex UI scenarios, the model may still exhibit policy rigidity, affecting task success rates.
- The automation level of the data generation pipeline is limited and may require significant manual intervention.
Future Work
Future research directions include further optimizing the data generation pipeline to improve automation and exploring more efficient mechanisms for handling multi-solution tasks.
AI Executive Summary
In modern computer usage, the development of GUI agents has garnered increasing attention. However, existing lightweight models often struggle with complex tasks, and traditional supervised fine-tuning methods tend to lead to overfitting and policy rigidity. To address these issues, this paper proposes a novel training paradigm that significantly enhances the performance of lightweight GUI agents through Guided On-policy Distillation and a Multi-solution Dual-level GRPO framework.
Guided On-policy Distillation integrates oracle reference trajectories with a dynamic retrieval mechanism to reduce hallucinations and mitigate cognitive misalignment in multi-solution tasks. The Multi-solution Dual-level GRPO framework aligns macro-level subtask planning with micro-level execution matching, improving exploration in long-horizon tasks. Additionally, an automated data generation pipeline is constructed to synthesize GUI task trajectories with rich multi-solution annotations.
Experimental results show that LiteGUI achieves state-of-the-art performance across multiple benchmarks, particularly excelling among lightweight models. Ablation studies further demonstrate the effectiveness of Guided On-policy Distillation and Multi-solution Dual-level exploration, fully unlocking the potential of 2B/3B scale agents. Future research directions include further optimizing the data generation pipeline and exploring more efficient mechanisms for handling multi-solution tasks.
Deep Analysis
Background
In recent years, automated computer use has received increasing attention, with GUI agents emerging as a research hotspot. These systems leverage visual understanding to interpret and execute computer-based tasks. While prior work has primarily focused on building GUI agents using large-scale models, the development of lightweight models suitable for on-device or edge deployment remains largely underexplored. Smaller models are typically considered insufficient for handling such complex tasks due to their limited capacity.
Core Problem
Existing lightweight GUI agents often struggle with complex tasks, and traditional supervised fine-tuning methods tend to lead to overfitting and policy rigidity. This limitation degrades zero-shot generalization and hinders effective error recovery in dynamic UI environments.
Innovation
This paper proposes a novel training paradigm that significantly enhances the performance of lightweight GUI agents through Guided On-policy Distillation and a Multi-solution Dual-level GRPO framework. Guided On-policy Distillation integrates oracle reference trajectories with a dynamic retrieval mechanism to reduce hallucinations and mitigate cognitive misalignment in multi-solution tasks.
Methodology
- �� Guided On-policy Distillation: Integrates oracle reference trajectories and dynamic retrieval mechanisms to reduce hallucinations.
- �� Multi-solution Dual-level GRPO framework: Aligns macro-level subtask planning with micro-level execution matching to enhance exploration capabilities.
- �� Data generation pipeline: Synthesizes GUI task trajectories with rich multi-solution annotations.
Experiments
Experiments were conducted using three benchmarks: ScreenSpot-Pro, OS-World, and Lite-Bench, to evaluate LiteGUI's performance. Training was conducted using the enhanced ShowUI-Desktop-8k dataset for ScreenSpot-Pro and the Lite-dataset for OS-World and Lite-Bench, which includes 30,000 complete long-horizon paths and 11,000 multi-solution annotations.
Results
LiteGUI achieves text and icon recognition accuracy of 52.79% and 15.62% on ScreenSpot-Pro, significantly outperforming other lightweight models. On OS-World and Lite-Bench, LiteGUI's task success rate is comparable to larger models.
Applications
LiteGUI can be used for cross-platform automated interaction, particularly suitable for resource-constrained on-device applications. Its efficient task execution capabilities make it widely applicable across various computer usage scenarios.
Limitations & Outlook
Despite its strong performance across multiple benchmarks, LiteGUI may still exhibit policy rigidity in some complex UI scenarios. Additionally, the automation level of the data generation pipeline is limited and may require significant manual intervention.
Plain Language Accessible to non-experts
Imagine you're cooking in a kitchen. Traditionally, you follow a recipe step by step, but sometimes you find the recipe too rigid, leaving you stuck when facing different ingredients. LiteGUI is like a smart assistant that not only suggests based on the recipe but also adjusts steps flexibly. For instance, if you're missing an ingredient, it suggests alternatives. This way, it helps you complete tasks more efficiently in the kitchen.
ELI14 Explained like you're 14
Hey there! Imagine you're playing a super complex game where you need to control lots of characters at once. LiteGUI is like a super smart helper that guides you to make the best choices in the game. It remembers all your past decisions and gives you the best advice based on the current situation. Just like in a game, it helps you find the fastest way to win, making you a game master!
Glossary
Guided On-policy Distillation
Integrates oracle reference trajectories and dynamic retrieval mechanisms to reduce hallucinations and mitigate cognitive misalignment in multi-solution tasks.
Used to enhance the performance of lightweight GUI agents.
Multi-solution Dual-level GRPO
Aligns macro-level subtask planning with micro-level execution matching to enhance exploration capabilities in long-horizon tasks.
Used to improve exploration capabilities in long-horizon tasks.
ScreenSpot-Pro
A benchmark for evaluating GUI agent performance, containing various task types.
Used to evaluate LiteGUI's performance.
Lite-dataset
A GUI dataset containing 30,000 complete long-horizon paths and 11,000 multi-solution annotations.
Used to train LiteGUI.
Reinforcement Learning
A machine learning method that learns optimal policies through interaction with the environment.
Used to enhance LiteGUI's task execution capabilities.
Open Questions Unanswered questions from this research
- 1 How to further reduce policy rigidity in complex UI scenarios?
- 2 How to improve the automation level of the data generation pipeline?
Applications
Immediate Applications
Cross-platform Automated Interaction
LiteGUI can enhance task execution efficiency for on-device applications, especially in resource-constrained environments.
Long-term Vision
Intelligent UI Agents
As technology advances, LiteGUI may become a core component of intelligent UI agents, offering more efficient user interaction experiences.
Abstract
Developing lightweight, on-device vision-language GUI agents is essential for efficient cross-platform automated interaction. However, current on-device agents are constrained by limited model capacity, and further performance improvements remain urgently needed. Traditional Supervised Fine-Tuning (SFT) for small-scale models often leads to overfitting, catastrophic forgetting and policy rigidity, and thus fails to fully address these challenges. In this work, we propose a novel SFT-free training paradigm that significantly enhances the performance of small-scale models. We first present the initial systematic integration of generalized knowledge distillation into the GUI agent domain via Guided On-policy Distillation. By incorporating oracle reference trajectories together with a dynamic retrieval mechanism, our method reduces hallucinations and mitigates the cognitive misalignment inherent in multi-solution GUI tasks. Building on this foundation, we further introduce a Multi-solution Dual-level GRPO framework that jointly aligns macro-level subtask planning with micro-level execution matching, thereby improving exploration in long-horizon GUI agent scenarios. In addition, we construct an automated data generation pipeline to synthesize GUI task trajectories with rich multi-solution annotations. Extensive experiments show that our method achieves state-of-the-art performance among lightweight models while remaining competitive with substantially larger-scale models across all benchmarks. Ablation studies further demonstrate that structured on-policy distillation and multi-solution dual-level exploration can fully unlock the capabilities of 2B/3B scale agents, surpassing the performance limits of conventional imitation learning.