Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation
Proposes a reflection-guided self-distillation framework for test-time GUI grounding, achieving a 7.4% accuracy boost without human annotations.
Key Findings
Methodology
The proposed framework integrates four core stages: exploration, evaluation, reflection, and internalization. During deployment, the model predicts UI element coordinates based on instructions. An MLLM-based reflector assesses the prediction's correctness and generates a detailed reasoning reflection. To internalize this reflection, the authors develop Reflection-Guided On-Policy Self-Distillation (R-OPSD), which converts high-level reasoning into dense token-level supervision via a conditioned self-teacher. To prevent corrupted supervision from incorrect auto-regressive prefixes during failures, a Contrastive Calibration (CC) mechanism is introduced, leveraging inverse prompts to suppress erroneous tokens. The entire process is validated through extensive experiments across six benchmarks, demonstrating an average accuracy improvement of 7.4%, outperforming existing methods like GUI-RCPO.
Key Results
- Across six benchmarks, the framework consistently improves accuracy by an average of 7.4%. On the SSv2 dataset, the accuracy of Qwen2.5-VL-3B increases from 50.2% to 57.4%, a 7.2% gain, while Qwen3-VL-2B achieves a 4.6% increase on MMBench-GUI, reaching 72.6%. These results confirm the method's robustness and effectiveness in unseen environments.
- The introduction of reflection and contrastive calibration significantly enhances the model's ability to learn from both success and failure explorations, especially in scenarios with complete exploration failures where traditional reinforcement learning methods struggle.
- Comparison with prior test-time RL approaches shows that the proposed method achieves superior performance, with up to 7.7% higher accuracy, demonstrating the advantage of leveraging rich textual reflections and dense supervision signals.
Significance
This work pioneers the integration of reflection-based reasoning into test-time self-evolving frameworks for GUI visual grounding. It addresses the critical challenge of enabling models to adapt continuously without human annotations, a key step toward autonomous and resilient AI agents. By transforming high-level reasoning into dense supervision signals, the approach bridges the gap between symbolic reflection and gradient-based learning, opening new avenues for self-supervised adaptation in multimodal tasks. The framework's scalability and effectiveness across diverse benchmarks highlight its potential to revolutionize real-world human-computer interaction systems, making them more flexible, robust, and capable of lifelong learning.
Technical Contribution
The main technical innovations include: 1) the design of a closed-loop self-evolving framework that combines exploration, evaluation, reflection, and parameter internalization; 2) the development of Reflection-Guided On-Policy Self-Distillation (R-OPSD), which transforms reflection text into dense token supervision conditioned on evaluation outcomes; 3) the introduction of Contrastive Calibration (CC), which mitigates supervision corruption caused by incorrect auto-regressive prefixes through inverse prompting and advantage suppression; 4) extensive empirical validation demonstrating a 7.4% average accuracy gain over baseline models and outperforming prior test-time RL methods. These contributions collectively enable models to learn continuously from their own interactions, significantly advancing the state-of-the-art in GUI visual grounding.
Novelty
This research is the first to incorporate detailed textual reflections into a self-distillation framework for test-time adaptation in GUI visual grounding. Unlike prior methods relying solely on sparse rewards or static fine-tuning, it leverages high-level reasoning to guide dense token-level supervision, effectively internalizing failure analysis. The use of a multi-modal large language model as a reflector, combined with the novel Contrastive Calibration mechanism, marks a significant departure from existing reinforcement learning and supervised fine-tuning approaches, establishing a new paradigm for autonomous, reflection-driven model evolution.
Limitations
- The effectiveness heavily depends on the accuracy of the reflector; in complex or ambiguous interfaces, inaccurate reflections could misguide the internalization process.
- The computational overhead introduced by the reflection assessment, self-distillation, and contrastive calibration may limit real-time deployment, especially on large models or resource-constrained devices.
- The current framework primarily addresses single-step exploration and may face challenges in multi-step or long-horizon tasks requiring deeper reasoning and planning.
- Handling completely novel interface elements or drastic layout changes remains difficult, indicating a need for more robust generalization mechanisms.
Future Work
Future directions include enhancing the reflector's robustness through better multimodal reasoning capabilities, integrating multi-turn interaction for more complex tasks, and reducing computational costs via model compression or efficient inference techniques. Exploring meta-learning strategies to accelerate adaptation and extending the framework to other vision-language tasks, such as visual question answering or robotic manipulation, are promising avenues. Additionally, improving interpretability and safety measures will be crucial for deploying such autonomous systems in real-world applications.
AI Executive Summary
In the rapidly evolving landscape of human-computer interaction, graphical user interfaces (GUIs) serve as the primary bridge between users and digital systems. Automating the understanding and interaction with GUIs through AI models has become a critical research focus, especially for applications like automated testing, assistive technologies, and intelligent automation. Traditional models for GUI visual grounding rely heavily on supervised learning with large annotated datasets, which are costly and inflexible in dynamic environments. Once trained, these models typically operate in a static manner, unable to adapt to unseen interfaces or layout changes, limiting their practical deployment.
Recognizing this limitation, recent efforts have explored test-time adaptation techniques, such as reinforcement learning-based methods, to enable models to improve during deployment. However, these approaches often depend on sparse reward signals that only indicate success or failure, providing little insight into the reasons behind errors. Consequently, models struggle to internalize failure causes and cannot effectively learn from their mistakes, especially in complex or novel scenarios.
This paper introduces a novel framework called Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation. The core idea is to empower GUI agents with the ability to self-reflect and adapt continuously without human annotations. The framework constructs a closed-loop process comprising four stages: exploration, evaluation, reflection, and internalization. During exploration, the model predicts UI element coordinates based on instructions. An innovative multi-modal large language model (MLLM)-based reflector then assesses the correctness of these predictions and generates detailed reasoning reflections. These reflections analyze why a prediction succeeded or failed, providing rich semantic feedback.
The critical challenge is how to internalize these textual reflections into the model parameters. To address this, the authors propose Reflection-Guided On-Policy Self-Distillation (R-OPSD), which conditions a self-teacher model on the reflection and evaluation results. This teacher then provides dense token-level supervision signals, guiding the model to improve its predictions. To prevent corrupted supervision during failed explorations, a Contrastive Calibration (CC) mechanism is introduced. CC uses inverse prompts to contrast and suppress incorrect prefixes, ensuring stable learning even from failures.
Extensive experiments across six benchmark datasets demonstrate the effectiveness of the approach. The framework achieves an average accuracy improvement of 7.4%, significantly surpassing existing test-time RL methods like GUI-RCPO. Notably, it enables models to adapt and improve autonomously in unseen environments, addressing a long-standing challenge in GUI automation. The innovations in reflection-based supervision and contrastive calibration open new avenues for self-supervised, lifelong learning in multimodal AI systems.
Overall, this work marks a significant step toward autonomous GUI agents capable of continuous self-improvement. By integrating detailed reasoning, dense supervision, and robust failure mitigation, it paves the way for more resilient, adaptable, and intelligent human-computer interfaces. The potential applications span automated testing, assistive technologies, and beyond, promising a future where AI systems learn and evolve seamlessly in real-world settings.
Deep Analysis
Background
The evolution of GUI visual grounding has been driven by advances in deep learning, particularly supervised fine-tuning of large-scale datasets such as those used in Qin et al. (2025) and Cheng et al. (2024). Early methods focused on extracting visual features and aligning them with natural language instructions, employing attention mechanisms and specialized action heads. Reinforcement learning approaches, like Luo et al. (2025a), introduced reward signals based on interaction success, improving robustness in dynamic environments. Recently, test-time adaptation techniques, exemplified by GUI-RCPO, attempted to refine models during deployment by leveraging region consistency and test-time reinforcement learning. Despite these efforts, most models remain static post-training, unable to learn from new or unseen interfaces without human annotations, which limits their generalization and adaptability in real-world applications.
Core Problem
The core challenge addressed in this paper is enabling GUI visual grounding models to adapt continuously after deployment without relying on human-labeled data. Existing models are static once trained, and while test-time reinforcement learning provides some online adaptation, it relies on sparse rewards that do not offer insights into why predictions fail. This results in limited capacity for models to internalize failure reasons and improve iteratively. The problem is compounded in complex, unseen interfaces where layout variations and ambiguous visual cues hinder accurate localization. Overcoming these limitations requires a mechanism for models to self-assess, reflect, and learn from their own interactions, effectively closing the loop of autonomous self-improvement.
Innovation
The paper introduces a comprehensive self-evolving framework that integrates reflection and dense supervision to facilitate continuous learning. Key innovations include:
1) A reflection module powered by a multi-modal large language model (MLLM) that evaluates and explains the model’s predictions, providing rich semantic feedback.
2) Reflection-Guided On-Policy Self-Distillation (R-OPSD), which converts textual reflections into dense token-level supervision signals conditioned on evaluation outcomes, enabling effective internalization.
3) Contrastive Calibration (CC), a novel mechanism that mitigates supervision corruption by contrasting incorrect prefixes with inverse prompts, suppressing error propagation during failed explorations.
4) A closed-loop learning process that allows the model to explore, evaluate, reflect, and internalize knowledge iteratively, leading to progressive performance gains.
These innovations collectively enable the model to learn from its own reasoning process, significantly advancing the state-of-the-art in test-time adaptation for GUI visual grounding.
Methodology
- �� Exploration: The model predicts UI element coordinates based on an input screenshot and instruction, forming the initial exploration step.
- �� Evaluation & Reflection: The reflector, built on a multi-modal large language model, assesses the correctness of the prediction by analyzing the visual and textual inputs. It outputs a binary success/failure score and generates a detailed reasoning reflection explaining the evaluation.
- �� Reflection Processing: The reflection includes high-level reasoning about why the prediction succeeded or failed, such as misidentification of UI elements or incorrect localization.
- �� Internalization via R-OPSD: The reflection and evaluation results are used to condition a self-teacher model. This teacher provides dense token-level supervision signals by comparing the model’s generated coordinates with the reflection-based target.
- �� Contrastive Calibration: During failed explorations, the model employs inverse prompts (L(¬S)) to generate alternative supervision signals. The inverse student model is used to contrast and suppress incorrect prefixes, ensuring the supervision remains reliable.
- �� Model Update: The combined advantage signals guide the gradient updates, allowing the model to internalize the reflection insights and improve its localization accuracy iteratively.
- �� Validation: The entire process is validated across multiple datasets, with hyperparameters tuned for stability and performance, demonstrating the framework’s robustness.
Experiments
The experimental setup involves six benchmark datasets, including ScreenSpot, ScreenSpot-v2, ScreenSpot-Pro, MMBench-GUI, OSWorld-G, and OSWorld-G-Refine. The primary metric is Element Accuracy, measuring the percentage of predicted points within the target UI element's bounding box. The models are based on Qwen2.5-VL and Qwen3-VL architectures, fine-tuned with LoRA adapters to enable parameter-efficient training. During deployment, the models perform multiple exploration-evaluation-reflection-internalization cycles without human annotations, simulating real-world post-deployment adaptation. The reflection module is trained offline using a large dataset with annotated success/failure labels, and the entire framework is validated through ablation studies, hyperparameter tuning, and comparison with baseline methods like GUI-RCPO. The experiments demonstrate consistent performance gains, robustness to failure cases, and scalability to larger models, confirming the effectiveness of the proposed approach.
Results
The results show that the framework achieves an average accuracy increase of 7.4% across six benchmarks. On SSv2, the accuracy of Qwen2.5-VL-3B improves from 50.2% to 57.4%, while on MMBench-GUI, the accuracy of Qwen3-VL-2B increases from 68.0% to 72.6%. The method outperforms prior test-time reinforcement learning approaches, such as GUI-RCPO, by up to 7.7%. Ablation studies reveal that reflection and contrastive calibration are critical for performance, especially in scenarios with complete exploration failures. The framework demonstrates strong generalization, robustness, and ability to learn from failures, validating its potential for real-world deployment.
Applications
This approach is highly applicable to automated testing of GUIs, assistive technologies for users with disabilities, and autonomous interface interaction systems. Its ability to adapt continuously without human annotations reduces operational costs and enhances flexibility in dynamic environments. In industry, it can be integrated into software testing pipelines, robotic process automation, and intelligent virtual assistants, enabling these systems to self-improve over time. The framework's scalability suggests potential extensions to multi-turn dialogue systems, multi-modal reasoning tasks, and cross-platform interface adaptation, fostering more resilient and autonomous AI agents.
Limitations & Outlook
The reliance on the reflector's evaluation accuracy means that in highly complex or ambiguous interfaces, the reflection quality may degrade, affecting the overall learning process. The computational overhead of multiple inference steps, reflection generation, and contrastive calibration could limit real-time applications, especially on resource-constrained devices. The current framework primarily addresses single-step localization tasks; extending it to multi-step reasoning or sequential decision-making remains challenging. Additionally, handling drastic interface changes or entirely novel UI elements requires further robustness enhancements, possibly through meta-learning or continual learning strategies.
Plain Language Accessible to non-experts
Imagine you have a smart robot assistant that helps you find buttons or icons on your tablet or computer screen. When you tell it, 'Click the 'Next' button,' it tries to find that button and click it. Sometimes, it gets it right, but other times, it clicks the wrong spot, like on a blank space or a different button. Traditional robots learn by memorizing where things are, but if the layout changes, they forget.
Now, think of a robot that can think about its own mistakes. When it clicks the wrong place, it pauses and asks itself, 'Why did I get it wrong?' It then explains in simple words what went wrong—maybe it clicked the wrong icon or didn't see the button clearly. Based on this self-explanation, the robot adjusts its next move, learning from its own reasoning.
This process is like a student who makes a mistake on a test, then reviews the answer, figures out what went wrong, and remembers it for next time. The robot keeps doing this over and over, constantly learning and improving without anyone telling it the right answers every time. Over time, it becomes better at finding buttons on new, unfamiliar screens, just like a student gets better at tests by reviewing their mistakes.
This way, the robot becomes smarter and more adaptable, able to handle new interfaces and tricky layouts all by itself, making it a truly autonomous assistant that keeps learning and improving on the fly.
Abstract
GUI Visual Grounding is a fundamental capability for GUI agents. Existing models typically freeze their parameters after deployment, limiting their ability to adapt to unseen interfaces. Although recent methods attempt to adapt models via test-time reinforcement learning, they cannot reflect upon failed exploration. To overcome this, we propose a Test-Time Self-Evolving framework that enables models to improve after deployment without human-annotated ground truth. It constructs a closed-loop of Exploration, Evaluation, Reflection, and Internalization. Specifically, the agent first explores unseen interfaces by predicting grounding coordinates for given instructions. To evaluate these explorations, we introduce an MLLM-based Reflector to assess the generated results and provide the corresponding reasoning reflections. To internalize reflection knowledge into the model weights, we propose Reflection-Guided On-Policy Self-Distillation, which translates high-level reasoning into dense token-level supervision via a conditioned self-teacher. Furthermore, we design a Contrastive Calibration method to prevent incorrect auto-regressive prefixes from corrupting the supervisory signals during failed explorations. Extensive experiments across six benchmarks demonstrate our framework's effectiveness, achieving an average accuracy improvement of 7.4% over the base model. To the best of our knowledge, this is the first work to successfully exploit on-policy self-distillation for test-time adaptation in GUI visual grounding. By filling the gap in post-deployment adaptation, our framework completes the self-evolving capability of GUI agents. The code will be released.
References (20)
Qwen3-VL Technical Report
Shuai Bai, Yuxuan Cai, Ruizhe Chen et al.
UI-TARS: Pioneering Automated GUI Interaction with Native Agents
Yujia Qin, Yining Ye, Junjie Fang et al.
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents
Kanzhi Cheng, Qiushi Sun, Yougang Chu et al.
Test-Time Reinforcement Learning for GUI Grounding via Region Consistency
Yongkun Du, Yuchen Yan, Fei Tang et al.
MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents
Xuehui Wang, Zhenyu Wu, Jingjing Xie et al.
Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding
Yan Zhang, Daiqing Wu, Huawen Shen et al.
Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents
Boyu Gou, Ruohan Wang, Boyuan Zheng et al.
Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation
Jiaze Li, Hao Yin, Haoran Xu et al.
Grounding Computer Use Agents on Human Demonstrations
Aarash Feizi, Shravan Nayak, Xiangru Jian et al.
Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements
Ziwei Liu, Tao Feng, Borui Kang et al.
GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents
Yuqi Zhou, Sunhao Dai, Shuai Wang et al.
ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use
Kaixin Li, Ziyang Meng, Hongzhan Lin et al.
RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation
Leyi Pan, Shuchang Tao, Yunpeng Zhai et al.
OPSDL: On-Policy Self-Distillation for Long-Context Language Models
Xinsen Zhang, Zhe Ding, Tian Pan et al.
Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning
Xinbin Yuan, Jian Zhang, Kaixin Li et al.
InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners
Yuhang Liu, Pengxiang Li, Congkai Xie et al.
Scaling Computer-Use Grounding via User Interface Decomposition and Synthesis
Tianbao Xie, Jiaqi Deng, Xiaochuan Li et al.
ShowUI: One Vision-Language-Action Model for GUI Visual Agent
Kevin Qinghong Lin, Linjie Li, Difei Gao et al.
GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents
Qianhui Wu, Kanzhi Cheng, Rui Yang et al.
Entropy-Aware On-Policy Distillation of Language Models
Woogyeol Jin, Taywon Min, Yongjin Yang et al.