Visual Verification Enables Inference-time Steering and Autonomous Policy Improvement

In the rapidly evolving field of robotics, a persistent challenge has been enabling robots to learn and adapt autonomously in complex, real-world environments. Traditional approaches rely heavily on large-scale human demonstrations and offline training, which are costly, time-consuming, and often limited in generalization. Despite advances in deep learning and large pre-trained models, deploying robots that can continuously improve during operation remains a significant hurdle. This gap between offline training and online adaptation has motivated researchers to explore mechanisms that allow robots to self-assess and refine their behaviors in real-time.

The paper introduces VERITAS, a novel framework that leverages inference-time verification to enable autonomous policy improvement. The core idea is to treat the robot’s policy as a stochastic generator that produces multiple candidate actions at each decision point. These actions are then evaluated by a plug-and-play visual verifier based on vision-language models (VLM), which assess their task relevance and physical plausibility through geometric projections. The highest-scoring actions are executed immediately, leading to instant performance gains. Crucially, the verified successful trajectories are stored and used for offline policy fine-tuning via behavior cloning, creating a self-reinforcing cycle that continuously enhances the robot’s capabilities.

This approach addresses a fundamental bottleneck in robotic learning—the high cost of data collection—by replacing it with an inference-time verification mechanism that requires no additional training of the core policy. The verification process is efficient, leveraging static visual traces generated by the VLM, which simplifies online geometric consistency checks. The entire system forms a closed-loop, where the robot learns from its own successful experiences, effectively forming a data flywheel that accelerates policy improvement.

Extensive experiments in both simulation and real-world robotic platforms demonstrate the effectiveness of VERITAS. In simulated manipulation tasks, the framework achieves an average success rate increase of 10%, with verified trajectories enabling the policy to outperform baseline methods. In real-world experiments, the approach attains comparable performance to expert demonstrations, with significant improvements in task success and robustness. The visual verifier’s geometric scoring outperforms traditional value-function-based methods, confirming its efficacy.

The significance of this work lies in its practical, scalable solution to autonomous robot learning. By integrating inference-time verification, robots can explore, validate, and improve behaviors during deployment, reducing reliance on costly data collection and human supervision. This paradigm shift opens new avenues for lifelong learning, adaptive control, and scalable deployment across diverse applications such as manufacturing, logistics, and service robotics. Despite its success, challenges remain, including the dependence on the accuracy of visual models and computational costs, which future research aims to address. Overall, VERITAS marks a substantial step toward truly autonomous, self-improving robotic systems, promising a future where robots can learn and adapt continuously in dynamic environments.