EMERGE-Policy: A Robot Mind Emerges Beyond a Single Policy

The evolution of robot intelligence has transitioned from single-model approaches like end-to-end deep learning to multi-capability systems integrating perception, planning, and control. Early works such as VLMs and World Models laid foundational capabilities, but lacked explicit coordination mechanisms. Recent efforts like RoboClaw, RoboHarness, and VLA demonstrated capability orchestration, yet often relied on monolithic policies or fine-tuning for each task. These limitations hinder scalability and robustness in complex environments. The current trend emphasizes modular, multi-agent architectures that can dynamically coordinate heterogeneous skills, inspired by human cognition. EMERGE-Policy builds upon these developments, proposing a structured, explicit role-based framework that unifies diverse capabilities into a coherent system, aiming to address the core challenges of information overload, task decomposition, and local correction.

Existing robot control systems struggle with integrating multiple capabilities efficiently, especially in complex, unpredictable environments. The main issues include excessive information processing, lack of explicit role boundaries, and limited local correction mechanisms. These bottlenecks reduce robustness, scalability, and generalization. The challenge is to design a system that can coordinate diverse modules—perception, reasoning, verification, memory—while controlling information flow and enabling effective recovery from errors. Achieving this requires a structured approach that explicitly defines roles, interfaces, and decision-making processes, ensuring that each component contributes effectively without interference, and that the overall system can adapt to new tasks and environments.

The core innovation is the introduction of a graph-structured multi-role agent system, where the Main Agent manages high-level task planning, and role-specific Sub Agents handle perception, verification, and memory in isolated contexts. This explicit role separation reduces interference and enhances robustness. The framework employs structured evidence and capability interfaces, supporting heterogeneous models like VLA, WM, and Verifiers, which are invoked based on task needs. Criterion-grounded verification ensures task success, while Branch Stack enables localized recovery. Token-aware external memory maintains task state across interactions. Unlike prior monolithic models, this design emphasizes modularity, explicit information boundaries, and localized correction, enabling scalable, robust, and generalizable robot policies.

• �� Construct a graph-structured multi-agent system with Main Agent and role-specific Sub Agents.
• �� Main Agent maintains task state, decomposes instructions into subgoals, and coordinates Sub Agents.
• �� Sub Agents operate asynchronously, processing evidence for perception, verification, and memory, returning structured results.
• �� Capabilities are categorized into Operational, Imagination, and Evaluation Skills, invoked via explicit interfaces.
• �� Use Criterion-grounded verification to assess subgoal completion.
• �� Implement Branch Stack for local recovery, storing transient subgoals.
• �� External token-aware memory consolidates task-relevant information, supporting continuity.
• �� Experiments validate the system on LIBERO, Robodojo, with success rate improvements and robustness analysis.

The evaluation employed LIBERO, LIBERO-plus, and Robodojo benchmarks, comparing success rates, task completion times, and memory performance against baselines. Different backend combinations (VLA, WAM) tested adaptability. Ablation studies analyzed the impact of role separation, verification, and memory strategies. Real robot deployments involved long-horizon tasks, assessing robustness and recovery efficiency. Metrics included success rate improvements (up to 2-3%), recovery iterations, and task efficiency. Results demonstrated that the structured multi-agent approach outperformed monolithic policies, especially in complex or perturbed scenarios, confirming its scalability and robustness.

The system achieved success rate increases of 2.0% on LIBERO and 0.7% on LIBERO-plus, with Memory accuracy significantly improved, success rate rising by 3.59%. In Robodojo, success rates in long-horizon tasks exceeded baselines, validating the effectiveness of explicit role separation and structured evidence. Combining different backends (VLA, WAM) showed robustness across modalities. The system generalized well across multiple tasks without fine-tuning, indicating strong scalability. Ablation results confirmed the importance of each component—verification, memory, role separation—in performance gains.

This framework suits complex industrial automation, autonomous household robots, and service applications requiring robust multi-capability coordination. It supports dynamic task decomposition, local correction, and multi-modal perception, making it ideal for environments with high variability. Future integration with reinforcement learning could enable autonomous adaptation, further broadening its industrial and consumer applications.

Dependence on pre-trained perception and world models limits performance in highly unstructured or extreme environments. Asynchronous sub-agent execution may introduce delays, affecting real-time responsiveness. Handling highly unpredictable scenarios remains challenging, requiring further enhancements in recovery and planning algorithms. Computational costs of multi-model integration could hinder deployment in resource-constrained settings. Future work should focus on adaptive learning, online fine-tuning, and reducing latency to address these issues.