Zeva: In-Context Causal Learning for Generalizable Embodied Manipulation
Zeva enables in-context causal learning, allowing robots to self-evolve without parameter updates, improving success rates over repeated attempts.
Key Findings
Methodology
Zeva integrates a Causal Transition Encoder (CTE) to encode action-effect pairs into causal signals, stored in dual-timescale memory modules (BIT and PIM). During deployment, the frozen policy model retrieves relevant causal signals as context, guiding behavior adjustments without gradient updates. The framework comprises causal interaction extraction, memory management, and causal prompt injection. Experiments in simulation and real-world tasks demonstrate Zeva's superior performance, with success rates increasing over attempts, showcasing effective self-evolution and cross-task generalization.
Key Results
- In RoboCasa365-Atomic5, Zeva achieved a success rate of 76.8%, outperforming baselines like Fast-WAM (72.4%). In real ChemLab-Evo tasks, success rates improved by 5-6 percentage points across difficulty levels. The success rate scaled from 26% at the first attempt to 73% after four attempts in simulation, indicating effective experience accumulation.
- Adding human teleoperation demonstrations further boosted performance by up to 15%, confirming multi-source knowledge integration. Ablation studies showed removing either short-term or long-term memory components significantly degraded success, emphasizing their complementary roles.
- Across multiple tasks, Zeva maintained robust performance, with continuous improvement over repeated attempts, validating the causal memory's role in enabling autonomous self-improvement.
Significance
This work addresses the critical challenge of enabling robots to adapt in real-time without retraining, overcoming the limitations of static pretraining models. By explicitly modeling causality, Zeva enhances generalization and robustness in diverse, unseen environments. Its ability to continuously improve through experience paves the way for autonomous systems capable of long-term learning, crucial for industrial automation, domestic robots, and laboratory automation. The framework's modular design and reliance on causal memory open new avenues for scalable, adaptive robotic intelligence, significantly advancing the field of embodied AI.
Technical Contribution
Zeva introduces a novel causal learning paradigm that operates entirely online during deployment, without gradient updates. The core innovations include the Causal Transition Encoder for explicit causality encoding, dual-timescale causal memory for experience organization, and causal prompt injection for context-based policy adaptation. This design allows a frozen policy to dynamically adjust actions based on accumulated causal knowledge, providing theoretical guarantees of causal inference and practical benefits of continual self-improvement. Compared to prior methods relying on offline training or gradient-based adaptation, Zeva offers a scalable, efficient, and theoretically grounded solution for real-time autonomous learning.
Novelty
This research is the first to implement in-deployment, gradient-free causal learning in embodied agents, enabling continuous self-evolution. Unlike existing models that depend on pretraining or offline fine-tuning, Zeva's explicit causal encoding and memory-based context injection allow the policy to adapt dynamically to unseen physical conditions. Its ability to improve success rates over repeated attempts without parameter updates marks a significant departure from traditional paradigms, setting a new standard for autonomous, lifelong learning in robotics.
Limitations
- The effectiveness of causal extraction heavily depends on the quality of interaction signals; noisy or ambiguous data can impair performance. In highly complex or unpredictable environments, the causal signals may be insufficient for accurate inference.
- Memory scalability remains a challenge, especially in long-term, multi-task scenarios where experience accumulation could lead to computational bottlenecks.
- The current framework assumes reasonably structured environments; in highly unstructured or novel scenarios, the causal models may need further enhancement to maintain robustness.
Future Work
Future research will focus on integrating multi-modal sensory data, such as tactile and auditory signals, to enrich causal inference. Combining Zeva with reinforcement learning could further enhance autonomous exploration. Extending the framework to multi-agent systems will enable collaborative tasks, sharing causal knowledge across robots. Additionally, improving memory efficiency and scalability, along with robustness in highly noisy or unstructured environments, will be key to deploying Zeva in real-world, long-term applications.
AI Executive Summary
Robotic manipulation in real-world environments faces persistent challenges due to environmental variability and unseen physical conditions. Traditional approaches rely heavily on pretraining large models, which remain static after deployment, limiting adaptability. Zeva introduces a groundbreaking framework—In-Context Causal Learning (ICCL)—that empowers robots to learn from their own interactions during deployment without any parameter updates. The core innovation involves encoding action-induced environment changes into explicit causal signals via a Causal Transition Encoder (CTE). These signals are stored in a dual-timescale causal memory, comprising a short-term buffer (BIT) for recent interactions and a long-term repository (PIM) for experience accumulation across attempts. During operation, the robot retrieves relevant causal signals based on current task phases and injects them as context into a frozen policy model, enabling dynamic behavior adjustment.
This approach allows the robot to continuously improve its success rate as it gathers more experience, demonstrating a form of self-evolution. Extensive experiments in simulation (RoboCasa365-Atomic5) and real-world chemical manipulation (ChemLab-Evo) validate Zeva’s effectiveness. In simulation, success rates increased from 26% initially to over 73% after four attempts, outperforming existing models. In real-world tasks, Zeva achieved success rates exceeding 70% across multiple levels of task complexity, surpassing baselines by significant margins. Notably, incorporating human demonstrations further enhanced performance, illustrating the framework’s flexibility.
Zeva’s significance lies in its ability to enable autonomous, continual learning without costly retraining or fine-tuning, addressing a major bottleneck in embodied AI. Its explicit causal reasoning mechanism enhances generalization and robustness, making it suitable for diverse applications such as industrial automation, domestic robots, and laboratory automation. While promising, challenges remain in scaling memory and handling highly unstructured environments. Future directions include multi-modal causal inference, multi-agent collaboration, and integrating reinforcement learning for exploration. Overall, Zeva marks a pivotal step toward truly autonomous, self-improving robotic systems, opening new horizons for embodied intelligence.
Deep Dive
Abstract
Generalizable embodied manipulation remains difficult to achieve through pretraining alone, due to unseen physical conditions in the real world. We argue that robots need to learn from their own physical interactions on the fly during real-world deployment and use this knowledge to inform subsequent actions. We present Zeva, the first framework that enables in-context learning from a robot's own physical interaction experience while keeping the policy model frozen. Zeva employs a Causal Interaction Extractor to encode an executed action and its induced state change into a causal interaction signal, which is stored in a dual-timescale causal memory. For subsequent actions, relevant causal interaction signals are retrieved from memory and injected into the frozen policy model as context. Experiments in simulation and real-world manipulation demonstrate that Zeva achieves the best performance among the compared frontier VLAs and WAMs and, more importantly, enables self-evolution during deployment without gradient updates. Its success rate continues to improve as the robot accumulates interaction experience. Furthermore, the acquired interaction experience can generalize across tasks.