When State Becomes an Attack Surface: State-Semantic Injection in LLM-Driven Embodied Agents
Proposes ESTI to detect state-semantic injection in LLM-driven embodied agents, evaluating downstream propagation of false environment states.
Key Findings
Methodology
This paper introduces Environment State-Text Injection (ESTI), a framework designed to evaluate whether corrupted environment state information can be adopted by the planner and influence final behaviors in LLM-driven embodied agents. The approach defines a component-scoped, predicate-local threat model, employing state re-grounding and semantic construction techniques. It encodes adversarial goals as false state evidence embedded into objects, relations, task-stage rules, and feedback, without altering user instructions or model parameters. Experiments across platforms like ProgPrompt/VirtualHome, VoxPoser/RLBench, and AI2-THOR/iTHOR demonstrate that ESTI significantly increases attack success rates—up to 89.32% at planning level and 43.69% at execution level—highlighting the importance of state authenticity and consistency for system security.
Key Results
- In simulation environments, ESTI achieved a maximum of 89.32% success rate in planning-level attacks and 43.69% in execution-level attacks, outperforming baselines such as Vanilla IPI, EIRAD, and BADROBOT, confirming the effectiveness of state semantics injection.
- Ablation studies showed that carrier compatibility and representation consistency critically influence attack adoption, with runtime re-grounding only marginally affecting success rates (1.92 and 3.85 percentage points), emphasizing the importance of semantic authenticity.
- Results indicate that the feasibility of the false state depends on entity reachability and task constraints, ensuring the manipulated state is plausible within the environment, thus increasing attack realism.
Significance
This research uncovers a novel security vulnerability in LLM-driven embodied agents, shifting focus from traditional digital prompt injection to the integrity of environment state semantics. It demonstrates how false environment information, if adopted by the planner, can lead to unsafe or unintended robot behaviors, raising awareness about the critical need for environment state verification. The ESTI framework offers a systematic method to evaluate downstream propagation of corrupted states, informing future design of robust and trustworthy robotic systems. Its implications extend to autonomous navigation, manipulation, and human-robot interaction, where environment perception fidelity is paramount. The findings urge developers and researchers to incorporate state authenticity checks into system architectures, fostering safer deployment of embodied AI.
Technical Contribution
The core technical contribution is the development of ESTI, which encodes adversarial goals as schema-compatible false state evidence, integrated into existing environment representations without modifying user instructions or model parameters. The framework employs state re-grounding to ensure the manipulated state is feasible within the current scene, and constructs evidence by rewriting object attributes, relations, and feedback fields. The evaluation platform ESTI-Bench enables systematic measurement of the transfer from state-level deception to final environment outcomes, distinguishing between planning adoption and execution realization. This approach advances the understanding of downstream effects of environment state manipulation, providing a new perspective on embodied agent security.
Novelty
This work is the first to formalize and evaluate downstream state-semantic injection attacks in embodied agents, focusing on how false environment states, encoded as native records, can be adopted by the planner and influence physical outcomes. Unlike prior digital prompt injection methods, ESTI emphasizes the importance of environment representation fidelity and semantic consistency, ensuring the manipulated state remains plausible and schema-compatible. The threat model restricts adversarial rewriting to task-relevant records, making the attack both realistic and targeted. This novel approach bridges the gap between environment perception and physical action, offering a new dimension for security analysis in embodied AI systems.
Limitations
- The effectiveness of ESTI relies on the environment state being groundable; if the target state is physically unachievable or unsupported by the scene, the attack may fail.
- The current threat model assumes single-component compromise; multi-component or dynamic component attacks are not yet explored, limiting scope.
- Most experiments are conducted in simulated environments; real-world physical robots may present additional challenges such as sensor noise and environmental variability, which are not fully addressed.
Future Work
Future research will explore multi-component and adaptive attack strategies, aiming to understand compound vulnerabilities. Additionally, efforts will focus on deploying ESTI in real robotic systems to validate transferability and robustness. Developing defense mechanisms, such as environment state verification and semantic consistency checks, will be prioritized to enhance system resilience. Integrating learning-based detection methods and extending the framework to dynamic, multi-modal environments are also promising directions, aiming to build safer and more trustworthy embodied AI systems.
AI Executive Summary
The rapid advancement of large language models (LLMs) has revolutionized autonomous robotic systems, enabling high-level planning, reasoning, and decision-making capabilities. These embodied agents leverage environment state information—such as object attributes, spatial relations, and task feedback—to ground their understanding and generate appropriate actions. However, this reliance on environment semantics introduces new security vulnerabilities that have been largely overlooked.
Traditional research on AI security primarily focused on digital prompt injection and tool hijacking, which manipulate textual inputs or external APIs. In contrast, embodied agents operate within physical or simulated environments, where the integrity of environment state information is critical. If adversaries can manipulate or inject false environment semantics, they can mislead the planner into generating unsafe or unintended behaviors, potentially causing physical harm or system failure.
This paper introduces Environment State-Text Injection (ESTI), a novel framework designed to evaluate whether false environment state information, once introduced into the agent’s perception, can be adopted by the planner and propagate through the execution pipeline. ESTI assumes a scenario where a single component responsible for producing environment state records is compromised. The attacker encodes a preselected adversarial goal as false evidence embedded within objects, relations, task rules, or feedback, without altering user instructions or model parameters.
The core innovation lies in the semantic construction process, which ensures that the manipulated state remains schema-compatible and plausible within the current environment. This involves a state re-grounding step to verify entity support and a targeted rewriting of object attributes, spatial relations, and feedback fields. The framework then evaluates whether this false evidence influences the planner’s decision-making and whether the resulting plan leads to the adversarial goal during execution.
Extensive experiments across multiple platforms—such as VirtualHome, RLBench, and AI2-THOR—demonstrate that ESTI significantly increases attack success rates, with the highest reaching 89.32% at the planning level and 43.69% at the execution level. Ablation studies reveal that the success heavily depends on the semantic compatibility and consistency of the manipulated state, emphasizing the importance of environment authenticity.
These findings have profound implications for the security of embodied AI systems. They highlight that environment perception, often assumed trustworthy, can be exploited to induce dangerous behaviors. The ESTI framework provides a systematic method for evaluating and mitigating such risks, urging the community to incorporate environment state verification into system design. Moving forward, research will focus on extending this approach to multi-component attacks, real-world deployment, and developing robust defenses to ensure the safe and trustworthy deployment of embodied AI in complex environments.
Deep Dive
Abstract
Large Language Models (LLMs) have demonstrated capabilities in in-context learning, task decomposition, step-by-step reasoning, and code generation, driving their gradual evolution from text generation models into the core of agents capable of perceiving environments, invoking tools, and executing tasks. Traditional LLM Agents typically obtain information through webpages, documents, databases, or external tools and generate corresponding invocation sequences according to user goals; when this technology is further integrated with robotic systems, large language models begin to undertake functions such as task understanding, high-level planning, and behavioral decision-making. SayCan combines the task reasoning capability of language models with the affordances of robotic skills, while Code as Policies and ProgPrompt generate robot task plans through policy code and programmatic prompting, respectively, and VoxPoser uses language models and vision-language models to construct three-dimensional value maps to guide robotic manipulation \cite{6,7,8,9}. Vision-language-action models such as PaLM-E, RT-2, and GR00T N1 further strengthen the connection among language, visual perception, and robotic actions \cite{10,11,12}. In such LLM-driven embodied agents, the model not only needs to understand user instructions, but also needs to combine scene states, object attributes, spatial relations, and execution feedback to complete task grounding, and then hand the generated action plan to skill libraries, motion planners, or controllers for execution.
References (20)
Adversarial examples in the physical world
Alexey Kurakin, I. Goodfellow, Samy Bengio
ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks
Mohit Shridhar, Jesse Thomason, Daniel Gordon et al.
Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang et al.
Robust Physical-World Attacks on Deep Learning Visual Classification
Kevin Eykholt, I. Evtimov, Earlence Fernandes et al.
Large Language Models are Zero-Shot Reasoners
Takeshi Kojima, S. Gu, Machel Reid et al.
RIPA: Sensory-Vector Prompt Injection Attacks on LLM-Controlled ROS 2 Robots
N.-Zh. B. Dorzhiev
Visual Language Maps for Robot Navigation
Chen Huang, Oier Mees, Andy Zeng et al.
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments
S. Srivastava, Chengshu Li, Michael Lingelbach et al.
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models
Ishika Singh, Valts Blukis, A. Mousavian et al.
Explaining and Harnessing Adversarial Examples
I. Goodfellow, Jonathon Shlens, Christian Szegedy
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
Qiusi Zhan, Zhixiang Liang, Zifan Ying et al.
Habitat: A Platform for Embodied AI Research
M. Savva, Abhishek Kadian, Oleksandr Maksymets et al.
Evaluating Large Language Models Trained on Code
Mark Chen, Jerry Tworek, Heewoo Jun et al.
Computing Reachable Sets of Hybrid Systems Using a Combination of Zonotopes and Polytopes
M. Althoff, O. Stursberg, M. Buss
ReAct: Synergizing Reasoning and Acting in Language Models
Shunyu Yao, Jeffrey Zhao, Dian Yu et al.
Ignore Previous Prompt: Attack Techniques For Language Models
Fábio Perez, I. Ribeiro
Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Wenlong Huang, P. Abbeel, Deepak Pathak et al.
Jailbroken: How Does LLM Safety Training Fail?
Alexander Wei, Nika Haghtalab, J. Steinhardt
Code as Policies: Language Model Programs for Embodied Control
Jacky Liang, Wenlong Huang, F. Xia et al.
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder et al.