AgentChaos: Chaos Engineering for Agent Systems via Programmatic Fault Injection
Proposes AgentChaos, a runtime, non-intrusive HTTP-layer fault injection framework to evaluate agent system robustness against 65 LLM API response faults.
Key Findings
Methodology
This study develops a fault taxonomy for LLM API responses based on classical distributed system faults—crash, omission, and value faults—covering six specific types. Using an HTTP-layer wrapper, faults are injected dynamically at runtime without source code modification, across 65 configurations. A trigger verification process ensures only triggered faults are considered, avoiding false assessments. Evaluations across multiple agent systems, benchmarks, and backbone LLMs reveal performance degradation up to 50% in pass@1. Fault diagnosis methods achieve below 53% accuracy, indicating challenges in fault identification. The findings highlight that system implementation impacts robustness more than model capability, guiding future design improvements.
Key Results
- Across 65 fault configurations, all agent systems showed significant performance drops, with pass@1 decreasing by up to 50 percentage points, confirming the effectiveness of fault injection.
- Robustness rankings were consistent across different models, indicating implementation quality outweighs model strength.
- Fault diagnosis accuracy remained below 56%, suggesting room for improved fault detection methods.
Significance
This work introduces a novel, real-time, non-intrusive fault injection framework for LLM API responses, filling a critical gap in robustness evaluation tools. It demonstrates that system-level implementation choices heavily influence resilience, providing actionable insights for designing more reliable AI systems. The methodology enables systematic testing of diverse fault scenarios, helping developers identify vulnerabilities before deployment, ultimately advancing AI safety and dependability in real-world applications.
Technical Contribution
AgentChaos leverages HTTP request interception to inject detailed response faults—such as truncation, encoding errors, and schema mismatches—without source code changes. The framework supports multiple fault configurations, including transient, persistent, intermittent, and burst scenarios, validated through trigger verification. This approach surpasses traditional network-layer fault simulation by enabling content-level fault injection, offering a fine-grained, scalable, and realistic robustness assessment tool for AI agent systems.
Novelty
This is the first systematic fault taxonomy tailored for LLM API responses, encompassing content and tool call fields. The integration of an HTTP-layer, runtime, content-aware fault injection mechanism with trigger verification represents a significant innovation over existing offline or network-only fault simulation methods. It enables detailed, real-time testing of response content faults, such as truncation and encoding errors, providing richer fault scenarios and insights into system robustness.
Limitations
- The current framework mainly targets response content faults at the HTTP layer, with limited support for internal model semantic errors like hallucinations.
- Fault configurations rely on predefined scenarios, which may not cover all real-world failure modes, necessitating adaptive strategies.
- Performance impact under high concurrency remains to be thoroughly evaluated, especially in large-scale deployment environments.
Future Work
Future directions include integrating adaptive fault strategies based on machine learning, expanding multi-protocol support, and developing automated diagnosis and self-healing mechanisms. Enhancing scalability and reducing overhead will be key to deploying in production environments. Additionally, extending fault models to cover deeper semantic failures within models will further improve robustness assessments.
AI Executive Summary
The rapid adoption of large language models (LLMs) in intelligent agent systems has heightened the importance of evaluating their robustness. Traditional fault injection methods, often offline or limited to network-level errors, fall short in simulating the nuanced response content faults—such as truncation, encoding errors, or schema mismatches—that frequently occur in real deployments. To address this gap, this paper introduces AgentChaos, a novel chaos engineering framework that performs runtime, non-intrusive fault injection at the HTTP request-response layer. By leveraging a wrapper around the shared communication interface, AgentChaos can inject 65 diverse fault configurations, including crash, omission, and value faults, into LLM API responses without modifying source code.
The core innovation lies in the systematic fault taxonomy and content-aware injection mechanism. This approach allows precise simulation of realistic failures, such as truncated responses or corrupted encodings, which are challenging for traditional tools. The framework also employs a trigger verification process, ensuring only triggered faults influence evaluation outcomes. Extensive experiments across multiple agent systems, benchmarks, and backbone LLMs demonstrate performance degradation up to 50%, confirming the effectiveness of the method. The results reveal that robustness depends more on system implementation than model capability, providing critical insights for future design.
Despite its strengths, the framework has limitations, including focus on response content faults and predefined fault scenarios. Future work aims to incorporate adaptive fault strategies, expand multi-protocol support, and develop automated diagnosis and self-healing capabilities. Overall, AgentChaos offers a powerful tool for systematically assessing and improving the resilience of AI-driven agent systems, paving the way for safer, more reliable AI applications in complex environments.
Deep Dive
Plain Language Accessible to non-experts
想象你在一家工厂工作，工厂里有很多工人（代表系统中的不同部分）在合作完成一件大事。每个工人都依赖工具（像API）来完成任务。有时候，工具会出错，比如突然停工（崩溃）、只完成一部分（遗漏）或给出错误的指令（数值错误）。如果工厂想提前知道这些问题会不会影响整体生产，就需要模拟这些故障。本文提出一种方法，就像在工厂里偷偷调皮地让工具出错一样，但不用改动工厂的机器，只是在工具的“信号线”上做手脚。这样可以在工厂正常运转时，测试出哪些故障会导致生产停滞，从而提前修补漏洞，确保真正的生产不会出大问题。
ELI14 Explained like you're 14
Imagine you're playing a video game with friends, and sometimes the game suddenly freezes, graphics glitch, or only half of the message appears. Developers want to test how tough the game is against these glitches without changing the game code directly. So, they create a special tool that can secretly cause these errors during gameplay—like making the game freeze or cut off part of the message—without rewriting the game's core. This way, they can see which parts of the game are fragile and fix them before the game is released to everyone. It's like practicing for unexpected problems so the real game runs smoothly and players have a better experience. This research developed such a tool for AI systems, letting developers simulate errors in responses from AI models, helping make AI smarter and safer for everyone.
Abstract
Agent systems rely on LLM APIs for every response, but these APIs can return server errors, truncated responses, or corrupted content that propagates through downstream agents and causes task failure. Evaluating robustness under these faults is crucial for reliable deployment. Existing fault injection methods are offline, require source code modification, or cannot modify specific response fields. A comprehensive evaluation also requires a systematic fault taxonomy because different fault types affect downstream agents differently. We propose AgentChaos, a chaos engineering framework for controlled, runtime, non-intrusive LLM API fault injection. Since all agent systems access LLMs through the same HTTP interface, we inject faults at this shared layer without modifying source code. We define crash, omission, and value faults on content and tool call fields, intercept and modify LLM API responses at runtime, and verify whether each fault is triggered to filter untriggered tasks and avoid underestimating fault impact. Evaluations across agent systems, benchmarks, and backbone LLMs under 65 fault configurations show that all systems degrade under fault injection, with pass@1 dropping by up to 50 percentage points. The ranking is consistent across models, suggesting that robustness depends on system implementation rather than model capability. Existing fault diagnosis methods achieve below 53% accuracy on fault type and below 56% on fault step, leaving room for improvement. We further reveal practical findings for agent system developers.