EVOMAL: Self-Poisoning in Self-Evolving Coding Agents
EVOMAL wraps malicious skills in banners, causing self-poisoning with ASP R up to 86.7%.
Key Findings
Methodology
The paper introduces EVOMAL, an attack framework that plants structurally crafted banners containing malicious payloads into skill libraries. These banners are designed to be faithfully reproduced during imitation by self-evolving agents, leading to self-replication of harmful code. Experiments across six models, including DeepSeek-V4-Pro and Qwen3, on 153 tool-relevant tasks, measure the agent self-poisoning rate (ASPR). Results show ASP R ranges from 20.3% to 41.8%, with poisoned libraries containing 4.9 to 9.0 times more malicious skills than initially planted. The attack exploits the agent’s imitation mechanism, enabling persistent self-propagation even after removing the original skills. A countermeasure, Counter-Prompt, is proposed, which incorporates instructions in system prompts to suppress banner copying, reducing ASP R to below 6.7%.
Key Results
- Across six models, EVOMAL increased ASP R from 11.1% (no banner) to as high as 41.8%, with malicious skills in the library being 4.9 to 9.0 times the initial planted amount.
- Targeted description fine-tuning raised ASP R up to 86.7%, demonstrating high diffusion potential.
- Even after removing planted skills, Qwen3 maintained a 68% ASP R, indicating high persistence and self-sustaining infection.
Significance
This work uncovers a critical security vulnerability in self-evolving coding agents—self-poisoning via imitation of planted malicious skills. Unlike traditional attacks relying on invoking malicious tools, EVOMAL exploits the agent’s imitation mechanism, making detection and removal challenging. The findings highlight the need for new defense strategies that address the CREATE-path, where the agent authors new skills. The proposed Counter-Prompt offers an effective, low-cost solution, significantly reducing infection rates without impairing task performance. These insights are vital for designing safer autonomous AI systems, especially as skill libraries grow in size and complexity, emphasizing the importance of structural content awareness in security protocols.
Technical Contribution
The paper pioneers the formalization of self-poisoning in self-evolving agents, introducing the ASP R metric to quantify infection levels. It demonstrates how structural banners can induce imitation-based copying of malicious payloads, creating a self-propagating worm. The research develops a comprehensive attack framework, EVOMAL, validated across multiple models and tasks, showing high efficacy and persistence. Additionally, it proposes Counter-Prompt, a prompt-based defense mechanism that leverages system instructions to suppress banner copying, achieving a significant reduction in ASP R. This work advances understanding of autonomous agent security, bridging gaps between imitation mechanisms and malicious content propagation, and offers practical mitigation strategies.
Novelty
This is the first study to identify and exploit self-poisoning via imitation in self-evolving coding agents. Unlike prior work focusing on direct invocation of attacker-submitted skills, EVOMAL leverages structural imitation to induce self-replication of malicious payloads, forming a persistent self-propagating worm. The introduction of the ASP R metric and the Counter-Prompt defense further distinguish this work, providing new theoretical and practical insights into autonomous system security. The approach demonstrates that structural content, such as banners, can be exploited to bypass traditional defenses, marking a significant shift in understanding AI security vulnerabilities.
Limitations
- Current defenses relying on system prompts may be less effective against more sophisticated or adaptive banner structures. The attack’s success depends on the structural similarity of banners, which could be mitigated with more complex designs.
- Experiments are primarily conducted on specific models and datasets; scalability and robustness in larger, more diverse environments remain to be validated.
- The approach assumes the agent’s imitation mechanism faithfully reproduces banners; in real-world scenarios, slight variations could reduce effectiveness. Future work should explore adaptive detection and more resilient structural designs.
Future Work
Future research will focus on developing more robust structural defenses, including dynamic content analysis and behavioral anomaly detection. Exploring multi-modal defenses integrating visual and contextual cues can enhance detection. Extending the framework to larger, real-world deployment scenarios will validate scalability. Additionally, integrating formal verification and differential privacy techniques can further mitigate self-poisoning risks, ensuring safer autonomous AI systems in complex environments.
AI Executive Summary
The rapid advancement of large language models (LLMs) has led to the development of self-evolving coding agents capable of autonomously creating and storing tools within shared skill libraries. While these agents significantly improve automation and efficiency, they also introduce new security vulnerabilities. This study uncovers a critical flaw—self-poisoning—where malicious skills, embedded within structured banners, can be silently propagated through imitation mechanisms inherent in self-evolving agents. Unlike traditional attack vectors that rely on invoking malicious tools directly, EVOMAL exploits the agent’s tendency to reproduce retrieved skills, including their structural patterns, leading to persistent self-replication of harmful code.
The researchers conducted extensive experiments across six models, including DeepSeek-V4-Pro and Qwen3, on 153 tool-relevant tasks. They introduced the agent self-poisoning rate (ASPR) metric to quantify the extent of infection. Results revealed that, with Banner wrapping, ASP R could reach as high as 41.8%, with poisoned libraries containing nearly ten times more malicious skills than initially planted. Notably, targeted description fine-tuning increased ASP R to 86.7%, illustrating the attack’s scalability and adaptability.
A key insight was that even after removing the initially planted malicious skills, the infection persisted, with some models maintaining ASP R levels above 68%. This persistence underscores the self-sustaining nature of the worm created by the imitation mechanism. To counter this threat, the authors proposed Counter-Prompt, a prompt-based defense strategy that instructs agents to avoid copying structural banners. This simple yet effective measure reduced ASP R to below 6.7% across tested models without impairing task performance.
This work fundamentally shifts the understanding of security in autonomous, self-evolving systems. It demonstrates that structural content like banners can be exploited to induce self-replication of malicious code, creating a new class of persistent threats. The findings emphasize the importance of designing structural-aware defenses and highlight the need for ongoing research into resilient, secure AI architectures. Overall, the paper provides both a warning and a practical solution, advancing the field of AI safety and robustness in self-adaptive systems.
Deep Dive
Abstract
Self-evolving LLM coding agents write their own tools by imitating retrieved skills from shared skill libraries. We identify a vulnerability in this loop: during authoring, a retrieved malicious skill can become the template for a new skill that preserves the payload. We call this self-poisoning: the agent authors, stores, and runs the resulting malicious skill. We exploit it through EvoMal, an attack that amplifies self-poisoning by wrapping an interchangeable payload in a banner, a set of benign-looking structural elements that induces an imitating agent to reproduce the enclosed code. The attacker plants malicious skills in the library without invoking them. The agent then authors and executes new skills carrying the harmful code. Each authored copy can re-enter the library and be imitated again, forming a self-propagating worm that persists after the planted skills are removed. We define the agent self-poisoning rate (ASPR) as the fraction of tasks that add a newly authored malicious skill to the library. Across six models on 153 tool-relevant SWE-bench Verified tasks, ASPR ranges from 20.3% to 41.8%, and the poisoned libraries hold 4.9 to 9.0 times as many malicious skills as were planted. The vulnerability also appears without a banner: DeepSeek-V4-Pro reaches 11.1% ASPR with the payload alone. Tailoring the planted skill descriptions to one task family raises ASPR to 86.7%. After the planted skills are removed, Qwen3 retains a round-5 ASPR of 68% because agent-authored copies remain. These copies evade existing defenses, which focus on attacker-submitted names, code, and signatures. We propose counter-prompt, a defense that discourages banner-style copying and reduces EvoMal's ASPR to at most 6.7% with no significant task-completion loss.