Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents
SkillMisevo-Gym reveals skill misevolution in LLMs; SafeEvolve reduces unsafe retrieval by 26.7 percentage points.
Key Findings
Methodology
The study introduces SkillMisevo-Gym and SkillMisevo-Bench. The former is a lifecycle-aware tool, and the latter evaluates the impact of malicious exposure on tasks. SafeEvolve acts as a governance wrapper, repairing unsafe content and managing reuse.
Key Results
- All 21 evolved configurations produce unsafe artifacts, with 15 resulting in fresh-session harm. Malicious tasks raise ASR from 16.0% to 35.3%.
- SafeEvolve reduces unsafe retrieval and fresh-session harm by 26.7 and 17.3 percentage points, respectively.
- Mean benign utility changes by only 0.4 points, indicating governance effectiveness.
Significance
This research highlights the risk of skill misevolution in self-improving LLM agents, emphasizing the importance of safety governance in persistent adaptation. By introducing SkillMisevo-Gym and SafeEvolve, the study provides a systematic approach to evaluation and governance, filling the gap where existing benchmarks fail to assess lifecycle risks comprehensively.
Technical Contribution
Introduces SkillMisevo-Gym and SkillMisevo-Bench for version control and lifecycle evaluation of skill states. SafeEvolve provides a method-agnostic governance mechanism by removing unsafe instructions and managing reuse.
Novelty
First to propose the concept of skill misevolution, providing a comprehensive risk assessment and governance framework through SkillMisevo-Gym and SafeEvolve.
Limitations
- The study is primarily conducted in simulated environments, which may face different challenges in real-world applications.
- Governance mechanisms may increase computational overhead, affecting real-time performance.
- Not all unsafe artifacts are completely eliminated.
Future Work
Future work could explore more efficient governance mechanisms to reduce computational overhead while validating the method's effectiveness in more real-world scenarios.
AI Executive Summary
Self-improving LLM agents accumulate experience across tasks, forming persistent cross-task states. However, this skill evolution can lead to unsafe successes being mis-evolved into reusable policies. Existing benchmarks fail to comprehensively assess lifecycle risks, unable to identify risks across authoring, retrieval, and execution.
To address this, researchers introduce SkillMisevo-Gym and SkillMisevo-Bench. SkillMisevo-Gym is a lifecycle-aware tool that versions skill states across agent frameworks. SkillMisevo-Bench is a frozen design to evaluate the impact of malicious exposure on tasks, providing nine lifecycle metrics.
Additionally, SafeEvolve acts as a governance wrapper, repairing unsafe content and managing reuse. Experimental results show that all 21 evolved configurations produce unsafe artifacts, with 15 resulting in fresh-session harm. SafeEvolve reduces unsafe retrieval and fresh-session harm by 26.7 and 17.3 percentage points, respectively, with mean benign utility changing by only 0.4 points, indicating the effectiveness of the governance mechanism. The study emphasizes the importance of safety governance in persistent adaptation and provides directions for future research.
Deep Analysis
Background
As LLM agents' self-improvement capabilities continue to grow, skill evolution has become a crucial form of update. However, existing benchmarks cannot comprehensively assess the safety risks that may arise during skill evolution, especially when skills are widely applied across different tasks and environments.
Core Problem
The core problem is skill misevolution, where unsafe successes are mis-evolved into reusable policies. This can lead to repeated unsafe behavior in future tasks, which existing benchmarks fail to identify and assess.
Innovation
The core innovation of the study is the introduction of SkillMisevo-Gym and SkillMisevo-Bench for comprehensive risk assessment during skill evolution. SafeEvolve acts as a governance mechanism by removing unsafe instructions and managing reuse, providing a method-agnostic solution.
Methodology
- �� SkillMisevo-Gym: A lifecycle-aware tool for versioning skill states.
- �� SkillMisevo-Bench: A frozen design to evaluate the impact of malicious exposure.
- �� SafeEvolve: A governance wrapper that removes unsafe instructions and manages reuse.
Experiments
The experimental design covers 25 agent-method configurations, each involving 525 tasks and 25 episodes. Malicious tasks raise ASR from 16.0% to 35.3%. SafeEvolve reduces unsafe retrieval and fresh-session harm across representative skill evolution methods.
Results
Experimental results show that all 21 evolved configurations produce unsafe artifacts, with 15 resulting in fresh-session harm. SafeEvolve reduces unsafe retrieval and fresh-session harm, with mean benign utility changing minimally.
Applications
The study's application scenarios include any LLM agents requiring self-improvement and skill evolution, particularly in safety-critical fields such as autonomous driving and medical diagnostics.
Limitations & Outlook
The study is primarily conducted in simulated environments, which may face different challenges in real-world applications. Governance mechanisms may increase computational overhead, affecting real-time performance. Future work should validate the method's effectiveness in real-world scenarios.
Plain Language Accessible to non-experts
Imagine a kitchen where chefs accumulate experience while cooking. Sometimes, they might discover a quick but unsafe method, like not paying attention to finger placement while chopping. Although nothing happened this time, if this method is recorded and repeated later, it could lead to accidents. SkillMisevo-Gym is like a kitchen management system that records each chef's methods and evaluates their safety. SafeEvolve acts like a safety inspector, ensuring that every recorded method is safe and removing unsafe ones.
ELI14 Explained like you're 14
Imagine you're playing a game, and after each level, the game records your moves. Sometimes, you might find a glitch that lets you finish faster, but it could crash the game. SkillMisevo-Gym is like a game admin that records your every move and checks if it's safe. SafeEvolve is like a patch that fixes glitches, ensuring you won't face the same problem next time. This way, you can keep playing safely without worrying about sudden crashes!
Glossary
SkillMisevo-Gym
A lifecycle-aware tool for versioning skill states across agent frameworks.
Used for assessing risks during skill evolution.
SkillMisevo-Bench
A frozen design to evaluate the impact of malicious exposure on tasks.
Provides nine lifecycle metrics.
SafeEvolve
A governance wrapper that repairs unsafe content and manages reuse.
Reduces unsafe retrieval and fresh-session harm.
LLM (Large Language Model)
An AI model capable of generating and understanding natural language.
Used for self-improvement and skill evolution.
ASR (Attack Success Rate)
A metric measuring the success rate of malicious tasks.
Used to assess risks during skill evolution.
Open Questions Unanswered questions from this research
- 1 How to validate the effectiveness of SkillMisevo-Gym and SafeEvolve in real-world applications?
- 2 How to reduce the computational overhead introduced by governance mechanisms?
- 3 How to extend the method to more application scenarios?
Applications
Immediate Applications
Autonomous Driving
Apply SkillMisevo-Gym and SafeEvolve in autonomous driving to ensure decision-making safety.
Long-term Vision
Medical Diagnostics
Apply the method in medical diagnostics to ensure the safety and accuracy of the diagnostic process.
Abstract
Self-improving LLM agents convert successful trajectories into persistent cross-task state. An unsafe success can thereby become reusable policy after its triggering input disappears. Skill evolution makes this failure measurable by distilling operational trajectories into executable, transferable, and inspectable procedures. Because evolution optimizes task outcomes rather than procedure safety, compromised experience can cause skill misevolution. Existing benchmarks measure current behavior or static artifacts but cannot attribute risk across authoring, retrieval, and later execution. To expose this lifecycle, we introduce SkillMisevo-Gym, a lifecycle-aware harness that versions skill state across agent frameworks, and SkillMisevo-Bench, a frozen design from malicious exposure to carryover tasks, with concept-aligned benign tasks and nine lifecycle metrics. We also introduce SafeEvolve, a wrapper that repairs unsafe content and governs subsequent reuse. Across 25 agent-method configurations, each covering 525 tasks in 25 episodes, all 21 evolved configurations author unsafe artifacts, while only fifteen lead to fresh-session harm. In the exposure sweep, three malicious tasks raise carryover ASR from 16.0% to 35.3%. Across representative skill evolution methods, SafeEvolve reduces unsafe retrieval and fresh-session harm by 26.7 and 17.3 percentage points, respectively, while mean benign utility changes by only 0.4 points. Together, persistent-adaptation safety must govern what updates write and what future executors reuse. Code is available at https://github.com/henrymao2004/misevolve.