EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents
EvoSafeHarness optimizes model- and domain-specific safety mechanisms, reducing attack success rate from 45.6% to 10.0%.
Key Findings
Methodology
EvoSafeHarness is an optimization framework focused on synthesizing deployable safety mechanisms for frozen models in target domains. The framework combines natural-language policy and executable code logic, guided by model behavior, domain specifications, and fresh-context adversarial review to reject benchmark-specific rules.
Key Results
- On DecodingTrust-Agent, EvoSafeHarness reduces the average attack success rate from 45.6% to 10.0% with only a 3.3-point utility cost, achieving the best score in 14 of 15 model×domain deployments.
- On AgentDojo, it reaches 82.8% utility at 0.0% ASR, twice the utility of CaMeL at the same zero-ASR operating point, and transfers unchanged to unseen AgentDyn suites.
- It achieves the best score on Agent-SafetyBench for every victim and keeps mean ASR below 20% under adaptive PAIR attacks with a refinement budget of 16.
Significance
EvoSafeHarness holds significant implications for both academia and industry. It addresses the adaptability issues of existing fixed expert-designed defenses across different models and domains by optimizing model- and domain-specific safety mechanisms, enhancing the safety-utility frontier and providing more flexible deployment solutions.
Technical Contribution
EvoSafeHarness's technical contributions lie in its flexible and adaptive optimization framework. Unlike existing fixed defenses, it can adjust safety mechanisms according to specific model and domain needs, offering new theoretical guarantees and engineering possibilities.
Novelty
EvoSafeHarness is the first to achieve model- and domain-specific safety mechanism optimization. Compared to related work like SafeHarness, its innovation lies in the adversarially validated and domain-specification-guided search process, providing a more efficient safety-utility balance.
Limitations
- EvoSafeHarness may require longer optimization times in complex domains, affecting deployment efficiency.
- In extreme adversarial environments, additional safety mechanisms may be needed to ensure comprehensive protection.
Future Work
Future work could explore the adaptability of EvoSafeHarness in more domains and its performance in more complex adversarial environments. Further optimization of the search algorithm to improve efficiency is also an important direction.
AI Executive Summary
EvoSafeHarness is an optimization framework for large language model agents, designed to address the adaptability issues of existing system-level safety mechanisms across different models and domains. Existing safety mechanisms are typically designed by experts and applied across heterogeneous models and domains, which can lead to over-blocking or missing safety relations in actual deployments.
EvoSafeHarness synthesizes deployable safety mechanisms for frozen models by jointly searching natural-language policy and executable code logic, guided by model behavior, domain specifications, and fresh-context adversarial review. Across four agent benchmarks, EvoSafeHarness outperforms fixed expert-designed defenses on the safety-utility frontier. On DecodingTrust-Agent, EvoSafeHarness reduces the average attack success rate from 45.6% to 10.0% with only a 3.3-point utility cost.
The success of EvoSafeHarness lies in its flexible optimization framework, which can adjust safety mechanisms according to specific model and domain needs. This approach not only enhances the safety-utility balance but also provides new insights for future safety mechanism design. Although longer optimization times may be required in complex domains, its adaptability and efficiency advantages make it a promising security solution.
Deep Analysis
Background
As large language model agents transition from demonstration to deployment, they increasingly access sensitive data, financial accounts, production systems, and external services, making safety an operational requirement. Existing model-level defenses improve the model's ability to distinguish trusted instructions but lack a system-level enforcement boundary independent of model behavior, prompting the implementation of system-level defenses.
Core Problem
Existing safety mechanisms are typically designed by experts and applied across heterogeneous models and domains, which can lead to over-blocking or missing safety relations in actual deployments. Achieving the best safety-utility trade-off across different models and domains is a pressing issue.
Innovation
EvoSafeHarness synthesizes deployable safety mechanisms for frozen models by jointly searching natural-language policy and executable code logic, guided by model behavior, domain specifications, and fresh-context adversarial review. Unlike existing fixed defenses, it can adjust safety mechanisms according to specific model and domain needs.
Methodology
- �� Joint search of natural-language policy and executable code logic
- �� Guided by model behavior, domain specifications, and fresh-context adversarial review
- �� Synthesizes deployable safety mechanisms for frozen models
- �� Validated across four agent benchmarks
Experiments
Experiments were conducted across four agent benchmarks, including DecodingTrust-Agent, AgentDojo, Agent-SafetyBench, and AgentDyn. Metrics used include attack success rate (ASR) and utility. Baseline comparisons include CaMeL, DRIFT, and Progent.
Results
EvoSafeHarness reduces the average attack success rate from 45.6% to 10.0% on DecodingTrust-Agent with only a 3.3-point utility cost. On AgentDojo, it achieves 82.8% utility at 0.0% ASR, twice the utility of CaMeL.
Applications
EvoSafeHarness can be applied in fields requiring high security and utility balance, such as finance, healthcare, and autonomous driving. Its flexible optimization framework makes it suitable for various models and domains.
Limitations & Outlook
EvoSafeHarness may require longer optimization times in complex domains, affecting deployment efficiency. In extreme adversarial environments, additional safety mechanisms may be needed to ensure comprehensive protection.
Plain Language Accessible to non-experts
Imagine you're in a kitchen cooking. EvoSafeHarness is like a smart chef assistant that automatically adjusts cooking methods based on different ingredients and recipes, ensuring each dish is both delicious and safe. Existing chef assistants might only be suitable for certain specific recipes, but EvoSafeHarness can flexibly adjust cooking strategies based on different ingredients and needs, avoiding overcooking or missing key steps. This way, no matter what dish you're making, it helps you achieve the best taste and safety standards.
ELI14 Explained like you're 14
Imagine you're playing a game where your task is to protect a castle. EvoSafeHarness is like a super-smart defense system that automatically adjusts defense strategies based on different enemies and attack methods. Existing defense systems might only be suitable for certain specific attacks, but EvoSafeHarness can flexibly adjust defense strategies based on different enemies and attack methods, ensuring your castle is always safe. This way, no matter how cunning the enemies are, you can easily handle them!
Glossary
Large Language Model
A type of artificial intelligence model capable of understanding and generating natural language, often used for tasks like conversation and translation.
Used in the paper to generate and execute natural-language policies.
Attack Success Rate
Measures the frequency of successful attacks, typically expressed as a percentage.
Used to evaluate the effectiveness of safety mechanisms.
Utility
Measures the system's performance under normal operation, often contrasted with security.
Used to assess the impact of safety mechanisms.
Adversarial Review
An evaluation method that tests system security by simulating attacks.
Guides the optimization of safety mechanisms.
System-Level Safety Harness
A security layer independent of the model, used to enhance overall system security.
The optimization target in the paper.
Open Questions Unanswered questions from this research
- 1 How to improve EvoSafeHarness performance in extreme adversarial environments?
- 2 How adaptable is EvoSafeHarness in more domains?
Applications
Immediate Applications
Financial Security
EvoSafeHarness can be used in financial systems to ensure transaction security and prevent fraud.
Long-term Vision
Autonomous Driving Safety
EvoSafeHarness can be used in autonomous driving systems to ensure vehicle safety in complex environments.
Abstract
Large Language Model (LLM) agents are turning language into real-world effects, making safety necessary against both indirect prompt injections and direct harmful requests. System-level safety harnesses add an enforcement layer beyond model-level defenses, but existing harnesses are usually designed once by experts and applied across heterogeneous models and domains. Effective protection is deployment-dependent: models differ in how much enforcement they need before utility declines, while domains differ in the effects, state, and action sequences that must be governed. A harness that is strict enough for one model may over-block another, and a policy that transfers across domains may miss application-specific safety relations. We present EvoSafeHarness, a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain. It jointly searches a natural-language policy and executable code logic, guided by model behavior, domain specifications, and fresh-context adversarial review to reject benchmark-specific rules. Across four agent benchmark families, EvoSafeHarness achieves a stronger safety-utility frontier than fixed expert-designed defenses. On DecodingTrust-Agent, it reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost and achieves the best score in 14 of 15 cells. On AgentDojo, it reaches 82.8% utility at 0.0% ASR, twice CaMeL's utility at the same operating point, and transfers unchanged to unseen AgentDyn suites. It also achieves the best score on Agent-SafetyBench for every victim and keeps mean ASR below 20% under adaptive PAIR attacks with a refinement budget of 16. Analysis shows that domain semantics determine which safety relations and trajectory state are needed, while model and runtime behavior determine how and where those relations should be enforced.