BraveGuard: From Open-World Threats to Safer Computer-Use Agents
BraveGuard uses open-source threat mining and trajectory supervision to boost agent safety detection from 38.79% to 82.38%.
Key Findings
Methodology
BraveGuard employs open-source research mining to build a dynamic threat taxonomy, instantiates threats into executable tasks, collects agent trajectories, and trains guard models with trajectory-level labels. The core includes threat extraction, task synthesis, trajectory collection, and iterative updates. Using multi-round feedback, it continuously expands the threat library and refines models such as Qwen3-Guard and Llama-Guard. Extensive evaluations on AgentHazard show detection accuracy improves from 38.79% to 82.38%, with recall rising from 20.17% to 90.94%.
Key Results
- On AgentHazard, BraveGuard-trained guards achieve 82.38% accuracy, 90.94% recall, outperforming off-the-shelf models by over 43%. The recall boost reduces false negatives significantly.
- On ATBench-500, Qwen3-Guard-8B reaches 86.4% accuracy, 95.2% recall, F1 86.1%, demonstrating good cross-format generalization.
- Ablation studies confirm threat knowledge, task instantiation, and trajectory supervision are essential for performance gains.
Significance
This work advances AI safety from static, taxonomy-based detection to a dynamic, self-evolving framework capable of adapting to emerging threats. It addresses the critical need for continuous, real-world threat detection in complex agent systems, offering a scalable and robust solution for industry deployment.
Technical Contribution
The paper introduces a novel framework integrating open-source threat mining, executable task instantiation, and trajectory supervision, enabling models to learn from realistic, evolving attack scenarios. The multi-round iterative process enhances robustness and generalization, surpassing traditional static methods.
Novelty
This is the first approach to leverage open research sources for continuous threat discovery, combined with trajectory-level supervision, to create a self-adaptive safety system. It differs fundamentally from prior static, fixed-category detection models, emphasizing ongoing learning and evolution.
Limitations
- Detection of highly complex, multi-step attack chains remains challenging, especially in real-world noisy environments.
- Trajectory collection relies on simulated environments, which may differ from deployment conditions, affecting performance.
- Multi-round iterative training incurs high computational costs, requiring optimization for practical use.
Future Work
Future directions include integrating reinforcement learning for better threat synthesis, multi-modal data fusion for richer context understanding, and deploying in real-world systems to validate robustness and scalability.
AI Executive Summary
As AI agents evolve from simple chatbots to complex multi-tool systems, their security risks grow exponentially. Traditional safety measures, often based on static rules and fixed categories, struggle to keep pace with the rapidly changing threat landscape. Attackers leverage open-source information, novel tools, and multi-step strategies that can bypass existing defenses, creating a pressing need for adaptive, continuous security solutions.
BraveGuard introduces a novel framework that tackles this challenge head-on. It mines recent open research literature to identify emerging threats, organizes them into a structured, evolving taxonomy, and instantiates these threats as executable tasks. By running these tasks with AI agents, it collects detailed operation trajectories—comprehensive logs of actions, tool calls, and environment changes. These trajectories are then annotated with safety labels and rationales, forming a rich supervision dataset.
The core innovation lies in the iterative, self-evolving process. Using validation errors, BraveGuard expands its threat knowledge base, synthesizes new attack scenarios, and retrains guard models—such as Qwen3-Guard and Llama-Guard—on this continuously updated data. Extensive experiments demonstrate that models trained this way significantly outperform static, off-the-shelf detectors, achieving detection accuracy of over 82% and recall exceeding 90%, compared to less than 40% accuracy and around 20% recall for baseline models.
This approach marks a paradigm shift in AI safety, enabling models to adapt to the evolving threat landscape dynamically. It offers a scalable, robust solution for deploying safer AI agents in real-world applications, where threats are unpredictable and constantly changing. Despite current limitations in handling extremely complex attack chains and the high computational cost of iterative training, the framework paves the way for future research integrating reinforcement learning and multi-modal data to further enhance safety measures.
Deep Analysis
Background
AI agents从最初的对话系统逐步演变为具有多工具、多步骤操作能力的智能体，极大拓展了应用场景，但也带来了复杂的安全挑战。现有研究如OpenAI的安全检测框架和AgentHazard基准，主要关注静态范畴和短期响应，难以应对多轮交互中的潜在风险。近年来，研究者开始关注长链轨迹的安全检测，提出基于行为分析、异常检测等方法，但多依赖预定义规则和有限数据，难以应对不断变化的攻击策略。开源威胁信息、攻击手法不断演化，传统方法逐渐暴露出适应性不足的问题。如何构建一个持续学习、动态更新的安全体系，成为当前的研究热点。
Core Problem
核心问题在于，现有安全检测模型多基于静态数据和固定范畴，难以应对新兴、多步骤、多工具的攻击链。代理的行为轨迹复杂，单一响应或静态范畴无法捕捉潜在风险。如何实现持续的威胁发现、实例化和轨迹监督，提升检测的适应性和鲁棒性，是亟待解决的难题。这不仅关系到模型的安全性，也影响到实际应用中的风险控制效果。
Innovation
本研究提出基于开源资料的动态威胁挖掘体系，结合实例化任务和轨迹级监督，构建持续演化的安全检测框架。创新点包括：1）利用开源研究资料自动构建威胁知识库，动态跟踪新兴攻击；2）将威胁知识转化为具体可执行的攻击场景，确保动作在单步中合理；3）采集完整操作轨迹，捕捉多步骤行为中的潜在风险；4）引入闭环机制，根据验证中的错误样本不断扩展威胁库和优化模型。该体系区别于传统静态检测，强调持续学习和适应性，显著提升检测效果。
Methodology
- �� 威胁知识抽取：从arXiv、OpenReview等公开源自动挖掘新兴风险和攻击模式，构建结构化威胁分类体系。
- �� 任务实例化：将威胁知识转化为具体可执行任务，模拟攻击场景，确保动作在单步中合理。
- �� 轨迹采集：利用OpenClaw代理执行任务，采集完整操作轨迹，包括工具调用、文件修改、命令输出等。
- �� 轨迹标注：由专家或自动机制对轨迹进行安全标签，标明风险类别和理由。
- �� 模型训练：基于标注轨迹，训练多模型（如Qwen3-Guard、Llama-Guard），实现轨迹级安全检测。
- �� 迭代优化：利用验证中的错误样本，扩展威胁库，合成新任务，持续更新训练数据，形成闭环。
Experiments
采用AgentHazard和ATBench-500两个基准，分别评估模型在长链轨迹中的检测能力。训练过程中，采集多轮模拟攻击轨迹，比较不同模型（如静态模型、AgentDoG、BraveGuard）在准确率、召回率和F1上的表现。通过逐步消融验证威胁挖掘、任务实例化和轨迹监督的贡献。多模型融合和跨格式迁移测试模型鲁棒性，确保在不同环境和数据格式下的适应能力。
Results
训练的BraveGuard模型在AgentHazard上检测准确率达82.38%，比传统模型提升43.59%；召回率从20.17%跃升至90.94%，极大减少漏检。跨格式测试显示，Qwen3-Guard-8B在ATBench-500上达86.4%准确率，95.2%召回，验证了模型的泛化能力。逐轮消融分析确认，威胁知识库、任务实例化和轨迹监督三环节共同驱动性能提升。
Key Concepts
轨迹监督
基于完整操作轨迹进行安全评估，区别于单次响应检测，强调多步骤行为的整体风险。
开源威胁挖掘
利用公开研究资料自动识别新兴攻击和风险，动态更新威胁知识库。
实例化任务
将抽象威胁知识转化为具体可执行的攻击场景，用于采集真实轨迹。
闭环自适应
通过验证错误不断扩展威胁库和优化模型，实现持续自我演化。
Open Questions Unanswered questions from this research
- 1 如何在真实部署环境中高效采集和标注轨迹数据，减少人工成本。
- 2 模型在极端复杂攻击链中的检测能力仍有限，需增强多步骤推理能力。
Abstract
Computer-use agents extend language models from text generation to sustained interaction with files, terminals, browsers, and external tools. This shift creates safety risks that are difficult to detect from isolated prompts or final responses, because harm often emerges only through multi-step execution traces whose individual actions appear locally benign. We introduce BraveGuard, a self-evolving defense framework for training guard models from open-world threat signals and realistic agent trajectories. BraveGuard mines recent research sources to identify emerging risks and attack patterns, instantiates them as executable computer-use tasks, collects agent rollouts, and derives trajectory-level supervision for guard model training. As new threats and validation failures appear, the pipeline can be repeated, yielding an adaptive defense loop rather than a static, benchmark-driven training process. We instantiate BraveGuard by training multiple guard backbones, including Qwen3-Guard and Llama-Guard variants, and evaluate the resulting guards on trajectory-level agent-safety benchmarks. BraveGuard consistently improves safety detection across computer-use trajectories. On AgentHazard, it substantially improves detection accuracy over off-the-shelf guard models, with accuracy increasing from 38.79% to 82.38% under the averaged guard-model setting. These results show that guard supervision grounded in open-world threat discovery and realistic agent execution can improve safety monitoring beyond fixed taxonomies and synthetic prompt-level data. BraveGuard offers a scalable path toward adaptive defenses for computer-use agents facing evolving real-world risks.