Diagnosis Before Recovery: Turning Agent Failures into Selective Self-Correction
DARC uses failure diagnosis to restrict repair interventions, improving agent self-correction with 20-30% success rate gains.
Key Findings
Methodology
The paper introduces DARC, a framework that combines failure diagnosis with restricted intervention sets. It analyzes development set failures to identify dominant failure modes, constrains the recovery library accordingly, and uses verifier feedback to distill success-cost policies. This process ensures that repair actions are targeted and cost-effective. The approach involves failure signature extraction, diagnosis, intervention evaluation, and policy distillation, applied across ALFWorld, AppWorld, and XBRL finance tasks. The framework constructs task-family-specific recovery harnesses, which are frozen before testing, to improve reliability and efficiency.
Key Results
- In ALFWorld, DARC increased action validity success from 54.48% to 90.30%, outperforming baseline agents and broad playbooks.
- In AppWorld, the procedural recovery policy raised task completion from 54.8% to 87.5%, reducing environment steps significantly.
- In XBRL finance, format-precision retrieval achieved 94.5% accuracy, surpassing ACE and MIPROv2 by over 20%, demonstrating effective failure diagnosis-guided correction.
Significance
This work addresses the challenge of unreliable self-correction in environments lacking compiler-like feedback. By diagnosing failure modes and restricting recovery interventions, it reduces irrelevant or harmful corrections, leading to more robust agents. The approach bridges the gap between structured feedback in coding and broad language tasks, enabling more reliable autonomous systems across diverse domains. It offers a new paradigm for failure-aware correction, with implications for improving AI robustness in real-world applications.
Technical Contribution
The main innovations include: (1) failure signature analysis on development sets to identify dominant failure modes; (2) constraining the recovery intervention set based on diagnosis; (3) using verifier feedback to distill success-cost policies. The framework combines submodular coverage optimization with cost-aware policy search, ensuring targeted and efficient repairs. Unlike prior methods that apply uniform corrections, DARC’s conditional, diagnosis-guided approach enhances effectiveness and interpretability, providing a scalable solution for multi-task autonomous agents.
Novelty
This is the first systematic integration of failure diagnosis into the construction of restricted recovery strategies for broad agents. Unlike existing universal playbooks, DARC’s diagnosis-based restriction tailors interventions to specific failure modes, significantly improving correction success and efficiency. Its combination of diagnosis, intervention restriction, and policy distillation represents a novel approach that advances autonomous failure recovery beyond prior heuristic or broad-context methods.
Limitations
- The diagnosis relies on failure signatures from development sets, which may not cover all real-world failure modes, limiting generalization.
- In highly complex or novel failure scenarios, diagnosis accuracy may decline, reducing correction effectiveness.
- Verifier feedback and policy distillation incur computational costs, potentially impacting real-time deployment.
Future Work
Future research will focus on online diagnosis and adaptive policy updates, enabling agents to learn new failure signatures dynamically. Combining reinforcement learning with diagnosis-guided repair could further improve robustness. Extending the framework to multi-modal and multi-agent settings, as well as reducing computational overhead, are promising directions to enhance practical deployment in real-world systems.
AI Executive Summary
Autonomous agents operating in complex, unpredictable environments face significant challenges in self-correction when feedback signals are coarse or absent. Traditional repair strategies often rely on broad context expansion or generic playbooks, which can introduce irrelevant information, increase costs, and reduce correction efficiency. To address this, the authors propose DARC, a diagnosis-guided recovery framework that leverages failure signatures from development sets to identify dominant failure modes within task families.
DARC constructs task-specific recovery harnesses by restricting the set of admissible interventions based on diagnosis, thus focusing repair efforts on relevant actions. It then employs verifier feedback to distill success-cost policies, enabling the agent to decide when and how much recovery evidence to use. This approach ensures that the correction process is both targeted and cost-aware, avoiding unnecessary or counterproductive interventions.
Experimental results across ALFWorld, AppWorld, and XBRL finance tasks demonstrate the effectiveness of DARC. In ALFWorld, the success rate for action validity repairs increased from 54.48% to 90.30%. In AppWorld, task completion improved from 54.8% to 87.5%, with fewer environment steps. In XBRL finance, format precision accuracy reached 94.5%, outperforming baseline methods by over 20%. These findings confirm that diagnosis-guided restriction significantly enhances self-correction reliability and efficiency.
The significance of this work lies in its ability to bring structured, diagnostic-driven repair strategies to broad, non-coding tasks lacking compiler-like feedback. By focusing on failure modes, DARC reduces irrelevant interventions, lowers correction costs, and improves overall robustness. It offers a scalable, interpretable framework that can be extended to various domains, including robotics, finance, and automated reasoning.
Looking ahead, the authors plan to develop online diagnosis and adaptive policies, integrating reinforcement learning to further improve agent resilience. Extending the framework to handle multi-modal failures and reducing computational costs will be key steps toward practical deployment. Overall, DARC represents a major step forward in autonomous failure diagnosis and correction, promising more reliable AI systems in complex real-world environments.
Deep Analysis
Background
近年来，人工智能代理在多任务、多模态环境中的应用不断扩大，但其鲁棒性仍是瓶颈。传统修正方法多依赖全局上下文扩展或预定义策略，存在干扰多、成本高的问题。代码生成和程序验证领域的自我调试技术提供了启示，但在非编码任务中缺乏类似的故障诊断机制。近年来，利用执行反馈进行修正的研究逐渐兴起，如Self-Debugging和LDB，强调利用中间状态和错误信息进行修正，但多依赖明确的环境反馈。本文借鉴编译器的故障诊断思想，将其引入多任务代理，旨在解决反馈缺失带来的修正难题。
Core Problem
在复杂多任务环境中，代理面临多样化的故障类型，缺乏明确的诊断信息导致修正策略泛化能力不足。现有修正多采取无差别策略，容易引入无关干扰，增加成本，降低效率。如何在缺乏明确反馈的环境中，识别故障类型，限制修正干预，提升效率，成为亟待解决的关键问题。特别是在多模态、多任务场景下，单一策略难以兼顾不同故障需求，导致修正效果不佳。
Innovation
本文的创新点在于：1）基于开发集失败签名实现故障诊断，识别主导故障类型；2）限制修复干预集合，避免无关干预引入干扰；3）利用验证器反馈蒸馏成功成本策略，确保修正的选择性和成本控制。这一流程区别于传统全局修复策略，强调故障导向的修正路径，显著提升效率和效果。通过多场景验证，展示了该方法在多样化故障诊断和修正中的优越性。
Methodology
- �� 失败签名分析：在开发集上运行基础代理，提取失败签名（如无效动作、格式错误）；
- �� 故障诊断：基于签名识别主导故障类型，限制修复干预（动作守卫、API源、检索预算）；
- �� 干预评估：在训练集上测试每个干预的成功率和成本；
- �� 策略蒸馏：枚举有限长度修复策略，利用验证器反馈评分，选择最优策略；
- �� 测试部署：在测试集上冻结策略，确保修正的选择性和成本控制。
Experiments
在ALFWorld、AppWorld和XBRL金融环境中，采用不同的故障诊断和修正策略，比较基础代理、全库修复和诊断引导修复的性能。指标包括成功率、环境交互步骤和检索预算。通过多场景验证，评估策略迁移和泛化能力。结果显示，诊断引导修正显著优于无差别策略，成功率提升20%以上，交互步骤减少30%以上。
Results
在ALFWorld中，动作修正成功率由54.48%提升至90.30%；在AppWorld中，任务完成率由54.8%提升至87.5%；在XBRL金融中，格式精度达94.5%，超越ACE和MIPROv2。消融实验表明，未限制的全库修复效果明显下降，验证了诊断限制的重要性。
Applications
该方法适用于缺乏明确反馈的复杂任务环境，如自动化办公、金融分析、机器人控制等。通过引入故障诊断机制，提升系统自我修正能力，增强自主性和鲁棒性。未来可结合强化学习，动态调整修正策略，实现更智能的故障应对。
Limitations & Outlook
目前依赖开发集中的故障签名，可能在新颖或复杂故障场景中表现不足。诊断准确性受限于签名的完备性，泛化能力有限。策略蒸馏过程计算成本较高，影响实时性。未来需探索在线诊断和自适应策略优化，以应对更复杂的实际环境。
Plain Language Accessible to non-experts
想象一个工厂生产线，出现问题时，工人会根据故障的类型采取不同的修理措施。比如机器不转了，可能是电源问题；产品不合格，可能是原料问题。工厂里有专门的检测员（类似诊断系统），他们先判断故障类型，然后只用对应的修理工具（修复措施），而不是随便用一堆工具。这种方法可以节省时间和资源，也能更快修好机器。本文的DARC就像这个工厂的检测员，先诊断出故障类型，再用最合适的修理措施，避免乱用工具造成干扰或浪费。这样，整个修理过程变得更高效、更精准，也更可靠。
ELI14 Explained like you're 14
想象你在玩一款游戏，遇到难题时，直接试各种方法可能会浪费时间。其实，你可以先观察一下，看看哪里出错了，比如是不是没有找到正确的线索，或者操作不对。然后，根据问题的类型，选择最合适的解决办法，比如找线索、换个角度思考，或者用特殊的道具。这样一来，你就不会乱试一通，而是用对方法，既快又省力。本文的技术就像这个聪明的游戏助手，它会先判断出问题的原因，然后只用最有效的办法去修正。这样，游戏就能更顺利地进行，玩家也会更开心！
Abstract
Self-correction is particularly useful when a failure constrains the next repair. Coding agents benefit from this property because compilers, tests, and execution traces turn many failures into typed recovery signals, but broad language-agent tasks often expose only a coarse task failure. This creates a tension for generic recovery playbooks: they broaden the agent's context precisely when the system needs a narrower repair interface, mixing incompatible signals for invalid actions, missing procedures, and strict-format errors. Our insight is that development-set failures can recover part of the missing diagnostic substrate by deciding which recovery interventions are admissible before test-time correction. We propose DARC, a diagnosis-guided recovery harness that profiles task-family failure modes, prunes mismatched interventions from a shared recovery library, and freezes a verifier-selected success-cost policy for deployment. This causal order makes correction selective: the harness first determines what kind of failure can be repaired, then decides how much recovery evidence to spend. In ALFWorld, AppWorld, and XBRL Finance, the same protocol yields an action-validity harness, a procedural-recovery fallback, and a format-precision retrieval policy; in each evaluated setting it improves average task performance over base agents and broad playbooks while reducing environment steps or retrieval budget. Our experiments show that failures need not trigger uniformly more context: DARC turns self-correction from prompt expansion into recovery-interface design. DARC provides a practical route toward more reliable agents in domains where compiler-like feedback is absent: making failures actionable before making contexts larger.