Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems
AgingBench evaluates long-lived AI agents via four mechanisms, enabling diagnosis and targeted repair to sustain reliability.
Key Findings
Methodology
This work develops AgingBench, a comprehensive framework that models agent aging through four mechanisms: compression, interference, revision, and maintenance. It employs temporal dependency graphs and counterfactual probes to analyze multi-session interactions, tracking how each mechanism impacts memory integrity and performance over time. The benchmark uses programmatically generated scenarios with controllable parameters such as dependency density, update frequency, and interference levels, allowing systematic and reproducible stress testing. During evaluation, the system records detailed diagnostic profiles at each stage of the memory pipeline—write, retrieval, and utilization—enabling precise localization of failures. The approach integrates mechanism-specific metrics to produce aging curves across diverse models and policies, providing a multi-dimensional view of agent degradation.
Key Results
- Across seven scenarios and fourteen models, results show that behavioral metrics like response fluency can remain stable while factual accuracy declines by up to 30%. Derived-state tracking often exhibits sharp collapse within a single session, indicating high risk of information loss. Different aging mechanisms manifest distinct signatures: compression causes detail omission, interference leads to confusion among similar memories, revision failures produce stale answers, and maintenance events trigger abrupt regressions. The diagnostic profiles effectively distinguish these causes, guiding targeted repair strategies. The experiments demonstrate that relying solely on end-performance scores obscures underlying degradation, emphasizing the need for mechanism-aware evaluation.
- The impact of each mechanism varies by deployment context: long-term knowledge bases mainly face compression aging, while enterprise assistants encounter interference and revision issues. Routine maintenance, such as memory recompaction, can unexpectedly cause regressions. The framework’s diagnostic tools accurately attribute failures to specific stages, enabling stage-targeted interventions. These findings highlight that ensuring reliability in long-lived AI systems requires continuous, mechanism-level monitoring and adaptive repair, rather than static model improvements alone.
Significance
This research advances the field by providing a systematic, mechanism-oriented approach to evaluating and maintaining AI reliability over extended deployment periods. It addresses a critical gap in existing benchmarks, which focus mainly on initial performance, by enabling detailed diagnosis of degradation causes. The framework supports the development of robust long-term AI systems capable of operating safely in real-world applications like autonomous vehicles, healthcare, and finance. By facilitating targeted repairs and lifecycle management, it paves the way for AI that can sustain trustworthiness and effectiveness over years, not just days. This work thus lays a foundational step toward truly dependable AI in complex, dynamic environments.
Technical Contribution
The paper introduces a novel taxonomy of agent aging mechanisms, formalized through a temporal dependency graph that encodes cross-session facts and their evolution. It develops a set of mechanism-specific metrics—such as version accuracy, interference degree, and update fidelity—that quantify degradation over time. The evaluation harness incorporates paired counterfactual probes, enabling causal attribution of failures to specific stages in the memory pipeline. The programmatic scenario generator allows systematic stress testing across diverse deployment conditions, supporting reproducibility and scalability. This integrated framework surpasses prior benchmarks by combining mechanism-level diagnosis with stage-specific repair guidance, offering a comprehensive toolset for lifecycle management of AI agents.
Novelty
This work is the first to systematically classify and quantify long-term agent aging into four distinct mechanisms, integrating them into a unified evaluation platform. Unlike existing benchmarks that focus on static performance or end-to-end scores, AgingBench emphasizes mechanism-specific diagnostics and stage-targeted repairs. The use of temporal dependency graphs and paired counterfactual probes for causal failure attribution represents a significant methodological innovation. This approach enables precise diagnosis and intervention, setting a new standard for evaluating the robustness of long-lived AI systems in real-world scenarios.
Limitations
- 模拟场景虽能控制变量，但与实际部署环境中的复杂交互和噪声仍有差距，可能影响诊断的准确性。
- 对超大模型和超长会话的扩展存在计算成本高的问题，需优化算法效率。
- 不同机制间的交互影响尚未充分研究，未来需结合多机制联合分析以提升诊断和修复效果。
Future Work
未来将结合真实部署数据，优化场景生成策略，提升模型在实际环境中的适应性。探索多机制联合诊断与修复策略，结合强化学习和自适应调节，提升系统的持续可靠性。还将扩展多模态、多任务场景，推动长寿AI在更复杂应用中的实践落地。
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
想象你有一个超级智能的机器人助手，它每天都在帮你完成各种任务，比如整理房间、做作业、安排日程。刚开始，它非常聪明，能帮你解决很多问题，但随着时间推移，它会变得不那么可靠。有时候，它会忘记你说过的事情，有时候会把相似的东西搞混，甚至会突然出错。你需要定期检查它，修理它，教它新技能，确保它能一直帮你忙。这个研究就像是发明了一套检测和修复机器人助手的系统，帮助你了解它哪里出了问题，怎么修好它，保证它能长时间帮你。通过模拟不同的日常场景，观察它的表现变化，找到潜在的隐患，然后修补它。这样，你的机器人助手就能一直陪伴你，变得更聪明、更可靠！
Abstract
Long-lived AI agents are increasingly deployed as persistent operational systems, yet they are still evaluated like freshly initialized models. Day-one benchmarks miss a basic systems question: how long does an agent remain reliable after deployment? Even when model weights are frozen, an agent's effective state keeps changing as it compresses interaction history, retrieves from a growing memory store, revises facts after updates, and undergoes routine maintenance. Reliability therefore becomes a lifespan property of the full agent harness, not only a snapshot property of the base model. We introduce AgingBench, a longitudinal reliability benchmark for agent lifespan engineering: measuring not only whether deployed agents degrade, but what form the degradation takes and where repair should target. AgingBench organizes agent aging into four mechanisms: compression aging, interference aging, revision aging, and maintenance aging. To diagnose these failures, AgingBench uses temporal dependency graphs and paired counterfactual probes that produce diagnostic profiles for the write, retrieval, and utilization stages of the memory pipeline. Across 7 scenarios, 14 models, multiple memory policies, and both runner-controlled and autonomous agents, over ~400 runs spanning 8 - 200 sessions show that agent aging is not one-dimensional: behavioral tests can remain clean while factual precision decays; derived-state tracking can collapse sharply within a single model; and the same wrong answer can require different repairs depending on what the diagnostic profile points to. These results suggest that reliable agent deployment requires lifespan evaluation, mechanism-level diagnosis, and stage-targeted repair, not only stronger day-one models.