Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads
Classified 10 agent memory systems, analyzed their system behaviors, and proposed 10 optimization strategies.
Key Findings
Methodology
This study adopts a system-oriented taxonomy, classifying agent memory systems along four axes: construction, storage, retrieval, and mutability. Using phase-aware profiling tools, it quantifies resource consumption during construction, retrieval, and generation phases. The evaluation involves two benchmarks—MemoryAgentBench and MemoryArena—covering 10 representative systems. Hardware and energy monitoring tools measure GPU utilization, latency, and power. The analysis reveals how design choices influence system costs, performance, and scalability, providing detailed insights into resource trade-offs and operational efficiencies.
Key Results
- The classification shows cost disparities up to two orders of magnitude based on construction mechanisms and storage structures. For example, Mem0 takes up to 13 hours for construction but achieves query latency as low as 0.1 seconds; conversely, A-Mem constructs faster but has a query delay of 38 seconds. External memory systems outperform pure long-context windows in multi-task scenarios, reducing average query latency below 0.2 seconds while maintaining accuracy between 36% and 47%. Energy analysis indicates over 70% of total energy is consumed during construction, highlighting optimization opportunities.
- In multi-task long-horizon scenarios, external memory systems significantly outperform traditional long-context approaches, enabling rapid retrieval and reasoning. Systems with extensive construction times (up to hours) can still deliver sub-second responses, demonstrating the efficiency of external memory architectures. The study finds that design choices in storage and retrieval directly impact bandwidth, latency, and energy consumption, guiding future system engineering. The results underscore the importance of balancing construction costs with query efficiency for scalable deployment.
- The analysis reveals that system design parameters—such as storage structure, retrieval policy, and update strategy—dramatically influence overall performance and resource utilization. The paper proposes 10 practical recommendations, including scheduling policies, capability thresholds, amortization via query volume, freshness-latency trade-offs, and large-scale management strategies. These insights aim to guide engineers and researchers in building efficient, scalable agent memory systems suitable for real-world applications, from enterprise AI to autonomous agents.
Significance
This work provides the first comprehensive system-level analysis of agent memory architectures, bridging the gap between theoretical models and practical deployment. By systematically evaluating cost-performance trade-offs, it offers crucial insights for designing scalable, efficient, and robust long-horizon AI agents. The findings address longstanding challenges in balancing memory capacity, retrieval speed, and energy consumption, paving the way for more intelligent, autonomous systems capable of sustained reasoning over extended periods. The research’s implications extend to industry, academia, and future AI system engineering, fostering innovations in persistent knowledge management and multi-task learning.
Technical Contribution
The paper introduces a novel taxonomy classifying agent memory systems along four axes—construction, storage, retrieval, and mutability—and develops a phase-aware profiling tool to measure resource consumption at each stage. It systematically evaluates 10 representative systems across two benchmarks, revealing how design choices influence costs and performance. The study also formulates 10 actionable system recommendations, emphasizing scheduling, capacity planning, and scalability. This comprehensive framework advances the understanding of system behaviors in long-horizon AI workloads, offering a unified approach to optimize memory architectures for large-scale deployment.
Novelty
This is the first work to systematically classify and analyze agent memory systems from a system engineering perspective, integrating detailed performance profiling with cost analysis. Unlike prior studies focusing solely on accuracy metrics, this research emphasizes resource efficiency, scalability, and system behavior. The phase-aware profiling tool and the comprehensive evaluation of diverse architectures provide new insights into the cost-performance landscape of long-term memory systems, marking a significant step forward in AI system design.
Limitations
- The analysis is primarily based on simulated environments and limited hardware configurations; real-world deployment may encounter additional challenges such as network latency, hardware heterogeneity, and long-term stability issues.
- The focus on construction and retrieval costs does not fully address maintenance strategies, such as pruning, deduplication, and conflict resolution, which are critical for long-term system health.
- Energy consumption during construction dominates the lifecycle costs, but the study does not explore hardware acceleration or energy-efficient algorithms in depth, leaving room for future optimization.
Future Work
Future research should explore adaptive scheduling algorithms for multi-user environments, optimize construction and maintenance processes for energy efficiency, and incorporate hardware acceleration techniques. Investigating dynamic storage organization and retrieval strategies tailored to evolving workloads will further enhance system scalability. Additionally, deploying these systems in real-world, long-term scenarios will validate their robustness, stability, and practical utility, ultimately enabling autonomous agents capable of sustained reasoning and learning.
AI Executive Summary
The rapid advancement of large language models (LLMs) like GPT-4 has revolutionized AI-powered agents, enabling complex reasoning and multi-turn interactions. However, managing long-term memory remains a critical bottleneck. Traditional approaches rely on the model's context window, which, despite recent scaling, faces fundamental limitations in handling multi-session, multi-task workloads due to quadratic growth in prefill costs and degraded recall fidelity. External memory systems, which persist interaction streams and selectively retrieve relevant information, offer a promising solution. Yet, their system-level behaviors, costs, and performance trade-offs are poorly understood.
This paper presents a comprehensive analysis of agent memory architectures, introducing a taxonomy that classifies systems along four axes: construction, storage, retrieval, and mutability. By developing a phase-aware profiling tool, the authors quantify resource consumption across different system components, revealing how design choices impact latency, energy, and scalability. Evaluations across 10 representative systems on two benchmarks demonstrate that construction costs can range from hours to minutes, while query latency varies from fractions of a second to over 30 seconds. External memory systems outperform pure long-context windows in multi-task scenarios, significantly reducing response times.
The findings highlight that system design profoundly influences operational costs and performance. Based on this, the authors propose 10 practical recommendations, including scheduling strategies, capacity management, and data freshness policies, to guide scalable deployment. These insights are vital for building efficient, robust, and scalable long-horizon AI agents capable of sustained reasoning and learning.
Overall, this work bridges the gap between theoretical memory models and real-world system engineering, providing a foundation for future innovations in persistent knowledge management. It addresses key challenges in AI system scalability, energy efficiency, and multi-task performance, paving the way for autonomous agents that can operate effectively over extended periods in complex environments.
Deep Dive
Plain Language Accessible to non-experts
想象你有一个超级大书架，里面装满了你所有的照片、笔记和故事。每次你想找某个照片或回忆，都可以从书架上快速找到。以前，你只能把所有东西都记在脑子里，但随着时间变长，脑子变得太满，找东西就变慢了。于是，你决定用一个电子书架，把重要的照片和笔记存进去，随时可以查找。这个电子书架可以帮你整理、更新，还能根据需要把内容变得更清晰。这样，不管你记得的事情有多长、多复杂，都能快速找到答案。这个系统就像我们研究的代理记忆一样，把信息存储在外面，随时调取，变得更聪明、更高效。
ELI14 Explained like you're 14
你知道吗，就像你在学校用笔记本记课本内容一样，电脑也需要记住很多信息。有时候，信息太多，记在脑子里就会乱，找起来也慢。于是，科学家们设计了一种“电子记忆系统”，就像一个超级大笔记本，把重要的事情都存进去。每次需要用的时候，它可以帮你快速找到答案。这个系统还可以自己整理、更新内容，就像你把旧笔记整理成新笔记一样。研究发现，有些系统花了很长时间整理内容，但查找速度特别快；有些系统花时间少，但查找慢。通过不断优化这些方法，我们可以让电脑变得更聪明，帮我们更快解决问题。未来，这些技术还能帮企业、学校和我们的日常生活变得更方便、更智能！
Abstract
LLM agents are increasingly deployed on long-horizon tasks requiring sustained reasoning over extended interaction histories. Realizing this at scale requires agents to persistently store, retrieve, and update their own memory across sessions. A rich ecosystem of agent memory systems has emerged spanning flat retrieval, LLM-mediated extraction, consolidating fact stores, and agentic control flows. Yet, their system-level behavior remains uncharacterized. We present the first systems characterization of agent memory. First, we introduce a system-oriented taxonomy classifying agent memory systems along four axes. Second, we build a phase-aware profiling harness attributing cost to construction, retrieval, and generation. Third, we characterize ten representative systems across two benchmark suites, uncovering how design choices shift cost across the write and read paths. Finally, we derive 10 system recommendations covering construction scheduling, capability floors, amortization via query volume, freshness-latency tradeoffs, and fleet-scale management.