SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent
SAM introduces an external, state-adaptive memory system that significantly improves long-horizon reasoning, outperforming baselines with 8-10% gains on benchmarks.
Key Findings
Methodology
SAM employs a page-based external memory architecture combining compact memory cues with raw trajectory pages. It uses expert-guided supervised fine-tuning and reinforcement learning (OAT-GRPO) to optimize the memory module. The cues serve as lightweight handles for intent-driven recall, enabling dynamic reconstruction of distant information. The tree-structured recall process, coupled with oracle-anchored rewards, ensures high relevance and recoverability. Extensive experiments on datasets like BrowseComp and HLE demonstrate consistent performance improvements over traditional compression or retrieval strategies across diverse backbone models, validating the framework’s robustness and generalization.
Key Results
- On benchmarks such as BrowseComp and BrowseComp-ZH, SAM achieves an average performance boost of approximately 8-10 percentage points, with the largest gains (up to 15%) in tasks requiring extensive long-distance memory. In scientific reasoning tasks like HLE, accuracy reaches 69.2%, surpassing baseline scores of 65.4%. The model maintains stable improvements across different base models, including Qwen3.5-35B-A3B and GLM-4.7, confirming its broad applicability.
- Through expert-guided fine-tuning and reinforcement learning, the memory module aligns with trajectory-level utility, significantly enhancing long-term reasoning efficiency. Ablation studies reveal that removing the tree structure or oracle reward reduces performance by 4-6%, underscoring their importance.
- In real-world scenarios, SAM effectively manages ultra-long interaction histories, reducing the burden on the core model and improving reasoning coherence. Its design paves the way for future multi-modal, multi-task long-term reasoning systems.
Significance
This work addresses a fundamental bottleneck in long-horizon reasoning: organizing and retrieving distant, scattered information efficiently. By explicitly modeling memory as a demand-driven, state-dependent resource, SAM substantially enhances the ability of large language models to perform complex, multi-step tasks. Its generalizable architecture offers a new paradigm for external memory systems, with implications for scientific research, decision support, and beyond. The approach mimics human active recall, providing a more natural and effective way to handle extended interactions, thus bridging a critical gap in current AI capabilities.
Technical Contribution
SAM introduces a novel page-based memory architecture that separates consolidation from retrieval, enabling flexible, demand-driven access. The framework combines expert supervision with reinforcement learning (OAT-GRPO) to optimize memory relevance and recoverability. The tree-structured recall mechanism, coupled with oracle-anchored rewards, ensures high-quality, context-specific information reconstruction. This design allows external memory to be both persistent and dynamically accessible, a significant departure from prior static or purely retrieval-based methods. The system’s modularity and training strategy facilitate broad applicability across models and tasks, setting new standards for long-term reasoning in AI.
Novelty
This research pioneers the formalization of long-horizon reasoning as a state-adaptive memory problem, moving beyond traditional recency-based or compression-only strategies. The integration of a cue-page architecture with a tree-structured recall process and oracle rewards is unprecedented. Unlike prior methods that treat memory as a static component, SAM’s demand-driven, intent-conditioned retrieval offers a dynamic, flexible approach. This combination of architectural and training innovations constitutes a significant leap forward in external memory design for AI reasoning systems.
Limitations
- Despite its strengths, SAM’s performance may decline in scenarios with extremely sparse or highly ambiguous memory cues, especially if page segmentation is suboptimal. The reliance on expert guidance and reinforcement learning increases training complexity and computational costs, limiting immediate scalability. Additionally, current implementation focuses on textual data; extending to multi-modal inputs remains a challenge. Future work must address these limitations to broaden applicability and efficiency.
Future Work
Future directions include integrating multi-modal memory, reducing training costs via unsupervised or semi-supervised methods, and enhancing autonomous memory management through meta-learning. Exploring adaptive page segmentation strategies and more efficient recall mechanisms will further improve scalability. Additionally, applying SAM to real-world applications such as scientific discovery, legal analysis, and complex decision-making will test its robustness and versatility, paving the way for more intelligent, context-aware AI systems.
AI Executive Summary
Long-horizon reasoning remains a significant challenge in AI, especially when models must access scattered, distant information over extended interactions. Traditional methods like summarization or retrieval often fall short when relevant data is distributed across long sequences or only becomes important later. To address this, the paper introduces SAM, a novel external memory framework that explicitly models long-term, demand-driven memory access. SAM divides interaction history into raw pages and compact cues, which serve as handles for intent-driven reconstruction. This architecture allows models to dynamically retrieve relevant distant information without retraining or excessive context length.
The core innovation lies in the combination of a page-based consolidation process, a tree-structured recall mechanism, and an oracle-anchored reward system. The consolidation compresses recent interactions into lightweight cues, stored externally alongside raw pages. When needed, the model selects cues based on current intent, reconstructs detailed information from raw pages, and integrates it into ongoing reasoning. This demand-driven approach enables models to focus on relevant distant information, significantly improving performance.
Extensive experiments across datasets like BrowseComp, HLE, and WideSearch demonstrate that SAM consistently outperforms traditional baselines, with improvements of 8-10% in key metrics. Its robustness across different backbone models and tasks confirms its generality. The framework’s modular design and training strategies, combining expert supervision with reinforcement learning, set a new standard for external memory in AI.
This work has broad implications for scientific research, legal analysis, and complex decision-making, where managing long-term dependencies is crucial. While promising, challenges remain in scaling to multi-modal data and reducing training costs. Future work will focus on these areas, aiming to develop more autonomous, efficient, and versatile long-term reasoning systems that can operate effectively in real-world environments.
Deep Analysis
Background
长远推理在人工智能中的研究逐步深入，早期工作如Memory Networks和Transformer增强机制解决了短期记忆问题，但面对超长交互历史时仍显不足。近年来，检索增强模型（如Retrieval-Augmented Generation）通过外部知识库改善信息访问，但多依赖静态索引，缺乏动态需求匹配。传统压缩策略（如Summarization）虽能减轻模型负担，但难以保证关键信息的完整性。随着大模型的发展，如何高效组织和利用长远信息成为研究热点。现有方案多在短期内表现良好，但在长距离信息的动态调取和重构方面仍存在瓶颈。
Core Problem
长远推理的核心难点在于信息的散布和时序跨度大，导致模型难以在有限上下文中找到关键线索。现有方法多依赖固定策略，忽视信息的动态需求匹配，导致推理准确率下降。尤其在复杂任务中，早期信息可能在后续决策中变得至关重要，但传统机制难以有效检索和重构这些信息，限制了模型的推理深度和连贯性。这一问题的解决需要一种能根据当前状态动态调节记忆访问的机制。
Innovation
本研究提出将长远推理中的记忆管理转化为状态自适应问题，区别于以往依赖recency或压缩的方案。引入页面-提示架构，将交互历史划分为多页面存储和轻量提示，结合专家微调和强化学习优化记忆质量。设计树状调用结构，使模型能根据当前需求动态选择和重构信息，显著提升远距离信息的可用性。这种机制突破了传统静态记忆的局限，为长远推理提供了新思路。
Methodology
- �� 将交互历史划分为连续页面，利用信息预算限制页面大小，确保局部连贯性。 • 每个页面通过Msum函数生成轻量提示，反映关键信息。 • 页面存储在外部存储器中，提示存入记忆库，保持在线上下文轻量。 • 通过专家指导微调记忆模型，确保提示与页面内容一致。 • 采用树状调用机制，模型根据意图选择提示，重构相关信息。 • 引入oracle锚定的奖励，强化记忆的相关性和恢复能力。 • 训练过程中结合强化学习（OAT-GRPO），优化记忆调用策略，提升整体推理性能。
Experiments
使用公开的长远推理任务数据集（如BrowseComp、HLE），对比多种基线（如压缩、检索、无管理策略）。评估指标包括任务成功率、推理准确性和记忆相关性。模型在不同基础模型（Qwen3.5-35B、GLM-4.7）上进行微调和强化训练，采用固定上下文窗口（如128K）进行推理。通过消融实验验证树结构和oracle奖励的贡献，确保设计的有效性。实验结果显示，SAM在所有任务中均优于传统方法，特别是在长距离记忆需求强烈的场景。
Results
在长远推理任务中，SAM平均提升性能约8-10个百分点，最大提升在长距离记忆场景中达15%。在HLE科学推理中，准确率达69.2%，优于65.4%的基线。模型在不同基础模型上表现一致，验证了其泛化能力。消融实验表明，去除树状调用或oracle奖励会导致性能下降4-6个百分点，强调设计的重要性。整体来看，SAM显著改善了远距离信息的访问和重构能力，为长远推理提供了新范式。
Applications
该方法适用于需要长时间交互和复杂推理的场景，如科学研究、法律分析、技术支持等。通过外部记忆管理，模型能更好地组织和利用历史信息，减少模型负担，提升推理连贯性。未来可结合多模态信息，实现跨模态长远推理，推动智能系统在实际应用中的广泛部署。
Limitations & Outlook
目前模型在极端长距离或信息稀疏场景下仍存在回忆不足的问题，尤其在页面划分策略未充分优化时。此外，训练过程依赖大量专家微调和强化学习，计算成本较高，实际部署时需考虑效率优化。未来需解决多模态信息融合和自主记忆调节等挑战，以实现更广泛的应用。
Plain Language Accessible to non-experts
想象你在一个大厨房里做饭，厨房里有很多食材和工具。每次你用到某个食材或工具，都把它放到一个特别的架子上。这个架子可以记住你用过的东西，但不会把所有东西都堆在一起，而是用标签标记每个食材或工具。等你需要用某个食材时，你可以根据标签找到它。这样，即使你忘记了某个食材放在哪里，也可以通过标签快速找到。这个系统让你在做大餐时，不会被乱糟糟的东西搞糊涂，也能随时找到需要的材料。它就像一个聪明的记忆助手，帮你整理和检索所有用过的东西，让做饭变得更轻松、更高效。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的游戏，你需要记住很多事情，比如任务目标、敌人位置和你之前的行动。每次你完成一部分任务，你会把重要的线索写在一张卡片上，放在一个特别的抽屉里。每次你需要做新任务时，你可以根据当前的情况，从抽屉里抽出几张卡片，快速找到之前的线索，帮你做出正确的决定。这就像有个聪明的助手帮你整理记忆，只在你需要时帮你找到重要的线索，而不是一直把所有事情都装在脑袋里。这样，你就能更快、更聪明地完成游戏中的挑战！
Abstract
Long-horizon agentic reasoning requires large language models to act over long interaction histories containing thoughts, tool calls, observations, and partial conclusions. The challenge is not merely that these histories grow long, but that information needed for the current decision may be scattered across distant steps and only become relevant later. Existing approaches address this difficulty by truncating the interaction history, compressing it into shorter surrogates, or retrieving selected parts of it for reuse, but they do not explicitly model how access to past interaction should adapt to the agent's evolving state. We instead cast long-horizon reasoning as a problem of state-adaptive memory. To this end, we propose State-Adaptive Memory~(SAM), a standalone framework that consolidates ongoing interaction into compact memory cues while preserving raw trajectory pages for intent-driven recall. These cues are not treated as replacements for history; rather, they serve as lightweight handles that allow the agent to reconstruct temporally distant information according to its current needs, without retraining the underlying backbone. We further optimize the memory module through expert-guided supervision and reinforcement learning, aligning it with trajectory-level utility. Across BrowseComp, BrowseComp-ZH, WideSearch, and HLE, SAM consistently outperforms strong baselines over diverse agent backbones. Our results suggest that explicit memory modeling provides a simple and effective foundation for long-horizon agentic reasoning.