When Your Agent Opens the Chat App: Agent-Controlled Search over Raw Chat Logs Rivals Structured Memory
ReFind is a keyword-based conversational search method achieving 58.2% accuracy without semantic indexing.
Key Findings
Methodology
ReFind employs lexical indexing (BM25) combined with four chat-native controls: session-aware rank fusion, local context expansion, temporal narrowing, and skipping inspected sessions. It performs multi-round keyword searches, collecting evidence for a separate reasoning stage. The system preserves raw conversations without semantic restructuring, relying on strategic search controls to enhance retrieval precision. Extensive experiments on MemoryAgentBench and LongMemEval demonstrate superior performance, with an average accuracy of 58.2%, outperforming graph/tree-based memory systems. Ablation studies confirm the importance of chat-native controls and agent-driven search strategies.
Key Results
- On MemoryAgentBench, ReFind achieves 58.2% mean accuracy, surpassing HippoRAG 2 (53.2%) and BM25 (48.8%), across 2800 questions in multi-task settings.
- In LongMemEval with gpt-4o, ReFind reaches 93.2% and 89.3% accuracy on S/M subsets, outperforming all structured memory models.
- Ablation results show that session-aware rank fusion and contextual controls significantly boost performance, while simple keyword search yields lower accuracy, validating the strategy control approach.
Significance
This work challenges the necessity of elaborate semantic structures for effective conversational retrieval. By demonstrating that strategic control over raw logs can match or exceed structured memory systems, it offers a cost-effective, flexible alternative. The approach simplifies system design, reduces maintenance overhead, and enhances adaptability for large-scale, long-term dialogue applications, marking a significant shift in retrieval paradigms for AI assistants.
Technical Contribution
The paper introduces a novel retrieval framework that eschews semantic indexing, relying solely on lexical matching combined with multi-dimensional search controls. It integrates RRF-based rank fusion, context windowing, temporal filtering, and session deduplication into an agent-controlled iterative search loop. This design allows dynamic, targeted evidence collection, enabling high-precision retrieval in unmodified conversation logs. The approach broadens the scope of retrieval strategies, emphasizing control and adaptivity over complex indexing, and demonstrates scalability with large language models like GPT-4 and GPT-5.
Novelty
This is the first work to eliminate semantic structure construction in conversational retrieval, instead leveraging a flexible, control-driven search paradigm over raw logs. Unlike prior methods relying on graph or tree structures, ReFind emphasizes iterative keyword search with strategic controls, aligning more closely with natural human search behaviors. This paradigm shift reduces system complexity and enhances real-time adaptability, representing a fundamental innovation in dialogue memory management.
Limitations
- The reliance on lexical matching may limit effectiveness in semantically ambiguous or paraphrased queries, especially in noisy or highly informal conversations.
- Performance may degrade with extremely long or complex dialogues where keyword signals are sparse or obscured.
- Current controls focus on temporal and session-based cues, but deeper semantic understanding remains an open challenge.
Future Work
Future research will explore integrating semantic understanding and multi-modal cues to improve retrieval in ambiguous contexts. Adaptive control mechanisms could be developed for better handling of noisy data or highly dynamic conversations. Combining this approach with structured memory or knowledge graphs may further enhance reasoning capabilities, enabling more robust, scalable dialogue systems.
AI Executive Summary
In the rapidly evolving field of conversational AI, effective long-term memory management remains a critical challenge. Traditional systems heavily depend on pre-built semantic structures, such as knowledge graphs or hierarchical summaries, to facilitate retrieval. While these methods improve accuracy, they come with high costs in index construction, maintenance, and inflexibility. This paper introduces ReFind, a novel approach that abandons semantic indexing altogether, relying instead on lexical search combined with strategic controls that mimic human search behaviors.
ReFind employs a simple yet powerful framework: it indexes raw conversation logs at turn level using BM25, and iteratively issues keyword queries. The system incorporates four chat-native controls—session-aware rank fusion, local context expansion, temporal narrowing, and skipping previously inspected sessions—to guide search dynamically. These controls allow the model to refine its evidence collection, focusing on relevant parts of the conversation without any semantic pre-processing.
Extensive experiments demonstrate that ReFind surpasses traditional structured memory systems across multiple benchmarks. On MemoryAgentBench, it achieves a mean accuracy of 58.2%, outperforming models like HippoRAG 2 by over 5%. In LongMemEval, with gpt-4o, it reaches 93.2% accuracy, validating the effectiveness of strategy-driven search over raw logs. Ablation studies confirm that these controls are crucial, with each component contributing significantly to overall performance.
This work fundamentally challenges the notion that elaborate memory structures are necessary for high-precision conversational retrieval. Instead, it shows that a flexible, agent-controlled search over unmodified records can achieve comparable or superior results, simplifying system design and reducing costs. The approach opens new avenues for scalable, adaptable dialogue systems, especially for long-term interactions where maintaining complex indices is impractical. Future directions include integrating deeper semantic understanding and multi-modal cues, further enhancing robustness and reasoning capabilities, and extending this paradigm to broader AI applications.
Deep Dive
Abstract
Agent-memory systems increasingly buy retrieval quality with structure, transforming raw conversation histories into summaries, embeddings, trees, or knowledge graphs before any question is asked. We ask how much of that benefit comes from the structure itself, rather than from competent retrieval over the raw history. We present ReFind, an agent-controlled search interface that builds no semantic structure at all: it leaves the conversation archive unmodified, indexes it lexically at turn granularity, and combines a generic iterative keyword-search loop with four chat-native controls grounded in empirical refinding work: session-aware rank fusion, local context expansion, temporal narrowing, and skipping already-inspected sessions. A separate reasoning stage answers from the collected evidence. Across a broad suite of conversational-memory tasks (single- and multi-hop QA, event ordering, and fact consolidation), roughly 2,800 questions on precise-retrieval and fact-tracking capabilities evaluated under the incremental multi-turn setting of MemoryAgentBench, ReFind attains the highest mean accuracy (58.2) of any system compared, above the strongest graph- and tree-based memory systems (HippoRAG 2, 53.2), all under a GPT-4o-mini backbone matched to every reused baseline. Controlled comparisons to single-shot BM25, a matched generic-agentic BM25 control, component removals, and agentic dense/hybrid variants separately support the roles of agent control, chat-native controls, and lexical retrieval. On LongMemEval-S/M, the same interface reaches 93.2 +/- 3.3 and 89.3 +/- 6.0 with GPT-5-mini. The results indicate that for precise, evidence-grounded questions over chat archives, much of the benefit credited to elaborate memory structures is recoverable by giving an agent controllable search over the unmodified record, with no LLM-based index construction at all.