Invalidation Contracts for Cross-Episode Agent Memory
Introduces invalidation contracts to enhance cache validity and compliance with version stamps and cacheability hints.
Key Findings
Methodology
The study introduces an invalidation contract protocol layer that attaches version stamps and cacheability hints to each recovery suggestion, allowing clients to evict stale entries without trial and error. The protocol decomposes realized savings into two independent factors: validity and compliance.
Key Results
- Row-level invalidation raised compliance by 0 to 66.7 percentage points across seven models, recovering 29-33% of baseline token cost.
- Table-level invalidation dropped post-drift first-try rates to 0% on five models.
- Eviction precision is 1.00 at row granularity under the row-level oracle.
Significance
The study effectively addresses the issue of cached recovery suggestions failing post-data drift by introducing invalidation contracts. By applying cache-control semantics to recovery suggestions, it significantly improves cache validity and compliance, reducing unnecessary API calls and token consumption.
Technical Contribution
Technical contributions include the introduction of an invalidation contract protocol layer with six measured levels, from simple version stamps to dependency vector comparisons, evaluated across multiple models and serving paths to demonstrate protocol validity and compliance.
Novelty
This study is the first to apply cache-control semantics to API recovery suggestions, providing a systematic approach to managing the impact of data drift on caches, significantly improving cache validity and compliance compared to existing methods.
Limitations
- The protocol adds 15% to response payload, potentially impacting system performance.
- Table-level invalidation may destroy co-located entries.
Future Work
Future work could explore more granular invalidation strategies and more efficient protocol implementations to further enhance cache validity and compliance.
AI Executive Summary
In modern API environments, data drift often leads to cached recovery suggestions becoming invalid, increasing unnecessary API calls and token consumption. Existing solutions typically require re-derivation in every session, failing to leverage the benefits of caching effectively.
This paper introduces an invalidation contract protocol layer that attaches version stamps and cacheability hints to each recovery suggestion, allowing clients to evict stale entries without trial and error. The protocol decomposes realized savings into two independent factors: validity and compliance, evaluated across multiple models and serving paths.
Experimental results show that row-level invalidation raised compliance by 0 to 66.7 percentage points, recovering 29-33% of baseline token cost, while table-level invalidation dropped post-drift first-try rates to 0% on five models. This study provides a systematic approach to cache-control for API recovery suggestions, significantly improving cache validity and compliance.
Deep Analysis
Background
With the widespread application of large language model (LLM) agents, cross-episode memory has become crucial for improving API call efficiency. However, server-side data drift often renders cached recovery suggestions invalid, increasing unnecessary API calls and token consumption. Existing solutions typically require re-derivation in every session, failing to leverage the benefits of caching effectively.
Core Problem
The core problem is how to effectively manage cached recovery suggestions in the presence of data drift. Existing methods lack systematic management of cache suggestion validity, leading to re-derivation when caches become invalid, increasing system burden.
Innovation
This paper introduces an invalidation contract protocol layer that attaches version stamps and cacheability hints to each recovery suggestion, allowing clients to evict stale entries without trial and error. The protocol decomposes realized savings into two independent factors: validity and compliance, evaluated across multiple models and serving paths.
Methodology
- �� Introduce an invalidation contract protocol layer with version stamps and cacheability hints.
- �� Decompose realized savings into validity and compliance.
- �� Evaluate across seven models, three serving paths, and two domains.
Experiments
The experimental design includes seven models, three serving paths, and two domains, totaling approximately 9,400 episodes. The performance of invalidation contracts is evaluated by comparing the validity and compliance of different protocol levels.
Results
Experimental results show that row-level invalidation raised compliance by 0 to 66.7 percentage points, recovering 29-33% of baseline token cost, while table-level invalidation dropped post-drift first-try rates to 0% on five models.
Applications
The study's application scenarios include any system requiring efficient API call management, especially in environments with frequent data changes. By improving cache validity and compliance, unnecessary API calls and token consumption can be significantly reduced.
Limitations & Outlook
Although invalidation contracts significantly improve cache validity and compliance, the protocol adds 15% to response payload, potentially impacting system performance. Additionally, table-level invalidation may destroy co-located entries.
Plain Language Accessible to non-experts
Imagine a library with many books, each with a tag indicating its version. When the library updates a book, it changes the tag. Invalidation contracts are like the library's manager, ensuring that every time someone borrows a book, they get the latest version. This way, readers don't need to check the content's currency each time; they just look at the tag. This method saves time and improves the library's efficiency.
ELI14 Explained like you're 14
Imagine you're playing a massive multiplayer online game. Every time you complete a mission, the game gives you some tips on how to do better next time. But sometimes, the game updates, and these tips become outdated. Invalidation contracts are like a smart assistant in the game that automatically updates your tips when the game updates, so you always have the latest information to play. This way, you don't have to explore everything again, and you can complete missions faster!
Glossary
Invalidation Contract
A protocol layer that attaches version stamps and cacheability hints to each recovery suggestion to manage cache validity.
Used to improve the validity of cached recovery suggestions post-data drift.
Version Stamp
An identifier that marks the version of a cached suggestion for invalidation handling when data updates occur.
Attached to each recovery suggestion to help clients evict stale entries.
Cacheability Hint
A flag indicating whether a cached suggestion is cacheable, guiding client caching strategies.
Helps clients decide whether to retain or evict cached suggestions.
Data Drift
Refers to changes in server-side data that may render cached suggestions invalid.
A primary cause of cache invalidation.
Compliance
The proportion of cached suggestions applied by the planner model on the first attempt.
Depends on the planner model and protocol level.
Open Questions Unanswered questions from this research
- 1 How to further enhance protocol validity and compliance without increasing system burden?
- 2 How does invalidation contract performance vary across different domains and application scenarios?
Applications
Immediate Applications
API Call Optimization
By improving cache validity and compliance, unnecessary API calls and token consumption are reduced.
Long-term Vision
Intelligent Cache Management
Develop smarter cache management systems that automatically adapt to data changes, improving system efficiency.
Abstract
LLM agents that cache recovery suggestions from API errors can skip re-derivation in later episodes, spending fewer tokens and fewer model calls on constraints they have already learned. Server-side data drift turns those cached fixes into silent failures, and the usual remedy, re-deriving on every episode, gives the savings back. We introduce invalidation contracts, a protocol layer that attaches version stamps and cacheability hints to every recovery suggestion so the client can evict stale entries without trial and error, and keep the rest. The contract decomposes realized savings into two independent factors: validity, the fraction of cached suggestions that remain correct after a drift event, and compliance, the fraction the planner applies on the first attempt. Validity depends only on the protocol and is vendor-independent. Compliance depends on the planner model: identical wire bytes yield 100% first-try compliance on Claude Haiku 4.5 and 11% or below on Claude Sonnet 5, which exhibits input-schema conservatism, refusing fixes that add fields the original request did not contain. We evaluate across seven models, three serving paths, two domains, and approximately 9,400 episodes. Row-level invalidation raises compliance by 0 to 66.7 percentage points across the seven models, 55.6 to 66.7 on three, and recovers 29-33% of baseline token cost on four of seven models, while table-level invalidation destroys co-located entries and drops post-drift first-try rates to 0% on five of seven. Eviction precision is 1.00 at row granularity on every model under the row-level oracle of Section 4.1. The contract adds 15% to response payload. Version-stamp validity is deterministic by construction and produced identical results across every model and serving path, with zero contract failures in the entire evaluation.