Functional Cache Grafting: Robust and Rapid Code-Policy Synthesis for Embodied Agents
FCGRAFT uses function-level KV-cache grafting to improve embodied-policy success by 18.31% and synthesis speed by 2.3×.
Key Findings
Methodology
FCGRAFT stores validated function skeletons with Transformer key–value states. It retrieves interfaces from a two-tier cache, uses cache-stitching to compose function calls, and invokes cache-patching to regenerate only error spans. Cache retention combines recency, invocation frequency, conditional co-occurrence, and perplexity-based semantic relevance.
Key Results
- Across ALFRED, TEACh, RLBench, and real-world manipulation tasks, FCGRAFT reportedly improves average task success by 18.31% over RAGCache and synthesizes policies 2.3× faster.
- Reusing validated implementations reduces internal API and control-logic failures; residual errors are primarily surface-level issues such as mismatched parameters or variables.
- The pipeline’s logic is complementary: stitching provides structural reuse and error isolation, while patching performs localized repair. Their combination produces the reported robustness–latency trade-off.
Significance
The work targets two major deployment barriers for embodied Code-as-Policies: repeated prefill over long specifications and unreliable full-code generation. It advances memory reuse from text retrieval to executable functions and attention states, offering a route toward responsive robotic programming in open-domain environments.
Technical Contribution
The central contribution is a two-tier function cache H=(I,C). Interface tier I stores signatures, metadata, and iKV for lightweight synthesis; code tier C stores executable implementations and cKV for linking and patching. The locality score explicitly combines immediate retention with long-term frequency, association, and semantic diversity.
Novelty
Unlike RAGCache and document-level KV reuse, FCGRAFT treats callable functions as cache units and unifies cache-stitching with cache-patching in an embodied-control loop. Its fundamental innovation is reusing execution-validated control structures and their attention states, rather than merely retrieving similar text.
Limitations
- The supplied paper text reports aggregate improvements but does not provide all per-dataset scores, confidence intervals, or significance tests, limiting detailed assessment of task-specific stability.
- The approach depends on compatible APIs, implementations, and error localization. Major changes in robot hardware, dynamics, or safety constraints may exceed local patching and require full regeneration or human verification.
Future Work
Future research should address cache transfer across models, robots, and APIs; improve exception localization and safety validation; and quantify GPU memory, DRAM offloading, cache pollution, and continual-update costs over long deployments.
AI Executive Summary
Robots that receive instructions such as “stop the gas leak” must translate language into executable control programs. Code-as-Policies enables CodeLLMs to call perception and motor APIs directly, but conventional full regeneration repeatedly prefills long prompts and can invent incompatible APIs, omit safety guards, or produce unstable logic. In time-critical environments, this latency and unreliability can become a physical failure.
FCGRAFT addresses both problems through function-level cache grafting. Successful programs are decomposed into a Function-Interface tier containing lightweight signatures and KV states, and a Function-Code tier containing validated implementations. For a new task, cache-stitching injects relevant interface states and composes a policy; implementations are linked for execution. If an exception occurs, cache-patching preserves the correct prefix and suffix and regenerates only the faulty middle span, guided by a temporary chain-of-thought diagnosis. Cache locality is managed using recency, frequency, co-occurrence, and semantic relevance.
Experiments on ALFRED, TEACh, RLBench, and real-world manipulation tasks report an 18.31% average task-success improvement and 2.3× faster policy synthesis over RAGCache. The result reframes robotic memory as reusable executable structure plus attention state. However, the provided text lacks complete per-dataset statistics, and major API or hardware shifts may defeat local repair. Safety guarantees, transferability, and long-term cache management remain open engineering challenges.
Deep Analysis
Background
Embodied control has moved from LLM planning toward executable Code-as-Policies. Representative systems by Liang et al. and Vemprala et al. let CodeLLMs invoke perception and motor APIs, while memory-based agents reuse trajectories and experiences. RAGCache and related methods reuse text or document-level KV states, and PromptCache supports modular cache composition. Yet function-level adaptation for dynamic robotic programs and low-latency execution remains underdeveloped.
Core Problem
Given a partial observation ot and natural-language instruction τ, a CodeLLM must generate executable policy πcode maximizing success while minimizing policy-synthesis latency and preserving behavioral consistency. Long prompts cause repeated prefill; unconstrained generation causes API mismatches, missing guards, and faulty control logic. The challenge is to reuse invariant skills while adapting task-specific parameters and constraints.
Innovation
FCGRAFT introduces three linked ideas. First, functions—not documents—are the cache units. Second, interface and implementation data are separated: I supports lightweight generation, whereas C supports execution and editing. Third, stitching and patching form one pipeline: stitching reuses validated structures and confines errors to adaptation surfaces; patching repairs only those surfaces, unlike text-level retrieval in RAGCache.
Methodology
- �� Objective: maximize SR−ηPSL+μCSIM, balancing success, synthesis latency, and behavioral consistency.
- �� Cache: H=(I,C), with I={(fk,itextk,iKVk)} and C={(fk,ctextk,cKVk)}.
- �� Management: score each function with ℓ(fk)=(1−ℓcurr)(αℓfreq+βΣℓasso+γℓsema)+ℓcurr, where α+β+γ=1; low-value entries move to DRAM.
- �� Stitching: condition πθ on observation, instruction, and selected interface KV states; generate calls and link matching validated implementations.
- �� Patching: split πexec into [xpre||xerr||xsuf], reuse xpre KV, generate xmid with an error-guided trace, and reconstruct the program.
- �� Loop: execute, catch exceptions, patch, and update the cache after success.
Experiments
The evaluation spans ALFRED, TEACh, RLBench, and real-world robotic manipulation. RAGCache is the principal baseline. Metrics emphasize task success rate and policy synthesis latency, with analyses of cache management and the stitching–patching pipeline. The supplied text specifies the aggregate gains—18.31% higher success and 2.3× faster synthesis—but omits complete per-dataset values and model hyperparameters.
Results
FCGRAFT delivers a better robustness–latency balance across open-domain settings. Relative to RAGCache, it raises average task success by 18.31% and accelerates synthesis by 2.3×. Mechanistically, validated functions suppress internal API and logic failures, while patching confines remaining problems to local parameter, variable, or runtime-error regions.
Applications
The framework suits mobile robots, manipulators, warehouses, homes, and laboratories where task instructions change but core APIs remain stable. Deployment requires validated function libraries, reliable exception reporting, and GPU/DRAM cache management. Its main industrial value is faster response without repeatedly regenerating safety-critical control structure.
Limitations & Outlook
Performance depends on the quality and compatibility of cached functions and on accurate error-span localization. GPU memory pressure and DRAM transfers may introduce overhead; entirely novel skills or major dynamics changes cannot be solved by local patching. Future work should provide fuller statistical reporting, stronger safety verification, cache migration, and long-horizon continual-learning evaluations.
Plain Language Accessible to non-experts
Imagine a restaurant receiving a new order every few seconds. A conventional robot chef rereads the entire cookbook and rewrites every instruction for each dish. That wastes time, and it may forget a crucial step such as turning off the stove or use the wrong tool name.
FCGRAFT keeps a cabinet of small recipes that have already worked: one for cutting, one for controlling heat, one for checking whether the food is ready. When a new order arrives, it selects and joins the useful recipes instead of rewriting the whole meal. This is stitching. If the customer asks for less salt, or one setting is wrong, it changes only that small step while keeping the rest intact. This is patching.
The paper tested this idea on ALFRED, TEACh, RLBench, and real robot manipulation. Compared with RAGCache, it reports 18.31% higher task success and 2.3× faster policy synthesis. The central lesson is simple: a robot can become faster and safer by reusing proven procedures rather than improvising every action.
ELI14 Explained like you're 14
Picture a robot in a video game. The mission says, “Find the cup, pick it up, and put it on the desk.” If a computer writes the entire strategy from scratch every time, it might use the wrong button name, forget to check the grip, or spend ages rereading the game manual.
FCGRAFT works like a smart collection of game moves. It saves reliable moves such as “grab,” “move,” “check success,” and “retry.” For a new level, it connects the useful moves into a combo. That is stitching. If one number, target position, or button is wrong, it does not delete the whole combo—it repairs just the broken part. That is patching.
The researchers tested it with ALFRED, TEACh, RLBench, and real robots. Compared with RAGCache, it achieved 18.31% better task success and made policy creation 2.3 times faster! Why? It avoids repeating work and reuses moves that have already been tested.
There is a catch: saved moves must match the robot’s buttons and rules. A completely new robot or totally different physics may need a brand-new strategy. Future systems will need stronger safety checks and better sharing across robots.
Glossary
Code-as-Policies
A paradigm in which an LLM translates instructions into executable control code. The generated program directly invokes perception and motor APIs.
FCGRAFT accelerates and stabilizes this paradigm for embodied agents.
KV cache
Previously computed Transformer key and value states used by attention. Reusing them avoids recomputing context representations.
The paper reorganizes KV caching at the function level.
Cache-stitching
Composing a new policy from cached function-interface states and calls. It primarily removes redundant prefill and preserves validated structure.
It is the first synthesis stage in FCGRAFT.
Cache-patching
Regenerating only the code span associated with an execution error. Correct prefix and suffix text remain unchanged.
It handles residual surface errors after stitching.
RAGCache
A retrieval-augmented generation baseline that reuses cached textual or document context. It is not specifically organized around executable functions.
FCGRAFT is compared against it for success and latency.
Locality score
A retention score combining recent use, frequency, conditional association, and semantic relevance. It determines whether cached states stay on GPU or move to DRAM.
Equation (3) manages the two-tier cache.
Open Questions Unanswered questions from this research
- 1 The paper text supplied here does not expose complete per-dataset scores, variance, or significance tests, so task-specific reliability cannot be fully judged.
- 2 Cross-model, cross-robot, and cross-API cache transfer remains unresolved; reusable attention states may be tightly coupled to interfaces and model internals.
- 3 Robust exception localization and formal safety guarantees for continuous control require deeper evaluation, especially when errors propagate across chained functions.
Applications
Immediate Applications
Warehouse mobile robots
Robots can reuse navigation, obstacle-avoidance, grasping, and verification functions while rapidly composing policies for new orders. Stable APIs, validated executions, and GPU/DRAM cache management are prerequisites; expected benefits are lower response latency and fewer structural code failures.
Laboratory manipulators
Manipulators can cache pick-and-place, switch operation, timeout, and retry procedures. When target coordinates or parameters change, patching modifies only the affected span, making the approach suitable for workstations with frequent task variation and stable underlying interfaces.
Long-term Vision
Continually learning robot fleets
Long-running robots could convert successful behaviors into executable functions with reusable KV states, building shared skill libraries. Major obstacles include cache poisoning, version control, safety auditing, hardware transfer, and managing incompatible model representations.
Abstract
Code-writing large language models (CodeLLMs) generate executable code policies for embodied agents by translating natural language goals and environmental constraints into structured control programs. However, policy generation in open-domain embodied environments suffers from two fundamental limitations: (i) delayed decoding caused by repetitive prefill computation over long prompts, and (ii) limited robustness due to fully generative decoding, which often produces API mismatches, missing safety guards, and unstable control logic. To address these limitations, we present FCGraft, a Functional Cache Grafting framework. FCGraft maintains a library of function-level validated code skeletons and their associated prompt-level Transformer key-value (KV) caches, and synthesizes new policies by retrieving relevant functions and grafting their KV caches when a new task is provided. Given retrieved function caches, FCGraft performs cache grafting via stitching, which composes cached function segments into a composite policy, and patching, which locally adapts only the necessary code regions to satisfy task-specific parameters and constraints with minimal additional decoding. By eliminating redundant prefill computation, this approach reduces generation latency, while reusing validated control structures improves robustness over prompt-level caching methods RAGCache, achieving 18.31% higher task success rate and 2.3x faster policy synthesis.