DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback

The field of AI agents powered by Large Language Models (LLMs) has rapidly evolved, enabling automation of complex tasks such as code repair, desktop automation, and web navigation. Benchmarks like SWE-bench, OSWorld, and AgentBench have driven advances by providing realistic testing environments. Modern AI agents employ tree-structured search strategies, including Monte Carlo Tree Search (MCTS) and Language Agent Tree Search (LATS), to systematically explore possible action sequences. Execution-guided sampling methods like Best-of-N further enhance exploration by running multiple parallel candidate trajectories. These approaches require frequent checkpoint and rollback operations to manage the agent’s state, which comprises both durable filesystem state (e.g., working directories, installed packages) and ephemeral process state (e.g., memory, interpreter heap).

Reinforcement learning training also imposes infrastructure demands, as multiple sandbox instances must be rapidly cloned and restored to a known-good state for rollouts. Existing sandbox technologies, such as Docker containers and Firecracker microVMs, rely on full state duplication for checkpoint/restore, resulting in latencies from hundreds of milliseconds to seconds. This latency bottleneck restricts the depth of search trees and the scale of parallelism achievable, limiting the effectiveness of AI agents in real-world scenarios.

The core problem addressed is the high latency and inefficiency of checkpoint and rollback operations in stateful AI agent environments. Traditional methods duplicate the entire sandbox state—including filesystem and process memory—at each checkpoint, incurring prohibitive overheads of hundreds of milliseconds to seconds. This latency severely constrains the agent’s ability to perform deep tree search and large-scale parallel exploration. Additionally, existing approaches often manage filesystem and process states separately, risking inconsistency and semantic divergence upon rollback. Efficient, atomic, and low-latency checkpoint/restore mechanisms that handle both states jointly are lacking. Furthermore, leveraging idle inference time to mask checkpoint overhead remains an open challenge. These bottlenecks hinder both test-time reasoning and reinforcement learning training workflows.

DeltaBox introduces several key innovations:

• �� DeltaState OS abstraction: treats filesystem and process memory as a coupled transactional state, storing only incremental changes between checkpoints to minimize duplication.

• �� DeltaFS: an extension of Linux OverlayFS enabling runtime reconfiguration of writable layers. It dynamically freezes the current writable layer as read-only and inserts a new writable layer without unmounting, enabling copy-on-write semantics and fast checkpoint/rollback via simple layer switching.

• �� DeltaCR: builds on CRIU to perform asynchronous incremental dumps and creates frozen template processes via fork() to enable millisecond-level fast rollback. An asynchronous warm-up thread preemptively handles copy-on-write page faults to reduce recovery latency.

• �� Inference-masked checkpointing: exploits the LLM inference wait window to hide checkpoint overhead, maintaining seamless agent operation.

These innovations collectively overcome the limitations of full state duplication and enable high-frequency, low-latency, and atomic checkpoint/restore suitable for complex AI agent workloads.

• �� DeltaState abstraction: conceptualizes the agent’s sandbox state as a pair of filesystem and process memory states, managed incrementally.

• �� DeltaFS mechanism:
• Based on Linux OverlayFS, supports multiple overlay layers.
• At checkpoint, freezes the current writable layer (upper layer) into a read-only lower layer.
• Inserts a fresh writable layer atop the stack for subsequent writes.
• Employs per-inode generation counters for lazy redirection of open file descriptors across checkpoints.
• Reduces file updates to copy-on-write operations, enabling rollback by switching overlay layers.

• �� DeltaCR mechanism:
• Performs asynchronous incremental CRIU dumps to capture process memory changes.
• Simultaneously creates a frozen template process via fork() at quiescent points.
• On rollback, forks the template process for rapid restoration; if template is evicted, falls back to CRIU restore.
• Runs a background asynchronous warm-up thread to pre-fault writable memory pages, mitigating copy-on-write overhead.

• �� StateManager coordination:
• Host-side Sandbox Controller maintains a global snapshot tree with metadata.
• Guest-side State Daemon executes checkpoint and restore operations locally.
• Ensures atomic consistency between filesystem and process states.

• �� Inference-masked checkpointing:
• Checkpoints are triggered during LLM inference wait times.
• Network connections to LLM are isolated via a proxy daemon to maintain uninterrupted communication during checkpoint.

• �� Underlying storage:
• Uses XFS with reflink support for block-level copy-on-write, reducing write amplification.

Experiments were conducted on SWE-bench, a benchmark suite for AI code repair agents, and reinforcement learning micro-benchmarks. Baselines included Firecracker VM snapshots, Docker commit, and naive file copying methods. Metrics measured checkpoint and rollback latency, state management overhead as a fraction of total trajectory time, search node exploration counts, and success rates. Ablation studies isolated the impact of DeltaFS dynamic layer switching, DeltaCR template fork recovery, and asynchronous warm-up. Key hyperparameters such as template pool size and CRIU dump frequency were tuned. All experiments ran on Linux systems with XFS reflink-enabled filesystems, ensuring efficient copy-on-write support.

DeltaBox achieved an average checkpoint latency of 14 ms and rollback latency of 5 ms across SWE-bench workloads, outperforming Firecracker VM snapshots which typically incur 200 ms to seconds of latency. State management overhead was reduced from 47–77% in baseline coupled filesystem approaches to 3–6%, enabling AI agents to explore significantly more search nodes within fixed time budgets and improving task success rates. Ablation studies revealed that DeltaFS’s dynamic layer switching and DeltaCR’s template fork mechanisms contributed approximately 40% and 35% of the performance gains respectively. The asynchronous warm-up thread further reduced latency spikes caused by copy-on-write page faults. The system demonstrated stable, low-latency operation suitable for high-frequency checkpoint/restore demands.

DeltaBox is applicable to AI agents requiring high-frequency, fine-grained state exploration, such as automated code repair, software testing, desktop automation, and web navigation. Its millisecond-level checkpoint and rollback capabilities support deep tree search, large-scale parallel trajectory exploration, and reinforcement learning training with rapid sandbox cloning and restoration. The system operates transparently to agents, requiring no code modifications, facilitating integration into existing AI frameworks. Future extensions to distributed multi-node environments will enable scalable cloud-based intelligent services and large-scale automated systems.

DeltaBox relies on underlying filesystems with reflink support (e.g., XFS), limiting deployment flexibility in environments lacking such features. The bounded template process pool can lead to eviction under frequent checkpoint/rollback, causing fallback to slower CRIU restore paths and increased latency. The current design targets single-node microVM environments and does not address distributed multi-node state synchronization and consistency, restricting applicability in large-scale cluster settings.