TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulation
TapSampling combines Action-VAE candidate sampling with task-progress verification, raising real-robot success from 78.3% to 83.3%.
Key Findings
Methodology
TapSampling is policy-agnostic. Action-VAE encodes policy-generated action chunks into low-dimensional Gaussian posteriors, mixes several posteriors, then samples and decodes candidates. A task-progress verifier conditions on vision, language, and a query action to predict its progress change. Expert trajectories provide automatic labels: forward subsequences are positive and reversed subsequences negative. Candidates below a threshold are removed; the remainder are selected through progress-weighted action fusion.
Key Results
- On CALVIN ABC→D, average success length increased from 2.41 to 2.58 for Diffusion Policy, 3.30 to 3.51 for OpenVLA, and 4.39 to 4.46 for VPP. OpenVLA’s five-step success rates improved by 1.1–6.4 percentage points.
- On LIBERO-Long, π0.5 improved from 96.8% to 98.0% average success. On a real Franka Research 3, average success increased from 78.3% to 83.3%, including substantial gains on unseen tasks.
- Learned-posterior sampling is about 5× faster than repeated policy sampling. With 16 candidates, TapSampling verification is approximately 12× faster than RoboMonkey because the visual backbone runs once and action heads evaluate candidates in parallel.
Significance
The paper introduces inference-time computation as a practical complement to scaling data and model size. It addresses a persistent deployment problem: stochastic diffusion and autoregressive policies can succeed or fail under nearly identical conditions because single-shot inference has no correction mechanism. The progress-conditioned score is also interpretable, unlike many generic preference or state-count scores. Since the wrapper does not require policy fine-tuning, it transfers across Diffusion Policy, OpenVLA, VPP, and π0.5.
Technical Contribution
The main contributions are correlation-aware Action-VAE sampling and action-conditioned task-progress verification. Unlike independent Gaussian perturbations, the mixture posterior preserves structure across action dimensions. Unlike value functions that score only the current state, the verifier estimates the expected consequence of a specific low-level action. Built on VLA-Adapter and Qwen2.5-0.5B, it shares visual hidden states across candidates and uses a lightweight action head for parallel regression.
Novelty
The novelty lies in integrating efficient candidate generation with semantically grounded verification, rather than merely repeating policy inference. Positive and negative supervision is extracted from trajectory order and reversed actions, avoiding manual preference labels or synthetic rollouts. The learned posterior also replaces many expensive policy calls, offering a practical trade-off among distribution fidelity, latency, and interpretability.
Limitations
- The label p_i=i/t assumes approximately linear progress and treats reversed actions as negative. This can be wrong for non-monotonic tasks, recoverable mistakes, waiting behaviors, and branching multi-stage tasks.
- Candidate quality remains bounded by the initial policy samples. The verifier also depends on visual understanding and training-distribution coverage, so occlusion, novel objects, and long-horizon causal effects may cause misranking.
- More candidates increase verification cost, while score-weighted averaging may be unsuitable for discrete, contact-rich, or strongly multimodal actions.
Future Work
Promising directions include phase-aware or nonlinear progress labels, real failure trajectories, calibrated uncertainty, and adaptive candidate counts. Longer-term work could verify entire action sequences, perform closed-loop search over extended horizons, transfer across robots and environments, and combine the verifier with safety constraints, model-predictive control, or reinforcement learning.
AI Executive Summary
Robotic policies are becoming stronger through larger models and datasets, yet stochastic diffusion and autoregressive policies remain unstable: the same scene may produce success in one rollout and failure in another. Single-shot inference commits to one action without a deployment-time correction mechanism. TapSampling treats inference computation as a new axis for improving reliability.
Its first component, Action-VAE, compresses a small set of policy actions into a learned low-dimensional posterior, then generates many plausible candidates by latent sampling and decoding. Its second component, a task-progress verifier, predicts how each candidate will affect completion using visual observations, language instructions, and the query action. Training labels come from trajectory order: forward actions increase progress, while reversed subsequences are assigned negative progress. At inference, bad candidates are discarded and the rest are weighted by predicted progress.
The gains are consistent. On CALVIN ABC→D, average success length rose from 2.41 to 2.58 for Diffusion Policy, 3.30 to 3.51 for OpenVLA, and 4.39 to 4.46 for VPP. On LIBERO-Long, π0.5 increased from 96.8% to 98.0%. In real Franka Research 3 experiments, average success rose from 78.3% to 83.3%. Learned-posterior sampling was about five times faster than repeated policy sampling, and 16-candidate verification was about 12 times faster than RoboMonkey. Remaining concerns include linear progress assumptions, dependence on initial policy quality, and the difficulty of averaging actions in contact-rich multimodal tasks.
Deep Analysis
Background
Vision-language models, video diffusion models, and action experts have advanced generalist manipulation. Representative systems include Diffusion Policy, OpenVLA, VPP, and π0.5, using diffusion, autoregressive token prediction, video features, or flow matching. Yet stochastic inference still creates deployment variance. LLMs can use self-consistency and image generation can use auxiliary scorers, but low-level robot actions lack a broadly applicable, semantically grounded evaluator.
Core Problem
Given state s, instruction l, and policy π, the system must choose an action likely to advance the task. Repeatedly querying π increases latency; independent Gaussian sampling ignores correlations within action chunks; state-only value functions cannot model the consequence of a particular low-level action. The central challenge is therefore to generate distribution-faithful candidates, rank them by physical task impact, and remain computationally practical.
Innovation
First, Action-VAE learns a low-dimensional posterior over action chunks and forms a mixture from a few policy samples, allowing arbitrary candidate counts. Second, the Task-Progress Verifier converts action selection into progress-change regression, giving scores an explicit meaning: harmful, stable, or accelerating. Third, the wrapper is policy-agnostic and leaves Diffusion Policy, OpenVLA, and VPP unchanged. Shared visual features and batched action heads make verification efficient.
Methodology
- �� Train Action-VAE: the encoder produces q_E(z|a)=N(μ_E(a),diag(σ_E²(a))); the decoder reconstructs â, optimized with L_VAE=L_rec+λ_KL L_KL.
- �� Sample candidates: obtain N actions from π(a|s,l), compute q_mix(z|Aπ)=1/N∑q_E(z|a_i), sample M latent codes, and decode them.
- �� Build labels: for trajectory length t, set p_i=i/t; a forward subsequence receives k/t, while its reversed counterpart receives −k/t.
- �� Train verifier: VLA-Adapter uses Qwen2.5-0.5B; its action head regresses Δp with L_tap=|V(s,l,a)−Δp|_1.
- �� Select actions: discard scores below a threshold, then fuse remaining candidates using progress-weighted averaging. Visual hidden states are computed once and replicated for batched scoring.
Experiments
Experiments cover CALVIN ABC→D, LIBERO-Long, and a real 7-DoF Franka Research 3. Baselines include Diffusion Policy, OpenVLA, VPP, and π0.5; metrics are success rate, CALVIN average success length, and sampling or verification latency. Real tasks are Knock Down, Pick and Place, and Stack, using third-view and wrist cameras. One hundred successful demonstrations per task were collected for π0 and TapSampling fine-tuning. Policy Sampling, Gaussian Sampling, and Learned Posterior Sampling were compared across candidate counts.
Results
On CALVIN, average length improved from 2.41 to 2.58 for Diffusion Policy, 3.30 to 3.51 for OpenVLA, and 4.39 to 4.46 for VPP; OpenVLA’s five-step success rose from 42.4% to 48.8%. LIBERO-Long π0.5 increased from 96.8% to 98.0%. Real average success increased from 78.3% to 83.3%, while unseen Stack rose from 75.0% to 85.0%. Action-VAE sampling was about 5× faster, and 16-candidate verification about 12× faster than RoboMonkey.
Applications
The method is suited to deployment-time enhancement of stochastic generalist policies in warehouse picking, sorting, placement, stacking, and household manipulation. Users need not retrain the main policy; they need an action interface, demonstrations for the verifier, and compatible visual-language inputs. Small initial sample sets suit latency-sensitive applications, while progress thresholds can reject risky actions in safety-sensitive operations.
Limitations & Outlook
Linear progress labels derived from expert trajectory length do not fully represent recovery, waiting, or parallel subgoals. Reversing an action is not always physically harmful, and contact dynamics, discrete gripper states, and multimodal actions may make weighted averaging invalid. Verification still becomes more expensive as candidate count grows. Future systems should add phase-aware value models, uncertainty estimates, failure data, and sequence-level planning.
Plain Language Accessible to non-experts
Imagine a restaurant kitchen. An ordinary robot policy is one cook: it sees an order and prepares one dish. Most of the time the dish is fine, but occasionally it adds too much salt or puts the food on the wrong plate. TapSampling gives the cook several quick attempts. Action-VAE is like a prep specialist who has learned which ingredients and movements usually belong together. It does not mix ingredients randomly; it produces several versions that still look like sensible dishes.
A task-progress verifier then acts as the head waiter. It sees the order, the current table, and every candidate dish, and predicts which choice moves service closer to completion. A negative score means the choice may create trouble; a small positive score means it is reliable; a larger positive score suggests faster progress. The system discards clearly bad options and combines the promising ones before acting.
The important point is that the original cook does not need retraining. Whether the cook is based on diffusion or step-by-step generation, this extra “try several, then inspect” layer can be attached. But it cannot invent a skill the original policy never proposes, and complicated physical tasks may not tolerate simply blending several movements.
ELI14 Explained like you're 14
Picture a robot playing a game where the mission is “put the red block in the box.” A normal controller may try one movement and commit to it. Because its decision process contains randomness, the exact same screen can lead to a win one time and a fail the next—like rolling dice before every move.
TapSampling gives the robot several trial ideas. Action-VAE studies how expert robots moved their arms before, then quickly creates more sensible possibilities from a few original ideas. It does not just wiggle randomly: it remembers that when the arm moves left, the height and gripper position usually need to change together.
The verifier is like a game referee watching a progress bar. It sees the picture, the mission, and each proposed movement, then predicts whether that movement moves the robot forward or backward. A negative number means “bad idea,” a small positive number means “safe,” and a larger positive number means “maybe faster.” The robot removes bad choices and uses the better ones.
The results are pretty cool! OpenVLA’s CALVIN average length rose from 3.30 to 3.51, π0.5’s LIBERO-Long success rose from 96.8% to 98.0%, and the real robot improved from 78.3% to 83.3%. But the trick still depends on the original controller having at least one good idea. For tricky contact tasks, mixing movements can also create a new mistake!
Glossary
Action-VAE
A variational autoencoder that compresses actions into a low-dimensional random representation and reconstructs them. Its reconstruction and KL terms learn a structured action distribution.
It generates many efficient candidate action chunks from a few policy samples.
Task-Progress Verifier
A model that predicts how a candidate action changes task completion. Its output is a progress change rather than an opaque preference score.
It filters and ranks candidate actions at inference time.
Action chunk
A short sequence of robot controls executed across several time steps. Its dimensions and temporal elements are often correlated.
Action-VAE models the chunk jointly rather than sampling each dimension independently.
CALVIN ABC→D
A language-conditioned long-horizon manipulation benchmark trained in environments A, B, and C and evaluated in unseen D. Average success length measures consecutive task completion.
It tests zero-shot generalization of TapSampling.
LIBERO-Long
A challenging suite for lifelong robot decision-making and long-horizon manipulation. It reports task success over multiple tasks.
It evaluates TapSampling with π0.5.
Open Questions Unanswered questions from this research
- 1 Should progress labels be causal and phase-aware rather than linear in time? Non-monotonic tasks require supervision for recovery, waiting, and reversible errors.
- 2 Verifier calibration across robots, cameras, objects, and environments remains unclear. Large failure datasets, uncertainty estimates, and systematic out-of-distribution tests are needed.
- 3 Weighted action fusion may fail for discrete or contact-rich control. How candidate selection should jointly enforce physical and safety constraints remains unresolved.
Applications
Immediate Applications
Warehouse picking and placement
Teams already using OpenVLA, Diffusion Policy, or VPP can attach Action-VAE and the verifier without changing the main policy. The system can reject collision-prone or poorly aligned candidates for sorting, packing, and shelf organization.
Household organization
A home robot can generate several grasp-and-place possibilities for the same instruction and choose the one predicted to advance the task. It requires demonstrations, compatible vision, and an action interface; the paper suggests improved robustness on unseen tasks.
Long-term Vision
Explainable general-purpose robots
Future systems could combine progress, risk, and uncertainty into a deployment dashboard. Before acting, a robot could explain whether a movement is expected to help or hinder the task and request human confirmation when confidence is low.
Abstract
Existing embodied control research demonstrates remarkable performance improvements by scaling training data and model size. We instead explore inference-time strategy as an alternative axis. Non-deterministic generative models, such as diffusion and autoregressive models, have been widely adopted in the field of embodied control. However, the single-shot inference paradigm limits their performance. In this paper, we propose \textbf{TapSampling}, a plug-and-play framework for inference-time sampling. First, we introduce an Action-VAE that represents actions in a low-dimensional latent space by mapping policy-generated initial actions into a compressed posterior distribution, from which any number of latent samples can be drawn and decoded into candidate actions that approximate the true action distribution. Second, we formulate action verification as task-progress outcome prediction, using the intrinsic sequential structure of robotic datasets to train a semantically grounded verifier for interpretable action selection. Furthermore, TapSampling is a policy-agnostic framework. Extensive experiments in both simulated and real-world environments demonstrate that our method substantially improves multiple generalist policies without further policy finetuning. Code and models are available at the project page.