V-Link: Recovering Lost Visual Representations in Action DiT for Vision-Language-Action Models
V-Link restores visual cues in Action DiT, improving GR00T N1.6 by 31.2% on LIBERO-Plus and 18.8% on RoboTwin 2.0.
Key Findings
Methodology
V-Link builds on GR00T N1.6 by introducing learnable Spatial Queries and Semantic Queries inside the VLM. A tailored causal mask lets them separately aggregate geometric and semantic evidence. Training-only depth and segmentation heads specialize the queries. In Action DiT, Semantic Queries complement image tokens through parallel cross-attention, while Spatial Queries enter a dedicated geometric-conditioning branch. Auxiliary heads and annotations are removed at inference.
Key Results
- On LIBERO, V-Link reaches 99.3% average success versus 97.4% for GR00T N1.6, a +1.9-point gain; LIBERO-Long improves by 5.1 points to 98.0%.
- On zero-shot LIBERO-Plus distribution shifts, V-Link scores 75.0% versus 43.8% for the baseline, a +31.2-point gain. Improvements are especially large under noise (+60.4%), language changes (+47.3%), and lighting changes (+30.6%).
- On six RoboTwin 2.0 tasks, V-Link achieves 56.8% versus 38.0%, or +18.8 points. On AGIBOT A3 Ultra, power-on/off success is 98%/94%, with only 1.58 ms extra latency.
Significance
The work reframes a major VLA bottleneck: strong perception in the VLM is insufficient if the action expert cannot access it. This explains why dual-system policies may recognize objects yet fail at precise placement or contact. V-Link offers a lightweight, deployment-friendly solution that improves robustness and fine-grained manipulation without requiring depth or segmentation inference at test time. The result is relevant both to multimodal representation research and to industrial robot-policy engineering.
Technical Contribution
The paper makes representation accessibility measurable through frozen-feature probes. GR00T Action DiT features degrade depth MAE from 0.015 in VLM features to 0.071 and segmentation mIoU from 0.665 to 0.290. V-Link then combines auxiliary representation specialization with asymmetric feature routing. It preserves GR00T’s alternating-attention schedule, injects semantic context and geometry through separate paths, and uses a learnable gate for semantic fusion. This is an architectural repair of VL-to-action transfer rather than simply adding a larger vision encoder.
Novelty
Most related methods, including Spatial Forcing, EVO-0, GEAR-VLA, and SpatialVLA, primarily enrich VLM representations; PointVLA and ABot-M0 directly add 3D features or foundation-model outputs. V-Link instead identifies visual information loss after VLM encoding and explicitly recovers it inside the action expert. Its combination of query specialization, asymmetric injection, and frozen-feature diagnostics is the fundamental novelty.
Limitations
- The evaluation centers on GR00T N1.6, three simulation benchmarks, and only two real-world A3 Ultra tasks, so generalization across VLA backbones, robot morphologies, and long-horizon behaviors remains unproven.
- Training requires depth and semantic supervision; real experiments use Lingbot-Depth, GroundingDINO, and SAM3 pseudo-labels. Sensor dropout, reflective surfaces, and noisy masks may therefore affect query quality.
Future Work
Future research should develop architecture-independent accessibility metrics, compare intermediate-layer transfers, and test V-Link on dynamic scenes, long-horizon tasks, and heterogeneous robots. Reducing dependence on depth sensors and pseudo-labels is important, as is online adaptation under changing cameras and embodiment. Larger real-world datasets could determine whether the gains persist beyond curated benchmark distributions.
AI Executive Summary
Vision-language-action models promise robots that can interpret images and instructions while producing continuous motor commands. Yet dual-system designs such as GR00T N1.6 pass only final-layer VLM features to Action DiT. V-Link shows that this interface is lossy: frozen-feature probes report depth MAE worsening from 0.015 to 0.071 and segmentation mIoU falling from 0.665 to 0.290. The action expert can exploit easy semantic shortcuts while neglecting geometry needed for accurate manipulation.
V-Link inserts two learnable token groups into the VLM. Spatial Queries capture depth and geometric structure; Semantic Queries capture task-relevant meaning. Training-only depth and segmentation heads force this specialization. During action generation, Semantic Queries are fused in parallel with image tokens, while Spatial Queries provide a subsequent, dedicated geometric cross-attention path. The architecture therefore restores access rather than merely increasing visual capacity.
The gains are substantial. V-Link reaches 99.3% on LIBERO, 75.0% on zero-shot LIBERO-Plus, and 56.8% on RoboTwin 2.0—improvements of 1.9, 31.2, and 18.8 points over GR00T N1.6. On A3 Ultra, power-on and power-off success reaches 98% and 94%, respectively, with only 1.58 ms additional latency. The broader lesson is that multimodal intelligence depends not only on what a model represents, but also on whether downstream control modules can actually retrieve it.
Deep Analysis
Background
VLA research includes unified systems such as UniVLA and Cosmos Policy and dual-system designs such as GR00T N1.6. Other work improves perception with SpatialVLA, GEAR-VLA, EVO-0, VGGT, and Spatial Forcing, while PointVLA and ABot-M0 inject 3D features into action modules. These methods largely ask whether richer visual features can be learned. V-Link asks a complementary question: are those features still accessible after transfer to the action expert?
Core Problem
GR00T transfers final-layer VLM features to Action DiT, where action-only supervision may favor recognizable semantics over depth, distance, pose, and contact geometry. This creates a representation accessibility gap. Diagnostic heads show depth MAE of 0.071 and segmentation mIoU of 0.290 for GR00T Action DiT, compared with 0.015 and 0.665 for VLM features. The gap is especially damaging for fine-grained manipulation.
Innovation
V-Link contributes three linked ideas. First, Spatial and Semantic Queries decouple geometry from semantics. Second, lightweight depth and segmentation heads explicitly specialize them, using L=Lact+0.1Ldepth+0.002Lseg. Third, asymmetric routing gives semantics and geometry different roles: Semantic Queries complement image tokens, while Spatial Queries form a dedicated geometric branch. Unlike direct 3D-feature injection, the method repairs the VL-to-action interface itself.
Methodology
- �� The VLM input is E=[Et;Eo;Ed;Es], containing language, multiview image, Spatial Query, and Semantic Query tokens.
- �� A causal mask Mq isolates the query sets while preserving native multimodal computation; final RMSNorm outputs E′d and E′s.
- �� The depth head uses LN, Linear(2048,256), GELU, two Transformer encoders, and a convolutional decoder. Its loss combines smooth-L1, absolute-relative, and hard-patch terms with coefficients 0.1 and 0.5.
- �� The segmentation head uses cross-entropy plus soft Dice, weighted by 0.5.
- �� At Action DiT image-cross-attention layers, E′o and E′s are fused first; E′d then conditions the fused features through spatial cross-attention. A learnable gate gl controls semantic strength.
- �� Injection occurs only at ℓ=4k+2 in the 32-layer AlternateVLDiT, preserving the original schedule.
Experiments
Models start from GR00T N1.6 3B, freeze vision and language backbones, and train rank-128 LoRA adapters in BF16 on four NVIDIA H100 GPUs. AdamW uses learning rate 1e-4, weight decay 1e-5, cosine decay, and 5% warm-up. LIBERO uses 80K steps; RoboTwin 2.0 uses 120K. Evaluation covers LIBERO, zero-shot LIBERO-Plus, six RoboTwin tasks, and two A3 Ultra tasks. Ablations test Spatial Queries, Semantic Queries, auxiliary supervision, injection, and query-grid size.
Results
V-Link achieves 99.3% on LIBERO, including 98.0% on LIBERO-Long, 5.1 points above the baseline. On LIBERO-Plus it reaches 75.0%, versus 43.8% for GR00T; under noise it scores 87.8% versus 27.4%. RoboTwin 2.0 rises from 38.0% to 56.8%. Ablations yield 51.2% with Spatial Queries, 47.8% with Semantic Queries, and 56.8% with both. Removing task supervision or injection reduces performance to 40.3% or 39.8%.
Applications
The method targets spatially precise grasping, placement, pressing, switching, and dual-arm interaction. Existing GR00T-style systems can adopt it without running depth or segmentation heads at deployment. The A3 Ultra results—98% power-on and 94% power-off—suggest practical value for humanoid panel interaction. A deployment pipeline still needs calibrated cameras, suitable action representations, and reliable training-time depth or pseudo-label data.
Limitations & Outlook
V-Link remains dependent on training-time depth and semantic supervision, and real-world pseudo-labels may be unreliable under reflections, missing RGB-D, or occlusion. The benchmark set is relatively small, while LIBERO is close to saturation and does not strongly test long-horizon planning. More query tokens improve capacity but increase cost: a 3×5×5 grid is best at 56.8%, whereas 3×6×6 slightly falls to 56.5%. Broader cross-embodiment and online-adaptation studies are needed.
Plain Language Accessible to non-experts
Imagine a restaurant where one chef studies the scene and reads the customer’s order, while another chef performs rapid cutting and serving. In the original system, the first chef hands over only a final note. The note may say “prepare the cup,” but omit how far the cup is from the hand, how high it sits, or where the handle points. The second chef can understand the object yet still make a clumsy move.
V-Link adds two assistants. One measures locations, distances, and shapes; the other identifies objects and their importance to the order. During training, the first assistant must draw a distance map and the second must color different objects. This forces them to learn different jobs instead of repeating the same label. At serving time, the assistants leave, but their organized notes remain available to the fast chef.
That simple change makes placement, pressing, and turning more reliable. Success rises from 43.8% to 75.0% on LIBERO-Plus and from 38.0% to 56.8% on RoboTwin 2.0, with only 1.58 milliseconds of extra delay. The central lesson is practical: information is useful only when it reaches the person who must act in a form that can be used immediately.
ELI14 Explained like you're 14
Think of a robot like a video-game character with two teammates. The first teammate is great at looking around and reading the mission: “Put the bread in the pan.” The second teammate controls the arms. In the old setup, the first teammate sends a short summary. It may know that the bread exists, but the arm controller may not know exactly how far away it is or where to move first. Result? The robot recognizes the goal but misses the grip!
V-Link adds two special power-ups. The Spatial Query is like a mini-map showing distance, height, and position. The Semantic Query is like a mission tracker saying which object matters and what it means. During practice, the robot must use the mini-map to make a depth picture and the mission tracker to color different objects. So each power-up learns a different skill instead of both guessing the same thing.
Then the robot passes these organized clues to its action controller. The semantic clue helps it understand the target, and the spatial clue tells it how to move in 3D. The extra teachers disappear during the real game, so the robot does not need to run them forever.
The scores jump: LIBERO reaches 99.3%, LIBERO-Plus reaches 75.0% instead of 43.8%, and RoboTwin 2.0 rises to 56.8% from 38.0%. On a real humanoid robot, power-on succeeds 98% of the time and power-off 94%. Pretty cool! But the system still needs special practice data, and researchers must test whether it works in more robots, messier rooms, and much longer missions.
Glossary
Vision-Language-Action (VLA)
A model that connects visual observations, language instructions, and robot actions. Technically, it maps multimodal context into continuous or chunked control trajectories.
The paper studies information transfer inside dual-system VLAs.
Action DiT
A diffusion-style Transformer action expert that predicts or denoises robot action sequences. It provides high-frequency control after receiving state and visual context.
V-Link aims to restore visual information inside this module.
Spatial Query
A learnable token set trained to encode geometry, depth, and spatial structure. It supplies explicit geometric conditioning for action generation.
It is supervised by the depth-estimation head and injected through spatial cross-attention.
Semantic Query
A learnable token set specialized for object identity, task relevance, and 2D semantic structure. It complements the original VLM image tokens.
It is trained with segmentation supervision and fused through a parallel attention path.
Flow Matching
A generative training approach that learns a vector field transforming noise into target data. In plain terms, it teaches the model how to move gradually toward a valid action trajectory.
The action loss Lact is the main policy-generation objective.
Open Questions Unanswered questions from this research
- 1 Depth MAE and segmentation mIoU measure representation accessibility indirectly. It remains unclear how causally these probe scores predict manipulation success, so better intervention-based diagnostics are needed.
- 2 Training depends on depth and pseudo-segmentation labels. Robust query learning without depth sensors, under severe occlusion, reflection, and dynamic motion, remains unresolved.
Applications
Immediate Applications
Fine-grained tabletop manipulation
Robot developers can add V-Link to GR00T N1.6 for grasping, placement, pressing, and switching. Training requires multiview depth and semantic labels, but deployment removes auxiliary heads. The expected benefit is stronger control under language, lighting, and sensor-noise shifts.
Humanoid panel interaction
A3 Ultra-like humanoids can use V-Link for autonomous power-on, power-off, and contact-rich control. Its 1.58 ms latency overhead is small for real-time systems, provided cameras, action spaces, and training pseudo-labels are properly calibrated.
Long-term Vision
General spatially grounded robot policies
If validated across embodiments and long-horizon tasks, V-Link could become a reusable interface for converting high-level multimodal understanding into reliable 3D action. This may support household, warehouse, and service robots, although large-scale real-world validation remains necessary.
Abstract
Vision-language-action (VLA) models provide a scalable path toward generalist robotic manipulation by integrating visual perception, language understanding, and continuous action control. However, we reveal a critical limitation of VLA architectures: the action expert has limited access to the 3D geometric and 2D semantic information available in VLM features. This accessibility gap weakens perceptual grounding and limits performance on fine-grained robotic manipulation. To address this issue, we propose V-Link, which explicitly recovers visual representations during the vision-language (VL) to action (A) feature transfer. Specifically, V-Link learns complementary Spatial and Semantic Query representations within the VLM and injects them into Action DiT through asymmetric pathways. Semantic Queries complement the original VLM image tokens, whereas Spatial Queries provide dedicated geometric conditioning for spatially grounded action generation. Across LIBERO, LIBERO-Plus, and RoboTwin 2.0, our V-Link improves the average success rate over base model GR00T N1.6 by +1.9%, +31.2%, and +18.8%, respectively. On the AGIBOT A3 Ultra, V-Link further achieves gains of +20% and +24% on two real-world humanoid tasks.