VLA Foundry: A Unified Framework for Training Vision-Language-Action Models

In recent years, the advancement of robotic foundation models has been rapid, with many systems demonstrating capabilities that seemed out of reach just a few years ago. As the frontier moves faster, the tooling required to support rigorous research must keep pace. Many high-impact questions—about data scaling, backbone pretraining, and the interplay between robotics and non-robotics data—require both scale (compute, data, etc.) and modular algorithmic infrastructure that allows users full control over different parts of the model and training pipeline. However, most existing codebases have either not been extensively tested at scale or are largely focused on model releases, limiting research flexibility. At the same time, data scarcity remains a fundamental bottleneck in robotics. Robot interaction data is severely constrained relative to data used for language and vision models, especially in diversity and in signal density per token. Despite this data disparity, most open-source VLA frameworks focus narrowly on the action training stage, treating the upstream data recipe as fixed or out-of-scope. Such separation is problematic: data decisions made during LLM and VLM pretraining have direct consequences for downstream robotics performance. Exploring the design space requires a framework that treats the entire pipeline, from pretraining to policy learning, as a single, controllable system.

Existing open-source VLA frameworks often focus on the action training stage, stitching together incompatible pretraining pipelines. This approach leads to limitations in research flexibility, as researchers cannot conduct both from-scratch training and pretrained backbone initialization within the same codebase. Additionally, data scarcity remains a fundamental bottleneck in robotics, with robot interaction data severely constrained relative to data used for language and vision models, especially in diversity and in signal density per token.

The core innovations of VLA Foundry lie in its modularity and composability, allowing users to swap architectures, data pipelines, and training recipes through simple command-line or YAML changes. It supports pretrained backbones from Hugging Face and offers scalable distributed training, supporting multi-node, multi-GPU runs. By offering a shared data-loading and training stack, researchers can co-train across modalities, mix datasets, and prototype new architectures without stitching together disparate tools. VLA Foundry is the first to unify the training of LLM, VLM, and VLA in a single codebase, providing end-to-end control from language pretraining to action-expert fine-tuning.

VLA Foundry is designed around end-to-end control of the embodied-model pipeline: the same training loop, data abstractions, and configuration interface extend from language pretraining to vision-language training and action learning. • Modularity and Composability: Models, data pipelines, encoders, and loss handlers are instantiated by name from a YAML-based configuration system. • Scalable Distributed Training: Supports multi-node, multi-GPU runs with automatic gradient accumulation, mixed precision, and checkpoint synchronization. • Evaluation: Supports evaluation on lbm_eval_oss, the open-source release of the benchmark, using the high fidelity Drake physics engine to model the robots and scene dynamics. • Statistical Analysis: Provides rigorous statistical analysis via STEP to compare success rates of multiple policies.

The experimental design includes evaluating two types of models on the LBM Eval simulator: one trained fully from scratch through the LLM→VLM→VLA pipeline, and the other built on the pretrained Qwen3-VL backbone. The evaluation includes performance comparisons on 16 simulated tasks, varying in complexity and manipulation modes. The experiments also include ablation studies on multi-task vs. single-task training, as well as sim-only and real-only subsets.

On the LBM Eval simulator, the fully open from-scratch model performs on par with prior closed-source work in nominal evaluation settings, while the model using the Qwen3-VL backbone excels in multi-task tabletop manipulation policies, outperforming the baseline by 20 percentage points. The multi-task model trained using VLA Foundry significantly outperforms the prior closed-source multi-task model on 16 simulated tasks. The experimental results demonstrate that using a stronger VLM backbone can significantly enhance VLA performance.

VLA Foundry's application scenarios include the development and optimization of robotic manipulation policies, particularly in multi-task tabletop manipulation. It provides researchers with a flexible and scalable tool for exploring and optimizing the training of Vision-Language-Action models. The framework can also be applied to other domains requiring multimodal data integration and cross-modal training, such as autonomous driving and human-computer interaction.

Despite significant advances in unifying the training process, VLA Foundry still faces challenges in handling data scarcity in robotic interaction data. Additionally, using pretrained backbones may limit the model's flexibility and adaptability. The complexity of the framework may pose a learning curve for novice users. Future research directions include further optimizing training efficiency, exploring more diverse datasets and tasks, and improving the framework's user-friendliness and accessibility.