Frequency-Conditioned Flow Matching for Vision-Language-Action Models
FreqFM improves VLA models by frequency conditioning, achieving a 9.3-point gain on LIBERO-Plus.
Key Findings
Methodology
FreqFM introduces frequency conditioning into Flow Matching by leveraging discrete cosine transform (DCT) to explicitly model action frequencies. It integrates spectrum-matched source distributions, power-normalized multi-task objectives, and frequency-constrained inference guidance without modifying the VLA backbone.
Key Results
- On LIBERO-Plus, FreqFM achieves a 9.3-point improvement over temporal baselines, excelling under Camera and Robot perturbations.
- On VLA-Arena, FreqFM improves total success rate by 4.2 points on π0.5, with strong performance in long-horizon tasks.
- In real-robot tasks, FreqFM shows significant gains in precision-critical tasks, such as a 23% improvement in folder insertion.
Significance
FreqFM addresses the challenge of frequency heterogeneity in robot action generation, offering a structured design space for frequency-aware modeling. It enhances robustness and generalization in complex scenarios, benefiting both academia and industry.
Technical Contribution
FreqFM pioneers frequency conditioning in Flow Matching, introducing spectrum-matched sources, power-normalized objectives, and frequency-guided inference to mitigate frequency imbalance and improve generation quality.
Novelty
FreqFM is the first to elevate frequency from an implicit property to an explicit conditioning dimension in Flow Matching, distinguishing it from prior temporal-domain approaches.
Limitations
- Performance depends on training data distribution, and mismatches in test distribution may degrade results.
- Frequency-guided inference introduces additional computational overhead.
- Limited gains in tasks with minimal high-frequency components.
Future Work
Future work could explore dynamic frequency conditioning to handle distribution shifts and extend FreqFM to broader multimodal generation tasks.
AI Executive Summary
Existing Flow Matching-based vision-language-action (VLA) models generate robot actions in temporal coordinates, failing to explicitly model the frequency heterogeneity of trajectories. FreqFM introduces frequency conditioning into Flow Matching, elevating frequency from an implicit property to an explicit modeling dimension, significantly improving performance in complex scenarios.
FreqFM's innovations include: 1) spectrum-matched source distributions in DCT frequency coordinates to reduce covariance mismatch; 2) power-normalized multi-task objectives to balance frequency contributions; and 3) frequency-constrained inference guidance to ensure frequency consistency. These enhancements integrate seamlessly into existing VLA backbones.
Experiments demonstrate a 9.3-point gain on LIBERO-Plus, a 4.2-point improvement on VLA-Arena, and strong performance in real-robot tasks. While reliant on training data distributions, FreqFM opens new avenues for frequency-aware design in multimodal generative models, with broad applicability and significant potential for future advancements.
Deep Analysis
Background
Robot action generation involves temporally correlated trajectories with highly uneven frequency energy distributions. Existing Flow Matching-based VLA models operate in temporal coordinates, failing to leverage frequency heterogeneity, limiting their performance in complex scenarios.
Core Problem
Current methods struggle with frequency imbalance, particularly the energy disparity between low and high frequencies, which hampers generation quality in long-horizon and perturbed tasks.
Innovation
FreqFM's key innovations include: 1) explicit frequency modeling via DCT; 2) spectrum-matched source distributions to reduce covariance mismatch; 3) power-normalized multi-task objectives to balance frequency contributions; 4) frequency-constrained inference guidance for consistent trajectory generation.
Methodology
- �� Apply DCT to transform actions into frequency coordinates.
- �� Construct spectrum-matched source distributions to align source and target frequencies.
- �� Optimize with power-normalized multi-task objectives to balance frequency contributions.
- �� Constrain inference guidance residuals to ensure frequency consistency.
Experiments
Experiments span LIBERO, LIBERO-Plus, and VLA-Arena benchmarks, evaluating performance under various perturbations. Real-robot tasks validate practical applicability, focusing on precision-critical scenarios.
Results
FreqFM achieves a 9.3-point gain on LIBERO-Plus, a 4.2-point improvement on VLA-Arena, and excels in real-robot tasks requiring fine alignment.
Applications
FreqFM is suited for complex robot tasks like industrial automation, service robots, and human-robot interaction, enhancing precision and robustness.
Limitations & Outlook
FreqFM relies on training distribution statistics, which may limit performance under distribution shifts. Inference introduces additional computational costs.
Plain Language Accessible to non-experts
Imagine a factory where low-frequency tasks are like moving large boxes, and high-frequency tasks are like assembling tiny parts. FreqFM acts as a smart manager, ensuring every task gets the right attention, making the factory efficient and precise.
ELI14 Explained like you're 14
Think of a robot playing a game where it has to pick up toys and put them in a box. FreqFM is like a cheat code that makes the robot super smart, helping it handle tricky levels easily! Cool, right?
Glossary
Flow Matching
A generative method that learns a conditional velocity field to map a simple distribution to a target distribution.
Used to generate robot action trajectories.
Discrete Cosine Transform (DCT)
A mathematical tool that transforms signals into frequency domain.
Used to explicitly model action frequencies.
Spectrum-Matched Source
A source distribution adjusted to match the target's frequency characteristics.
Reduces source-target covariance mismatch.
Power Normalization
Normalizing frequency components by their power to balance optimization contributions.
Used in multi-task optimization objectives.
Frequency Guidance Residual
Inference-time frequency adjustments to ensure consistent trajectory generation.
Constrained during inference for quality control.
Open Questions Unanswered questions from this research
- 1 How can frequency conditioning dynamically adapt to distribution shifts?
- 2 How can FreqFM be generalized to non-robotic multimodal generation tasks?
Applications
Immediate Applications
Industrial Robotics
Enhances precision and robustness in complex tasks like assembly and material handling.
Service Robots
Improves performance in dynamic environments, such as home assistants or healthcare robots.
Long-term Vision
Generalized Robotic Intelligence
Advances adaptability in open-world scenarios, paving the way for higher-level general intelligence.
Abstract
Robot actions are temporally correlated trajectories whose frequency components encode motion at different scales with highly non-uniform energy distributions. Yet Flow Matching--based vision-language-action (VLA) models typically generate actions in temporal coordinates, without explicitly modeling or systematically leveraging this frequency heterogeneity. We introduce \emph{FreqFM}, a frequency-conditioned Flow Matching framework for VLA models. It raises action frequency from an implicit trajectory property to an explicit conditioning dimension that spans the entire generation pipeline. Concretely, in DCT frequency coordinates, FreqFM constructs a spectrum-matched source distribution, adaptively balances the objective across frequencies, and constrains per-frequency guidance residuals using the corresponding reference transport scales. FreqFM integrates into existing Flow Matching action experts without changing the VLA backbone. Across LIBERO, LIBERO-Plus, and VLA-Arena, FreqFM consistently improves performance, including a 9.3-point gain on LIBERO-Plus, and further demonstrates its effectiveness on six real-robot tasks.