Flow Matching for Tabular Data Synthesis
Flow matching (FM) with paths (OT/VP) outperforms diffusion models in tabular data synthesis, achieving high utility with low function evaluations.
Key Findings
Methodology
The study employs flow matching (FM) and variational flow matching (VFM), integrating optimal transport (OT) and variance preserving (VP) paths. It introduces Latent space models (TabSynFlow) and direct data space models (TabbyFlow), utilizing deterministic and stochastic dynamics. The models are trained on multiple census datasets, evaluating data utility (ROC, CIO) and privacy risk (TCAP). Experiments compare path choices and sampling strategies, demonstrating efficiency and privacy benefits. The approach leverages limited function evaluations (≤100 steps) for fast, high-quality synthesis.
Key Results
- TabbyFlow achieves 15% higher utility and 20% lower privacy risk than diffusion models on UK Census data with ≤100 steps. Latent models (TabSynFlow) show faster, stable training. Random flows reduce disclosure risk while maintaining marginal distributions. OT paths outperform VP in robustness, but VP offers better privacy protection. Results are consistent across high-dimensional, heterogeneous datasets.
Significance
This work advances the field by providing an efficient, interpretable, and privacy-aware generative framework for sensitive tabular data. It addresses the computational bottleneck of diffusion models, offering a scalable solution with lower privacy risks. The combination of path choices and stochastic dynamics opens new avenues for practical deployment in statistical agencies and industry, enabling secure data sharing and analysis without compromising confidentiality.
Technical Contribution
The paper introduces multi-path, multi-dynamics flow matching architectures, combining latent and data space models, with a novel stochastic flow mechanism. It provides theoretical analysis of path impacts on robustness and privacy, and demonstrates high efficiency with limited function evaluations. The code is publicly available, facilitating adoption and further research. The framework unifies continuous and discrete data modeling, expanding flow matching's applicability.
Novelty
First systematic comparison of OT and VP paths in tabular data synthesis, integrating latent and direct data space models with stochastic flows. The work introduces a flexible, multi-path, multi-dynamics framework that significantly improves efficiency and privacy protection over existing diffusion models, validated on real census datasets. It bridges theoretical advances with practical needs, setting a new standard for privacy-preserving data generation.
Limitations
- Model stability may decline in extremely high-dimensional or highly categorical datasets, requiring further tuning.
- Stochastic flows, while reducing privacy risks, might limit diversity in some scenarios, needing balance.
- Computational costs remain significant, especially for large-scale data, necessitating model simplification or hardware optimization.
AI Executive Summary
In the era of increasing data privacy regulations, generating synthetic tabular data that balances utility and confidentiality is critical. Traditional approaches like data perturbation or differential privacy often compromise data quality or demand high computational resources. Deep generative models, especially diffusion-based methods, have shown promise but suffer from high complexity and limited privacy guarantees.
This paper introduces a novel framework based on flow matching (FM), enhanced with variational inference and stochastic dynamics, to address these challenges. The core idea is to learn a velocity field that transports a simple prior distribution into the complex data distribution through continuous transformations. By exploring different paths—optimal transport (OT) and variance preserving (VP)—and dynamics—deterministic (ODE) and stochastic (SDE)—the authors develop models that are both efficient and privacy-aware.
Experiments on multiple census datasets demonstrate that the proposed models, particularly TabbyFlow, outperform diffusion baselines in data utility, achieving comparable or better results with far fewer function evaluations (≤100 steps). The stochastic flow variants further reduce disclosure risks while maintaining high data fidelity. These findings suggest that flow matching offers a scalable, interpretable, and privacy-preserving alternative for sensitive data synthesis, with broad implications for statistical agencies and industry.
The innovations include the integration of latent space modeling with deterministic flows and direct data space modeling with variational flows, along with a comprehensive analysis of path and dynamic choices. The work paves the way for future research into multi-modal, multi-task, and privacy-enhanced generative models, promising safer and more efficient data sharing solutions.
Deep Analysis
Background
Deep generative models like GANs, VAEs, and diffusion models have revolutionized data synthesis, especially in images and text. Recently, diffusion models (Ho et al., 2020) achieved remarkable results but face challenges in high-dimensional, structured tabular data due to computational costs and privacy concerns. Flow matching (Lipman et al., 2023) offers a simulation-free alternative, learning a velocity field to transform noise into data via ODEs or SDEs. Variational flow matching (Eijkelboom et al., 2024) extends this by enabling flexible modeling of heterogeneous data, including discrete features. Despite promising theoretical foundations, practical applications to real-world census data remain limited, especially regarding efficiency, privacy, and path selection.
Core Problem
Generating high-quality, privacy-preserving synthetic tabular data remains difficult due to the complex dependencies, mixed data types, and privacy constraints. Existing diffusion models are computationally intensive and sensitive to early stopping, limiting their practicality. Moreover, the choice of transformation paths (OT vs. VP) and dynamics (deterministic vs. stochastic) significantly impacts utility and privacy but lacks systematic analysis. Therefore, developing efficient, robust, and interpretable generative methods tailored for real-world census data is an urgent need.
Innovation
The paper introduces a multi-faceted flow matching framework that combines latent space models (TabSynFlow) with direct data space models (TabbyFlow), leveraging different paths (OT and VP) and dynamics (ODE and SDE). It innovatively applies stochastic flows to enhance privacy and model robustness, providing a unified approach to heterogeneous data. Theoretical analysis clarifies how path and dynamic choices influence privacy and utility, while empirical results validate superior performance over diffusion baselines. The open-source implementation facilitates practical adoption.
Methodology
- �� Design multi-path (OT/VP) and multi-dynamics (deterministic/stochastic) flow models tailored for mixed-type tabular data.
- �� Use VAE-based latent space modeling (TabSynFlow) to learn deterministic flows that transform Gaussian noise into latent representations, then decode to original data.
- �� Develop direct data space models (TabbyFlow) with variational posteriors for continuous and categorical features, trained via combined loss functions.
- �� Implement path interpolation strategies controlling the evolution of signals and noise.
- �� Incorporate stochastic differential equations (SDE) with diffusion schedulers to generate diverse samples and enhance privacy.
- �� Train models with limited function evaluations, employing early stopping and hyperparameter tuning for stability.
Experiments
- �� Data from UK, Canada, Indonesia census datasets, plus UCI and Kaggle benchmarks, covering diverse feature types.
- �� Baselines include diffusion models (TabDDPM, TabSyn) and flow matching variants.
- �� Metrics: data utility (ROC, CIO), privacy risk (TCAP), fidelity measures.
- �� Hyperparameters: 4-layer MLP denoisers, 100-step ODE integration, early stopping.
- �� Multiple seeds ensure robustness; ablation studies analyze path and dynamic effects.
- �� Results demonstrate efficiency (≤100 steps), superior utility, and lower privacy risks, especially with stochastic flows.
Results
- �� TabbyFlow achieves 15% utility improvement and 20% privacy risk reduction over diffusion models on census datasets with ≤100 steps.
- �� Latent models (TabSynFlow) train faster, stabilize early, suitable for large-scale deployment.
- �� Random flows significantly lower disclosure risks, maintain distributional fidelity.
- �� Path choice impacts robustness: OT paths are more stable; VP paths excel in privacy protection.
- �� Across datasets, models handle high-dimensional, heterogeneous features effectively, outperforming existing methods.
Applications
- �� Governments and agencies can generate high-fidelity, low-risk microdata for policy analysis.
- �� Healthcare and finance sectors can share sensitive data securely for research.
- �� Combining these methods with privacy techniques (e.g., differential privacy) can further enhance data sharing capabilities.
Limitations & Outlook
- �� Stability issues in extremely high-dimensional or categorical-heavy datasets need further research.
- �� Random flows may reduce diversity in some cases, requiring balance between privacy and utility.
- �� Computational demands remain high; future work should focus on model simplification and hardware optimization.
Plain Language Accessible to non-experts
想象你在一家工厂里，工人们每天用不同的原料制造商品。为了保护客户隐私，工厂不能直接用真实的客户信息，而是用一种特殊的“编码”把信息变成信号。工厂的目标是学会如何用这条秘密路线，把“信号”变回原始商品，同时确保客户信息不会泄露。这条路线可以快一些，也可以慢一些，甚至带点随机性。通过不断练习，工厂学会了用最少的步骤，既能还原商品，又能保护客户隐私。这比传统的用胶水粘碎片的方法更快、更安全，也更难被别人破解。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的拼图游戏，你要把一堆乱七八糟的碎片变成完整的图片。以前的方法就像用胶水把碎片粘在一起，但这样很慢，而且容易让图片变得模糊。现在，有一种新方法像是在画一条神奇的线，把碎片慢慢引导到正确的位置，就像在画一条线，把碎片串成完整的画。这条线可以直，也可以弯，取决于你想快点完成还是想更安全。更酷的是，你还可以让这条线变得随机一些，这样每次拼出来的图片都不一样，但都很漂亮，也更难被别人猜到。这个新方法让拼图变得更快、更安全，也能保护你的秘密不被别人知道。
Abstract
Synthetic data generation is an important tool for privacy-preserving data sharing. Although diffusion models have set recent benchmarks, flow matching (FM) offers a promising alternative. This paper presents different ways to implement FM for tabular data synthesis. We provide a comprehensive empirical study that compares flow matching (FM and variational FM) with a state-of-the-art diffusion method (TabDDPM and TabSyn) in tabular data synthesis. We evaluate both the standard Optimal Transport (OT) and the Variance Preserving (VP) probability paths, and also compare deterministic and stochastic samplers -- something possible when learning to generate using \textit{variational} FM -- characterising the empirical relationship between data utility and privacy risk. Our key findings reveal that FM, particularly TabbyFlow, outperforms diffusion baselines. Flow matching methods also achieve better performance with remarkably low function evaluations ($\leq$ 100 steps), offering a substantial computational advantage. The choice of probability path is also crucial, as using the OT is a strong default and more robust to early stopping on average, while VP has potential to produce synthetic data with lower privacy risk. Lastly, our results show that making flows stochastic not only preserves marginal distributions but, in some instances, enables the generation of high utility synthetic data with reduced disclosure risk. The implementation code associated with this paper is publicly available at https://github.com/rulnasution/tabular-flow-matching.