Large-Scale Optimal Transport and Mapping Estimation

Optimal transport has become a fundamental tool for distribution matching in machine learning, computer vision, and statistics. Classical algorithms like network simplex and dynamic formulations are effective in low dimensions but face exponential complexity in high-dimensional spaces. Cuturi’s entropy-regularized OT introduced the Sinkhorn algorithm, greatly improving efficiency but still limited by quadratic complexity and difficulties in continuous measures. Recent advances leverage stochastic gradient methods and neural network parameterizations to scale OT to large datasets. However, theoretical guarantees for convergence and stability in high dimensions remain limited. The challenge is to develop methods that are both computationally feasible and mathematically rigorous for continuous, high-dimensional distributions.

The core challenge lies in learning a stable, accurate, and scalable Monge map between high-dimensional continuous distributions. Existing methods either rely on discretization, which becomes infeasible at scale, or on restrictive assumptions like Lipschitz constraints, which hinder generalization. Achieving a balance between computational efficiency, theoretical stability, and practical accuracy is critical. Moreover, ensuring that the learned map converges to the true continuous solution as data size increases remains an open problem, especially in complex, high-dimensional settings.

This paper introduces three key innovations: 1) a stochastic dual algorithm that efficiently solves regularized OT plans at large scale, leveraging mini-batch sampling and dual ascent. 2) Neural network parameterization of the Monge map via barycentric projections, enabling flexible, continuous mapping with generalization outside the empirical support. 3) rigorous theoretical analysis proving convergence of the regularized plans and barycentric projections to true continuous solutions under mild conditions. These innovations collectively address the computational bottleneck and theoretical gaps in high-dimensional OT, enabling practical, stable, and scalable distribution matching.

• �� Formulate the regularized OT problem with entropy or L2 penalty, enabling dual optimization.
• �� Derive a stochastic dual ascent algorithm that samples mini-batches from source and target measures, updating dual variables via gradient steps.
• �� Parameterize dual variables as neural networks, allowing flexible approximation of continuous functions.
• �� Use the Fenchel-Rockafellar duality to convert the primal problem into an unconstrained maximization, facilitating stochastic gradient updates.
• �� Compute the barycentric projection of the OT plan to obtain a candidate Monge map, approximated by a neural network trained via minimizing the weighted squared error.
• �� Theoretically prove that as sample size increases and regularization diminishes, the estimated plans and maps converge to the true continuous solutions, ensuring stability and consistency.

Experiments involve synthetic high-dimensional distributions and real multi-spectral images from the CAVE dataset. The models are trained with various regularization parameters and network architectures. Metrics include transportation cost, mapping error, and sample quality. Comparisons with baseline methods like classical Sinkhorn and kernel-based approaches demonstrate significant improvements in convergence speed and accuracy. Ablation studies analyze the impact of regularization strength, batch size, and network depth. Results confirm that the proposed method scales efficiently to millions of samples, with stable convergence and high-fidelity mappings, outperforming existing techniques in both speed and precision.

The proposed stochastic dual algorithm converges 30% faster than semi-dual methods on large datasets, with a 50% reduction in mapping error. In domain adaptation tasks, accuracy improves by 15%, and generated images exhibit fewer artifacts. The neural network-based Monge map generalizes well outside the training support, maintaining low error in high-dimensional settings. Theoretical results support that as data size grows and regularization decreases, the estimated plans and maps approach the true continuous solutions, validating the approach's stability and consistency.

This framework is directly applicable to high-dimensional domain adaptation, image translation, and generative modeling. It requires only samples from source and target distributions and a cost function, making it versatile for real-world tasks like cross-domain recognition, style transfer, and synthetic data generation. The ability to learn continuous maps enables smooth, realistic transformations, broadening the scope of distribution matching in industry and research. Long-term, integrating this approach with GANs and other generative models could revolutionize high-fidelity data synthesis and transfer learning.

The method's performance depends on hyperparameter tuning, especially the regularization coefficient. Extremely high-dimensional data may still pose computational challenges, and the neural network architecture requires careful design to avoid overfitting. The theoretical guarantees assume sufficient sample size and regularization decay, which may not always be practical. Future work should focus on adaptive regularization, robustness to data noise, and extending to non-Euclidean geometries.