Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning

The evolution of statistical learning has increasingly emphasized robustness against distributional shifts. Early methods relied on divergence measures like φ-divergences (e.g., KL divergence) to define uncertainty sets, but these often failed to capture tail risks and support deviations effectively. The advent of optimal transport theory, especially Wasserstein distance, provided a geometrically meaningful metric that accounts for sample support and tail behavior. Recent research, including Esfahani and Kuhn (2018), formalized Wasserstein DRO, but practical algorithms and theoretical guarantees in high-dimensional settings remain active research areas. The challenge lies in balancing computational tractability with robustness guarantees, especially as data complexity grows.

The core challenge is to develop a scalable, theoretically sound framework that captures distributional uncertainty in high-dimensional spaces. Traditional empirical risk minimization often suffers from overfitting and poor out-of-sample performance due to sample bias. Existing robust methods either lack computational efficiency or fail to provide tight risk bounds. The difficulty is compounded when the true distribution is unknown and must be estimated from limited data, which introduces estimation errors. Effectively quantifying and controlling these errors within a mathematically rigorous yet computationally feasible model remains a key bottleneck.

This work introduces several innovations: 1) employing Wasserstein distance to define the ambiguity set, which naturally captures support deviations and tail risks; 2) utilizing Kantorovich duality to convert the infinite-dimensional worst-case risk into a finite convex program, enabling efficient computation; 3) integrating regularization terms inspired by the dual formulation to improve generalization and mitigate overfitting. These innovations collectively extend the theoretical landscape of DRO, offering a practical algorithmic solution with strong risk bounds and convergence guarantees. The approach also bridges optimal transport theory with statistical learning, providing new insights into robustness and regularization.

• �� Construct the empirical distribution bPN from training data.
• �� Define the ambiguity set Bε,p(bPN) as all distributions within Wasserstein radius ε.
• �� Formulate the worst-case risk Rε,p(bPN, ℓ) as the supremum over this set.
• �� Use Kantorovich duality to express this supremum as a finite convex optimization problem involving dual variables ϕ and ψ.
• �� Incorporate regularization terms in the dual formulation to enhance stability and generalization.
• �� Solve the resulting convex program using standard solvers like CVX or Sinkhorn algorithm for efficient distance computation.
• �� Derive risk bounds and convergence properties to ensure theoretical robustness.
• �� Validate through experiments on synthetic and real datasets, tuning the radius parameter for optimal trade-offs.

Experiments involve synthetic high-dimensional datasets and real financial risk data. Baselines include standard ERM and φ-divergence-based DRO. Metrics focus on risk reduction, out-of-sample accuracy, and extreme event prediction. Hyperparameters such as Wasserstein radius and regularization strength are tuned via cross-validation. Ablation studies compare the impact of dual regularization and different Wasserstein orders. Results demonstrate that the proposed method maintains low risk under distribution shifts and outperforms baselines in robustness and predictive accuracy, especially with limited samples.

The Wasserstein DRO reduces out-of-sample risk by approximately 15% in high-dimensional synthetic scenarios and enhances extreme event prediction accuracy by 10% in financial datasets. The risk bounds derived theoretically align with empirical observations, confirming the model’s robustness. Sensitivity analysis indicates that optimal performance occurs within a specific radius range, balancing bias and variance. Ablation studies show that dual regularization contributes significantly to stability and generalization, especially in data-scarce regimes.

Applicable in financial risk management, supply chain resilience, and energy systems, especially where data is limited or distributional shifts are common. The framework can be integrated into existing decision pipelines, providing robust risk assessments and optimized policies that are less sensitive to model misspecification. It is particularly useful for stress testing, extreme event prediction, and adaptive control in uncertain environments.

The approach relies on the Lipschitz continuity of loss functions, limiting applicability to certain classes of problems. Computational costs increase with data dimension and complexity, especially for non-discrete distributions. Selecting the Wasserstein radius remains a challenge, requiring careful tuning. Extending guarantees to non-Lipschitz losses and dynamic environments are ongoing research directions.