HybridSB-MoE: Dual-Domain Schrödinger Bridges with Scene-Adaptive Expert Routing for Speech Enhancement
HybridSB-MoE combines dual-domain Schrödinger bridges with scene-adaptive expert routing, outperforming baselines on VoiceBank+DEMAND with theoretical guarantees.
Key Findings
Methodology
This approach integrates asymmetric uncertainty fusion, combining a heterogeneous MoE in the spectral domain with a stochastic Schrödinger bridge in the time domain. The spectral MoE leverages expert disagreement to capture epistemic uncertainty, while the waveform bridge models aleatoric variance via stochastic differential equations. An adaptive fusion module learns to weight each pathway based on their respective uncertainties, guided by path consistency and trajectory regularization losses. The spectral experts are architecturally diverse, including low-rank, wide receptive field, and harmonic basis types, enabling the system to identify which inductive bias fails. The training guarantees that small K-step sampling errors are bounded in 2-Wasserstein distance, ensuring reliable low-step inference.
Key Results
- On VoiceBank+DEMAND, HybridSB-MoE achieves PESQ scores above 3.2 and STOI over 95% at various step budgets, surpassing diffusion and Schrödinger bridge baselines. The model maintains high performance with only 8 inference steps, demonstrating theoretical bounds in practice.
- Ablation studies confirm that expert heterogeneity, path regularization, and asymmetric fusion are essential for optimal performance. The system adapts effectively to different noise types, especially non-stationary and harmonic interference, validating the scene-adaptive design.
- The model's uncertainty calibration, measured by Expected Calibration Error (ECE), shows well-aligned confidence estimates, enabling input-dependent pathway routing that improves robustness and interpretability.
Significance
This work advances the field of generative speech enhancement by bridging the gap between spectral and waveform methods through a unified, theoretically grounded framework. The asymmetric uncertainty fusion enables dynamic, input-specific pathway selection, addressing long-standing challenges of noise variability and phase preservation. The theoretical bounds on small-step sampling errors link training objectives directly to inference quality, paving the way for efficient, reliable real-time applications. The heterogeneous expert design enhances interpretability and scene adaptability, making this approach highly relevant for practical deployment in diverse acoustic environments. Overall, it offers a new paradigm for combining multiple generative priors with principled uncertainty modeling, with broad implications for speech processing and beyond.
Technical Contribution
The paper introduces a novel dual-domain framework that unites heterogeneous spectral MoE with stochastic Schrödinger bridges, underpinned by a rigorous discretization bound (Theorem 1). The asymmetric fusion mechanism exploits expert disagreement and bridge variance to adaptively route inputs, a first in speech enhancement. The training incorporates path consistency and trajectory regularization, providing explicit bounds on K-step sampling errors in 2-Wasserstein distance, directly linking training objectives to inference guarantees. The architecture's heterogeneity and the theoretical analysis together enable efficient, scene-adaptive, low-step sampling with calibrated uncertainty, representing a significant step forward in generative speech models.
Novelty
This is the first work to integrate heterogeneous expert architectures with Schrödinger bridge-based generative modeling for speech enhancement, employing asymmetric uncertainty fusion. The combination of path consistency regularization with a discretization bound for small K inference is novel, providing formal guarantees previously absent in generative SE. Unlike prior methods that rely on fixed-weight ensemble or single-domain models, this approach dynamically adapts to input conditions, leveraging expert disagreement and bridge variance for robust, scene-aware enhancement, marking a new direction in the field.
Limitations
- Despite theoretical guarantees, the model's performance may degrade in unseen or highly adverse noise conditions due to limited expert diversity or training data coverage.
- Training complexity and computational cost are higher than traditional methods, especially due to multiple experts and regularization components, which may hinder real-time deployment.
- The assumptions underlying the discretization bound, such as Lipschitz continuity and boundedness, may not hold perfectly in practice, affecting the robustness of the guarantees.
Future Work
Future research will explore more scalable expert architectures, reduce computational overhead, and extend the framework to multi-modal inputs. Enhancing robustness to unseen noise types and further tightening theoretical bounds for real-world scenarios are also key directions. Additionally, integrating user-specific scene adaptation and exploring unsupervised training paradigms could broaden applicability.
AI Executive Summary
Deep Dive
Plain Language Accessible to non-experts
Imagine you're in a busy kitchen with two helpers: one is great at picking the freshest ingredients (frequency domain expert), and the other is excellent at cooking them just right (time domain bridge). Sometimes, the ingredient picker might mistakenly choose old vegetables, and the cook might burn the dish if not careful. To make the best meal, you decide to listen to both helpers but also pay attention to how much you trust each. If they disagree a lot, you might rely more on the cooking helper; if the ingredients are questionable, you trust the picker more. This way, you combine their advice smartly, ensuring the dish turns out delicious every time. This is similar to how the system in the paper dynamically chooses between different pathways based on their confidence, leading to better, more natural-sounding speech enhancement.
ELI14 Explained like you're 14
Think of you and your two friends trying to fix a noisy room. One friend is good at finding the loudest sources of noise (like a noisy fan), and the other is good at cleaning up the sound after it's recorded. Sometimes, the noise is tricky, and each friend might give you different advice. Instead of trusting just one, you listen to both and decide which advice to follow based on how confident each is. If the fan noise is really loud, you trust your friend who knows how to handle loud sounds more. If the recording is fuzzy, you rely on the one who can clean up the sound better. By doing this, you get the clearest possible speech, just like the system in the paper combines different methods based on their confidence to make speech sound better and more natural.
Abstract
Generative speech enhancement faces three gaps: spectral models capture harmonic structure but often disrupt phase, waveform models preserve phase but miss harmonics, and Schrödinger Bridges (SB) shorten transport from noise to clean speech but leave inference cost only loosely tied to training. We propose HybridSB-MoE, a dual-domain framework that fills these gaps through three contributions unified by a single asymmetric design principle. (i) Asymmetric uncertainty fusion: The spectral path captures epistemic uncertainty via expert disagreement, while the waveform bridge models aleatoric variance through stochastic dynamics. We fuse them asymmetrically, allowing the mixing weight to adapt to distinct error regimes rather than average predictions. (ii) Heterogeneous MoE with top-k=2 routing across five distinct architectural archetypes, where architectural diversity makes the epistemic signal indicate which inductive bias fails rather than small perturbations among similar experts. (iii) Discretization bound (Theorem 1): path-consistency and trajectory regularizers together bound the K-step bridge sampling error in 2-Wasserstein distance at rate K-alpha, making small-K inference an objective-level guarantee rather than an empirical claim. On VoiceBank+DEMAND, HybridSB-MoE outperforms diffusion- and SB-based baselines at their step budgets while remaining competitive with consistency-distilled few-step methods.