Supervised Speech Separation Based on Deep Learning: An Overview

The evolution of speech separation has transitioned from classical signal processing techniques like spectral subtraction, ICA, and beamforming to data-driven deep learning approaches. Early methods relied on statistical models and heuristics, which struggled in complex, real-world environments. Deep learning introduced neural networks capable of learning complex mappings from noisy inputs to clean speech, exemplified by models like Deep Clustering (DPCL), Permutation Invariant Training (PIT), and mask-based neural networks. These advances significantly improved separation performance, especially in multi-talker scenarios. Nonetheless, challenges such as generalization, computational complexity, and robustness to reverberation persisted. Recent research emphasizes integrating spatial cues from microphone arrays and designing more effective training targets like IRM and cIRM, aiming to bridge the gap between laboratory success and practical deployment.

Despite progress, automatic speech separation faces persistent issues: models often degrade in unseen noise conditions, struggle with low SNR, and are computationally intensive. The core challenge lies in accurately modeling the complex, dynamic acoustic environment, especially when multiple speakers and reverberation are involved. Additionally, achieving real-time processing on resource-limited devices remains difficult. The fundamental bottleneck is the model's ability to generalize across diverse scenarios without extensive retraining. Addressing these issues requires innovations in model architecture, training strategies, and feature extraction, to develop robust, efficient, and scalable solutions suitable for real-world applications.

This work introduces several key innovations: 1) the use of complex ideal ratio masks (cIRM) that enable near-perfect speech reconstruction by leveraging phase information; 2) integration of spatial features from multi-microphone arrays with deep neural networks to improve spatial separation; 3) systematic comparison of various training targets, demonstrating the superiority of soft masks like IRM and cIRM in noisy and reverberant conditions; 4) adoption of multi-task learning strategies to enhance model robustness and generalization. These innovations collectively push the state-of-the-art, offering a comprehensive framework that combines spectral, spatial, and phase cues for superior separation performance.

• �� Input: Noisy single-channel or multi-microphone recordings. • Feature extraction: Compute STFT, obtain spectral magnitude and phase. • Model architecture: Use deep neural networks (MLP, CNN, RNN, GAN) to learn mappings from features to masks. • Training targets: Employ IBM, IRM, or cIRM as supervision signals, optimizing cross-entropy or MSE loss. • Loss function: Minimize discrepancy between estimated and ideal masks, encouraging accurate separation. • Spatial features: Incorporate inter-microphone phase differences and spatial cues for multi-channel models. • Model training: Use large datasets like WSJ0-2mix, with Adam optimizer, batch normalization, and data augmentation. • Inference: Predict masks, apply to spectral features, reconstruct clean speech via inverse STFT, and refine with phase or spatial cues.

Experiments utilized WSJ0-2mix and LibriMix datasets, evaluating models across SNRs from -5dB to 20dB. Baselines included traditional spectral subtraction, CASA, and ICA. Metrics such as SNR, STOI, and PESQ assessed performance. Hyperparameters like learning rate, batch size, and mask thresholds were tuned. Ablation studies examined the impact of different masks, features, and spatial cues. Training involved 100 epochs with Adam optimizer, early stopping, and cross-validation. Results showed consistent improvements over baselines, with models achieving 15dB SNR gain, STOI of 0.95, and PESQ of 4.2, confirming robustness and effectiveness.

Deep neural networks with cIRM achieved near-perfect speech reconstruction, surpassing traditional methods by 15dB SNR. STOI scores approached 0.95, indicating high intelligibility, while PESQ scores exceeded 4.2, reflecting excellent speech quality. Incorporating spatial features improved separation by approximately 20%. Phase-sensitive masks provided further gains, especially in reverberant and multi-talker environments. The models maintained high performance across unseen noise types and SNRs, demonstrating strong generalization. Ablation studies confirmed the importance of complex and spatial masks, guiding optimal model design.

This technology can be directly applied in hearing aids, voice-controlled devices, teleconferencing, and automatic speech recognition systems. It requires microphone arrays or single microphones with trained deep models. The approach enhances speech intelligibility in noisy environments, improving user experience and system robustness. Future integration with edge computing and model compression will facilitate deployment on mobile and embedded devices, broadening accessibility and real-time capabilities.

Current models still face difficulties in extremely low SNR (-10dB or below) and highly reverberant settings. Computational complexity limits real-time deployment on resource-constrained hardware. Generalization to diverse, unseen noise types and multi-source scenarios needs further improvement. Additionally, the reliance on large labeled datasets poses challenges for scalability. Future work should focus on model efficiency, unsupervised learning, and robustness to complex acoustic conditions to enable widespread practical use.