Time-domain Ad-hoc Array Speech Enhancement Using a Triple-path Network

Multi-channel speech enhancement has evolved from classical beamforming and spatial filtering to deep learning-based methods like DCRN and FasNet-TAC, which significantly improve speech clarity. However, these methods typically assume fixed array geometries, limiting their applicability in real-world scenarios where microphone placement is often arbitrary and unknown. Recent advances incorporate self-attention and graph neural networks to address array invariance, but challenges remain in effectively utilizing distant microphones and handling asynchronous signals. The need for a flexible, robust framework that can adapt to arbitrary microphone configurations motivates this research, aiming to bridge the gap between theoretical models and practical deployment.

The core challenge is designing a speech enhancement system that is invariant to the number, order, and placement of microphones, especially in ad-hoc arrays. Existing models rely on fixed geometries, making them unsuitable for unpredictable environments. Additionally, distant microphones often contribute weak signals, which are difficult to leverage effectively. Asynchronous signals and reverberation further complicate the problem. Addressing these issues requires novel architectures capable of flexible, robust multichannel processing that can adapt to real-world variability, ensuring consistent speech quality regardless of array configuration.

The primary innovations include:

1) Spatial Path with Self-Attention: Enables order- and quantity-invariant modeling of multi-microphone signals, crucial for ad-hoc arrays.

2) Dual-path ARN: Captures both local and global temporal dependencies within each channel, improving reverberation and noise suppression.

3) MIMO Architecture: Simultaneously enhances all microphone signals, including distant and weak sources.

4) Chunk-based Processing: Facilitates scalable, detailed temporal modeling, balancing local and global features.

These innovations collectively enable robust, flexible speech enhancement in unpredictable environments, surpassing prior fixed-array and graph-based methods.

• �� Convert multi-channel signals into frames and chunks for manageable processing.
• �� Spatial path: Apply self-attention across channels to fuse microphone signals, ensuring permutation invariance.
• �� Intra-chunk ARN: Process frames within each chunk to learn local temporal features.
• �� Inter-chunk ARN: Model global temporal dependencies across chunks.
• �� Fuse multi-path outputs for final enhancement.
• �� Use phase-constrained magnitude loss during training for stability.
• �� Train on simulated ad-hoc array data with random room and microphone configurations, optimizing hyperparameters such as chunk size, attention heads, and ARN layers.

Experiments utilized the DNS challenge 2020 dataset, simulating random room environments with arbitrary microphone placements and noise sources. The model was trained with 4-second utterances, varying microphone counts from 2 to 6, and evaluated on unseen configurations. Metrics included SI-SDR, STOI, and PESQ, compared against baselines like FasNet-TAC, DCRN, and fixed-array models. Ablation studies examined the impact of self-attention, chunk size, and number of microphones. The robustness was tested by sorting microphones based on proximity to the source, demonstrating the model’s ability to leverage distant microphones effectively.

TADRN achieved SI-SDR of 6.4dB on 6 channels, outperforming FasNet-TAC’s 1.0dB. STOI improved to 90.9%, PESQ to 2.68, indicating high speech quality. Experiments showed performance gains with increasing microphone number, especially when distant microphones were added. The model maintained strong performance even with randomly ordered microphones, confirming array invariance. Cross-configuration tests revealed superior generalization, and the model effectively utilized distant microphone signals, significantly reducing performance gaps across channels.

This approach is suitable for real-world scenarios like smart speakers, conference systems, and wearable devices, where microphone placement is unpredictable. It enables robust speech enhancement without prior knowledge of array geometry, simplifying deployment. The model’s ability to leverage distant microphones expands the usable spatial coverage, improving clarity in large or complex environments. Future integration with edge computing could enable real-time, high-quality speech enhancement in diverse applications.

High computational complexity and training data requirements limit immediate real-time deployment. Performance may degrade in highly reverberant or extremely low SNR environments, especially with very distant microphones. Further work is needed to optimize model efficiency, reduce latency, and extend robustness to multi-speaker and multi-language scenarios. Additionally, real-world testing and hardware implementation remain future challenges.