Context Consistency between Training and Testing in Simultaneous Machine Translation

The evolution of SiMT has been driven by the need for low-latency, high-quality translation in multilingual settings. Early methods like wait-k provided simple yet effective strategies, but training often relied solely on cross-entropy loss, neglecting the discrepancy between training and testing contexts. Recent works introduced various strategies to balance latency and quality, yet the core issue of context mismatch persisted. Ma et al. (2018) first identified the counterintuitive phenomenon where models trained with consistent context underperform when tested with mismatched context settings. Subsequent efforts, such as multi-path sampling (Elbayad et al., 2020), improved robustness but did not fully address the underlying causes. This paper builds on these insights, analyzing the correlation between training loss and translation quality, and exposing the impact of exposure bias, to propose a comprehensive solution.

The key challenge in SiMT is the inconsistency between training and testing phases regarding context usage. Fixed context strategies during training do not reflect the dynamic nature of real-time translation, causing models to perform poorly in practice. Exposure bias exacerbates this issue, as models are trained on idealized data but tested on their own predictions, leading to accumulation of errors. This mismatch results in lower translation quality and higher latency, especially in low-latency scenarios where the model must balance speed and accuracy. Addressing these issues requires a training paradigm that aligns training conditions with testing realities, ensuring models learn to adapt to varying contexts.

The core innovation is the bi-objective context consistency training framework, which simultaneously optimizes BLEU and latency metrics. It introduces candidate generation during training, exposing the model to multiple predictions and reducing exposure bias. The approach employs a tunable parameter γ to balance the two objectives dynamically, enabling the model to adapt to different latency requirements. Unlike previous methods focusing solely on cross-entropy or fixed strategies, this framework systematically aligns training and testing conditions, leading to improved robustness and performance. The integration of candidate diversity and multi-objective optimization represents a significant advancement in SiMT training methodology.

• �� Define a bi-objective loss combining BLEU and AL metrics, with adjustable γ. • Generate multiple translation candidates during training using beam search and sampling, to expose the model to diverse outputs. • For each candidate, compute the BLEU score and AL, incorporating these into the loss function. • During training, optimize the model parameters to minimize the combined loss, balancing translation quality and latency. • Use a two-step training process: initial cross-entropy training followed by fine-tuning with the bi-objective loss. • Adjust γ to find the optimal trade-off based on validation performance, ensuring the model learns to handle varying contexts effectively.

Experiments were conducted on IWSLT14 De→En, IWSLT15 Vi→En, and WMT15 De→En datasets. Baselines included traditional consistency training and multi-path sampling approaches. The models used Transformer architectures with different configurations. Evaluation metrics included BLEU for translation quality and AL for latency. Hyperparameters such as candidate number, γ, and decoding strategies were tuned via grid search. Ablation studies examined the impact of candidate diversity and multi-objective optimization. Results showed consistent improvements across datasets, with BLEU gains exceeding 3 points and latency reductions, especially at low latency settings. The robustness was validated across different policies and language pairs.

The proposed method achieved an average BLEU increase of over 3 points across datasets, with improvements of over 5 points in low-latency scenarios (k=1). Latency was reduced significantly, with AL decreasing by approximately 4-6 units. The γ parameter tuning revealed optimal performance at 0.4, balancing quality and delay. Ablation studies confirmed that candidate generation and bi-objective optimization are critical to performance gains. The approach demonstrated strong generalization across policies, including wait-k and wait-info, and across languages, indicating broad applicability.

This framework can be directly applied to real-time multilingual conference translation, enabling more accurate and timely communication. It also benefits cross-lingual live broadcasting, international negotiations, and emergency response systems, where low latency and high accuracy are crucial. Long-term, integrating this approach with adaptive policies and multi-modal inputs could revolutionize intelligent translation systems, making them more flexible, robust, and context-aware, suitable for diverse and complex scenarios.

The increased computational cost due to candidate generation and multi-objective optimization may limit scalability in resource-constrained environments. Fine-tuning γ for different datasets and policies adds complexity. The approach's effectiveness in extremely long sentences or very low-resource languages remains to be validated. Future work should focus on reducing computational overhead, automating parameter tuning, and extending to multi-modal and multi-task settings.