Fabricator or dynamic translator?

Machine translation technology has made significant progress over the past decades, evolving from rule-based methods to modern Neural Machine Translation (NMT) models. NMT models achieve efficient conversion from source language to target language through an encoder-decoder architecture. However, with the advent of Large Language Models (LLMs), the translation field faces new challenges and opportunities. LLMs are known for their powerful generative capabilities but often generate excessive content during translation, a phenomenon known as overgeneration. Overgeneration not only affects translation accuracy but can also lead to misunderstandings and confusion. While previous research has explored the issue of neurobabble in NMT models, the overgeneration phenomenon in LLMs is more complex, involving self-explanations and unnecessary expansions.

The issue of LLMs generating excessive content during translation is becoming increasingly prominent. This overgeneration phenomenon not only affects translation accuracy but can also lead to misunderstandings and confusion. Unlike traditional NMT models, the overgeneration phenomenon in LLMs is more complex, involving self-explanations and unnecessary expansions. Effectively detecting and categorizing these overgeneration phenomena to improve translation quality is the core problem of current research. Solving this problem is significant for improving the practicality and reliability of machine translation.

The core innovation of this study lies in proposing a strategy that combines MTQE models and alignment detection methods to identify and categorize different types of overgeneration phenomena. Specifically, the MTQE model is a multilingual encoder regression model fine-tuned to predict translation quality, while the alignment detection method uses alignment as a proxy for attention weights to detect unaligned text chunks. This combined strategy not only improves detection accuracy but also provides a new methodological foundation for future translation quality assessment. Additionally, the study is the first to systematically explore the phenomenon of overgeneration in LLMs during translation, particularly in identifying minimally detached overgenerations.

• �� Use MTQE model for translation quality prediction: The model is based on the XLM-R large model, fine-tuned for multilingual translation tasks.
• �� Alignment detection method: Use the AwesomeAlign tool for alignment detection to identify unaligned text chunks.
• �� Dataset selection: Use WMT25 AOC task data and internally developed datasets for experimental validation.
• �� Combined strategy: Combine MTQE models and alignment detection methods to improve the accuracy of overgeneration detection.
• �� Result analysis: Validate the effectiveness of the combined strategy through experimental results, particularly in handling minimally detached overgenerations.

The experimental design includes using multiple datasets to validate the proposed detection strategies. The primary datasets include WMT25 AOC task data and internally developed datasets. These datasets cover various language pairs, such as English-Chinese, English-Russian, and English-Japanese. The baselines used in the experiments include traditional NMT models and translations generated by LLMs. Key evaluation metrics include detection accuracy, recall, and precision. Additionally, the study conducted ablation studies to assess the impact of different strategy combinations on detection performance.

Experimental results show that the combined strategy performs well in detecting overgeneration phenomena. On the WMT25 AOC dataset, the combined strategy can detect overgeneration phenomena with 95% accuracy. On internal datasets, the combined strategy performs well in detecting minimally detached overgenerations, achieving a recall rate of 77%, although the precision is low at 22%. These results indicate that the combined strategy has significant advantages in handling complex translation generation issues, particularly in identifying minimally detached overgenerations.

The findings of this study have significant implications for various application scenarios. Firstly, it can be used to improve the quality of machine translation systems, especially in scenarios requiring high-precision translations. Secondly, the strategy can help translation service providers better identify and handle overgeneration phenomena in translations, thereby improving customer satisfaction. Additionally, these detection strategies can be applied to other natural language processing tasks, such as text generation and summarization, to improve the accuracy and relevance of generated content.

Despite the excellent performance of the proposed combined strategy in detecting overgeneration phenomena, there are some limitations. Firstly, the alignment detection method may produce a high false positive rate when dealing with very short overgenerations, as these phrases may be poorly aligned with the source text. Secondly, the MTQE model has low precision in detecting minimally detached overgenerations, which may require further optimization. Additionally, the applicability of the current strategy across different language pairs has not been fully validated. Future research directions include exploring the applicability of these strategies across more language pairs.