Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog
Proposes Multi2WOZ dataset and conversational pretraining framework TOD-XLMR, significantly improving cross-lingual transfer for task-oriented dialogue.
Key Findings
Methodology
The study constructs Multi2WOZ, a high-quality, aligned multilingual dataset covering Chinese, German, Arabic, and Russian, derived from MultiWOZ via automatic translation and manual correction. Introduces TOD-XLMR, a multilingual pretrained language model based on XLM-R, further specialized through multi-task training (MLM, TLM, RS) on both monolingual and parallel corpora, including OpenSubtitles. The framework employs intermediate training steps for target language adaptation, leveraging cross-lingual response selection and translation language modeling to enhance transfer. Systematic evaluation on DST and RR tasks demonstrates the effectiveness of target language specialization, especially in few-shot scenarios.
Key Results
- In zero-shot transfer, the average dialog state tracking accuracy across four target languages improved from 1.33% (baseline XLM-R) to 5.39% with target language specialization, and response retrieval accuracy increased from 2.7% to 37.4%. Few-shot experiments showed that with only a small number of target language dialogs, models could approach full-data performance, confirming sample efficiency gains.
- Multi-task pretraining combining MLM, TLM, and RS objectives on parallel OpenSubtitles data yielded the best transfer results, outperforming single-task approaches. The results reveal that conversational target language training significantly boosts cross-lingual robustness, especially for low-resource languages.
- Performance varies with linguistic proximity to English, with German and Russian showing better transfer than Arabic and Chinese, aligning with language similarity. The framework demonstrates that targeted pretraining and specialization are key to effective multilingual dialogue systems.
Significance
This work addresses the critical bottleneck of data scarcity in multilingual task-oriented dialogue systems, providing a reliable benchmark dataset and a novel pretraining paradigm. It advances the state-of-the-art in cross-lingual transfer, especially for low-resource languages, by emphasizing dialogue-specific adaptation. The approach reduces reliance on large annotated datasets, making multilingual deployment more feasible. Its implications extend to real-world applications like multilingual customer service, travel assistants, and cross-cultural communication tools, fostering more inclusive and accessible AI systems.
Technical Contribution
The paper introduces Multi2WOZ, a large, aligned multilingual dialogue dataset, and the TOD-XLMR framework, which integrates multiple pretraining objectives tailored for dialogue understanding. The core innovation lies in target language-specific intermediate training using parallel corpora, especially the novel cross-lingual response selection task, which enhances transferability. The systematic comparison of zero-shot and few-shot strategies provides a comprehensive understanding of how dialogue-specific pretraining influences cross-lingual performance, setting a new benchmark for future research.
Novelty
This is the first work to create a fully aligned, multi-domain multilingual dialogue dataset covering Chinese, German, Arabic, and Russian, with high-quality manual correction. It also pioneers the integration of multi-task dialogue pretraining with target language-specific adaptation, demonstrating significant improvements in low-resource transfer scenarios. Unlike previous datasets relying solely on translation heuristics, Multi2WOZ ensures data quality and comparability across languages, enabling rigorous cross-lingual evaluation.
Limitations
- Despite high-quality manual correction, automatic translation introduces some errors, especially in idiomatic expressions, which may affect model training and evaluation.
- Training the multi-task framework requires substantial computational resources, limiting scalability and accessibility for smaller research groups.
- The current focus on DST and RR tasks leaves other dialogue tasks, such as response generation, less explored, requiring further validation.
Future Work
Future directions include expanding dataset coverage to more low-resource languages, optimizing training efficiency, and integrating external knowledge bases to improve understanding. Exploring multi-modal inputs, such as visual or speech data, could further enhance system robustness. Additionally, extending the framework to more complex dialogue tasks and real-world deployment scenarios will be crucial for advancing multilingual dialogue AI.
AI Executive Summary
The rapid globalization of digital services has intensified the need for multilingual task-oriented dialogue systems capable of understanding and responding across diverse languages. However, existing datasets and models predominantly focus on English, leaving low-resource languages underserved. Addressing this gap, the authors introduce Multi2WOZ, a comprehensive multilingual dialogue dataset derived from MultiWOZ through meticulous automatic translation and manual correction, covering Chinese, German, Arabic, and Russian. This dataset is fully aligned across languages, enabling precise cross-lingual performance evaluation.
Building upon this foundation, the paper proposes TOD-XLMR, a multilingual pretrained language model based on XLM-R, further specialized through multi-task training on dialogue-specific objectives. The framework employs intermediate training steps on monolingual and parallel corpora, including innovative cross-lingual response selection and translation language modeling tasks. These strategies aim to adapt the model to target languages, capturing their unique linguistic and contextual nuances.
Extensive experiments on dialog state tracking and response retrieval tasks demonstrate that target language-specific pretraining significantly boosts cross-lingual transfer performance. In zero-shot settings, accuracy improved from a baseline of around 1-2% to over 5% for DST and from 2.7% to 37.4% for RR. Few-shot experiments further confirmed that even with limited target language data, models could achieve near full-data performance, highlighting the efficiency of the proposed approach.
The results underscore the importance of dialogue-specific pretraining and language adaptation, paving the way for more inclusive and effective multilingual dialogue systems. This work not only advances academic understanding but also provides practical tools for deploying dialogue AI in diverse linguistic contexts, promising broader accessibility and user engagement. Future research will focus on expanding dataset diversity, reducing computational costs, and integrating multimodal data to further enhance system robustness and applicability.
Deep Analysis
Background
多语种任务导向对话（TOD）近年来成为自然语言处理的研究热点，旨在实现跨语种、多任务、多领域的智能交互系统。早期工作如MultiWOZ（Budzianowski et al., 2018）提供了丰富的多领域对话数据，但在多语种场景中仍面临数据不足的问题。为解决这一瓶颈，学界尝试通过翻译或少量采样构建多语种数据集（如GlobalWOZ、AllWOZ），但存在数据质量、规模和语义对齐不足的弊端。同时，预训练模型如XLM-R（Conneau et al., 2020）为多语迁移提供了基础，但在任务导向对话中的迁移效果仍有限。近年来，研究逐渐认识到，模型的对话理解能力不仅依赖预训练，还需针对目标语进行特化训练，以适应不同语言的表达习惯和语境差异。
Core Problem
当前多语种TOD研究面临数据稀缺、迁移性能不足、低资源语种表现差等核心难题。缺乏高质量、多语种、多领域对齐数据集，限制了模型在实际应用中的推广。现有迁移策略多依赖英语预训练模型，难以充分捕捉目标语的语义和表达差异，导致迁移效果不理想。如何构建可靠的多语种对话数据、设计有效的迁移策略，成为亟待突破的关键问题。
Innovation
本研究的创新点包括：1）构建了Multi2WOZ，涵盖中文、德语、阿拉伯语和俄语的高质量、多领域对齐对话数据，为跨语迁移提供了可靠基准；2）提出多任务对话预训练框架TOD-XLMR，结合MLM、TLM和RS目标，强化模型的对话理解能力；3）引入目标语对话特化策略，通过在OpenSubtitles语料上进行中间训练，有效提升迁移效果，尤其在少样本场景中表现优异。这些创新突破了以往单一预训练或翻译数据的局限，为多语种对话系统的研究提供了新思路。
Methodology
- �� 构建数据集：从MultiWOZ自动翻译到中文、德语、阿拉伯语和俄语，结合人工校验确保翻译质量。
- �� 预训练模型：在XLM-R基础上，采用MLM、TLM和RS多任务训练，强化模型对话理解能力。
- �� 目标语特化：利用OpenSubtitles平行语料，进行中间训练，设计多任务目标（MLM、TLM、RS）以适应目标语种。
- �� 跨语迁移：在目标语种上进行零样本和少样本微调，评估DST和RR性能。
- �� 实验设计：比较不同预训练策略的迁移效果，分析不同语言表现差异，验证目标语特化的有效性。
Experiments
采用Multi2WOZ数据集，评估DST和RR任务，指标包括准确率和召回率。模型基线为XLM-R，增强模型为经过中间训练的TOD-XLMR。在零样本迁移中，模型仅在英语训练数据上微调，直接在目标语种上测试；在少样本迁移中，加入少量目标语对话进行微调。超参数包括批次大小、学习率（10^-4至10^-6）、训练轮数（30轮），采用早停策略。对比不同中间训练目标（MLM、TLM、RS）对迁移性能的影响，验证目标语特化的效果。
Results
在DST任务中，零样本迁移准确率从1.33%提升至5.39%，RR任务从2.7%提升至37.4%，显示特化策略极大改善迁移效果。少样本实验中，少量目标语对话即可达到接近全样本的性能，验证了样本效率提升。多任务训练（MLM、TLM、RS）结合目标语特化，表现优于单一预训练，尤其在低资源场景中表现出强鲁棒性。不同语言间的迁移效果与语系距离相关，德语和俄语表现优于阿拉伯语和中文。
Applications
该方法适用于多语种智能客服、旅游导览、跨语种问答等场景，特别适合低资源语种。通过高质量多语对话数据和多任务预训练，提升多语种对话系统性能和适应性，降低开发成本，增强用户体验。未来还可结合知识图谱、多模态信息，扩展多语种对话能力。
Limitations & Outlook
数据虽高质量，但仍依赖自动翻译，存在误差。模型训练成本较高，特别在多任务预训练和目标语特化阶段，资源消耗大。当前主要验证DST和RR任务，其他任务的迁移效果尚未充分验证。未来需优化训练效率，扩展低资源语种，增强模型泛化能力。
Plain Language Accessible to non-experts
想象你在一家国际餐厅工作，菜单上有来自不同国家的菜肴。每次点菜时，顾客用不同语言描述他们想吃的东西。你需要理解他们的意思，然后告诉厨房准备食物。以前，你只懂一种语言，处理起来很麻烦。现在，你学会了多种语言的表达方式，还能根据不同国家的习惯调整菜单。这样，无论顾客用哪种语言点菜，你都能准确理解，厨房也能快速准备。这就像让电脑学会多种语言，理解不同国家的对话，变得更聪明、更贴心。
ELI14 Explained like you're 14
想象你在学校里，有很多朋友来自不同国家，他们说不同的语言。有时候，你听不懂他们在说什么，但你希望能帮到他们。于是，你开始学习他们的语言，记住他们说的话和习惯。慢慢地，你变得越来越擅长理解不同的表达方式。有一天，你用一种新学的语言和朋友聊天，发现自己能理解大部分意思，还能帮朋友解决问题。这就像让电脑变得更聪明，学会不同国家的说话方式，能帮人们更好地交流。这个研究就是在教电脑怎么学会多国语言的对话，让它们变得更聪明、更有用。
Abstract
Research on (multi-domain) task-oriented dialog (TOD) has predominantly focused on the English language, primarily due to the shortage of robust TOD datasets in other languages, preventing the systematic investigation of cross-lingual transfer for this crucial NLP application area. In this work, we introduce Multi2WOZ, a new multilingual multi-domain TOD dataset, derived from the well-established English dataset MultiWOZ, that spans four typologically diverse languages: Chinese, German, Arabic, and Russian. In contrast to concurrent efforts, Multi2WOZ contains gold-standard dialogs in target languages that are directly comparable with development and test portions of the English dataset, enabling reliable and comparative estimates of cross-lingual transfer performance for TOD. We then introduce a new framework for multilingual conversational specialization of pretrained language models (PrLMs) that aims to facilitate cross-lingual transfer for arbitrary downstream TOD tasks. Using such conversational PrLMs specialized for concrete target languages, we systematically benchmark a number of zero-shot and few-shot cross-lingual transfer approaches on two standard TOD tasks: Dialog State Tracking and Response Retrieval. Our experiments show that, in most setups, the best performance entails the combination of (I) conversational specialization in the target language and (ii) few-shot transfer for the concrete TOD task. Most importantly, we show that our conversational specialization in the target language allows for an exceptionally sample-efficient few-shot transfer for downstream TOD tasks.