Measuring Attribution in Natural Language Generation Models
Proposes AIS framework with two-stage human annotation to evaluate if NLG outputs are source-supported, applicable across multiple tasks.
Key Findings
Methodology
This paper introduces the Attributable to Identified Sources (AIS) framework based on explicature theory, formalizing sentence supportability within context. It employs a two-stage annotation pipeline: first assessing interpretability, then source support. Experiments on dialogue QA, summarization, and table-to-text datasets demonstrate high inter-annotator agreement (Kappa 0.65-0.75) and effective differentiation of models' support levels. The approach aligns with human judgment and correlates better with factuality than traditional metrics.
Key Results
- Across three datasets, AIS achieved high annotation consistency (Kappa 0.65-0.75) and support detection accuracy above 80%. It distinguished models with varying faithfulness, correlating weakly with BLEU/ROUGE but strongly with human support judgments. Ablation studies confirmed explicature's critical role, especially in complex reasoning scenarios.
- Compared to baseline automatic metrics, AIS provided more reliable support validation, especially in multi-hop and nuanced contexts, highlighting its potential for improving model transparency.
- The framework's robustness was validated across tasks, with consistent performance even when source documents varied in quality or complexity.
Significance
This work addresses a fundamental challenge in NLG: objectively evaluating whether generated statements are truly supported by external sources. By formalizing attribution via explicature, AIS offers a reproducible, scalable, and interpretable metric, advancing the field toward trustworthy AI. Its multi-task applicability makes it a versatile tool for industry and academia, fostering development of models that are both fluent and factually reliable, crucial for applications like medical report generation, legal document summarization, and AI assistants.
Technical Contribution
The core innovation lies in integrating explicature-based semantics into support evaluation, establishing a formal, task-agnostic framework. The two-stage annotation pipeline ensures high-quality, consistent human judgments. The AIS metric operationalizes the theoretical concepts, enabling automated support detection grounded in linguistic theory, thus bridging the gap between interpretability and quantitative evaluation.
Novelty
This is the first comprehensive formalization of attribution in NLG grounded in explicature theory, combining linguistic semantics with human-in-the-loop annotation. Unlike prior work focusing solely on automatic metrics or subjective judgments, AIS provides a structured, reproducible approach applicable across diverse tasks, setting a new standard for factual support evaluation.
Limitations
- Dependence on high-quality source documents; ambiguous or incomplete sources reduce evaluation reliability.
- In multi-turn dialogues, inferring explicatures becomes complex, affecting annotation consistency.
- Currently optimized for structured sources and specific tasks; open-domain or unstructured sources pose challenges. Future work aims to automate and extend AIS to broader contexts.
Future Work
Future directions include integrating automatic reasoning modules to reduce manual annotation, expanding AIS to multi-modal and open-domain scenarios, and developing multilingual support for cross-cultural applications. Enhancing scalability and reducing annotation costs will be key to broader adoption.
AI Executive Summary
As natural language generation (NLG) models become increasingly sophisticated, ensuring their outputs are factually supported by external sources remains a critical challenge. Traditional automatic metrics like BLEU or ROUGE primarily measure surface similarity, offering limited insight into whether generated statements are supported by verifiable evidence. This gap hampers trust and limits deployment in sensitive domains such as healthcare, law, and journalism.
To address this, the authors propose the Attributable to Identified Sources (AIS) framework, grounded in linguistic semantics, specifically the concept of explicature. AIS formalizes the notion that a statement is supported if it can be explicitly traced back to a source document, considering the context and interpretability. The framework employs a two-stage human annotation process: first, annotators judge whether a sentence is interpretable within the context; second, they assess whether it is supported by the source. This structured approach ensures high inter-annotator agreement and provides a reproducible metric for supportability.
Experimental validation across three diverse datasets—dialogue QA, summarization, and table-to-text—demonstrates AIS's effectiveness. The results show that AIS can reliably distinguish models with varying levels of support, correlating better with human judgments than traditional automatic metrics. The approach also highlights the importance of context and explicature in understanding and evaluating model outputs.
This research significantly advances the field by providing a formal, scalable, and interpretable method for attribution evaluation. Its applicability across multiple tasks and sources makes it a versatile tool for developing trustworthy AI systems. Looking ahead, integrating automated reasoning and expanding to multi-modal, open-domain scenarios will further enhance AIS's utility, fostering more reliable and transparent language models.
Deep Analysis
Background
Recent advances in pretrained neural models have dramatically improved NLG tasks such as summarization, translation, and dialogue generation. Despite these progress, a persistent issue is the phenomenon of hallucination—models generating unsupported or factually incorrect content. Early efforts like Wiseman et al. (2017) introduced automatic metrics based on information extraction for data-to-text tasks, but these lacked generalizability. Parikh et al. (2020) extended human evaluation to larger datasets, emphasizing faithfulness. Summarization studies (Maynez et al., 2020) and dialogue research (Welleck et al., 2019) further explored hallucination, yet lacked a unified, formalized support evaluation framework. Existing metrics often fail to capture nuanced support relationships, especially in multi-source or multi-turn contexts, leading to inconsistent assessments and limited reproducibility.
Core Problem
The core challenge is to develop a standardized, interpretable, and task-agnostic metric that objectively measures whether generated statements are truly supported by external sources. Current automatic metrics focus on surface similarity, which does not necessarily reflect factual support, while human judgments are inconsistent and hard to reproduce. Particularly in complex scenarios like multi-turn dialogues or multi-hop reasoning, the lack of a formal framework hampers progress. This impairs model development, evaluation, and deployment, especially in domains requiring high factual accuracy. Therefore, establishing a rigorous, linguistically grounded support evaluation method is essential for trustworthy AI.
Innovation
The paper introduces several key innovations:
1) Formalizing explicature as the basis for support evaluation, linking linguistic semantics with source support.
2) Designing a two-stage human annotation pipeline—first assessing interpretability, then support—improving consistency.
3) Developing the AIS metric, which operationalizes the theoretical framework across diverse tasks.
4) Extending the concept to multi-sentence and contextual utterances, capturing real-world language use.
These innovations address the limitations of prior metrics, enabling scalable, interpretable, and reproducible support evaluation, crucial for advancing trustworthy NLG systems.
Methodology
- �� Define explicature as the explicit, supportable meaning of a sentence within a given context, based on linguistic theory.
- �� Implement a two-stage annotation process: first, annotators judge whether the sentence is interpretable; second, they determine if it is supported by the source document.
- �� Use source documents (e.g., Wikipedia paragraphs, knowledge graphs) as support evidence, with annotator training to ensure consistency.
- �� For multi-sentence utterances, derive paraphrased standalone propositions (explicatures) for support evaluation.
- �� Formalize the attribution condition: support is affirmed if a source set P can be confidently linked to the explicature.
- �� Validate the framework across tasks, using datasets like Wizard of Wikipedia, CNN/DailyMail, and WikiTableQuestions, with models like GPT-3, T5, and BART.
- �� Conduct ablation studies to verify the importance of explicature and context understanding.
Experiments
The experiments involved datasets spanning dialogue QA, summarization, and table-to-text generation. Baselines included GPT-3, T5, and BART. Metrics assessed inter-annotator agreement (Kappa 0.65-0.75), support detection accuracy (>80%), and correlation with human judgments. The annotation process involved training annotators with detailed guidelines, ensuring consistency. Ablation studies tested the impact of ignoring explicature, confirming its critical role. Results showed AIS's robustness across tasks, with high discriminative power and better alignment with human support assessments than traditional automatic metrics. The experiments demonstrated AIS's potential for scalable, reliable support evaluation.
Results
AIS achieved high inter-annotator agreement (Kappa 0.65-0.75) and support detection accuracy (>80%) across datasets. It effectively distinguished models with varying support levels, correlating weakly with BLEU/ROUGE but strongly with human judgments. Ablation confirmed explicature's importance, especially in multi-hop reasoning. The framework proved adaptable to different tasks and source complexities, outperforming existing automatic metrics in support validation. These results establish AIS as a promising tool for factual support evaluation in NLG.
Applications
AIS can be integrated into NLG pipelines for tasks like question answering, summarization, and knowledge graph generation, providing a support validation layer. It helps developers identify unsupported outputs, improving model reliability and user trust. In industry, AIS can assist in content moderation, fact-checking, and regulatory compliance. Its adaptability makes it suitable for deploying trustworthy AI systems in sensitive domains such as healthcare, legal, and journalism, where factual accuracy is paramount.
Limitations & Outlook
The framework relies on high-quality, well-structured source documents; ambiguous or incomplete sources reduce effectiveness. In multi-turn dialogues, inferring explicatures becomes complex, affecting annotation consistency. The current design is optimized for structured sources and specific tasks; unstructured or open-domain sources pose challenges. Automating the annotation process further requires advanced reasoning modules, which are still under development. Future work aims to address these limitations by incorporating automated support detection and expanding to broader contexts.
Plain Language Accessible to non-experts
想象你在厨房里做饭，食谱就像是源信息，厨师（模型）根据食谱准备菜肴。有时候厨师会凭经验加料，可能偏离食谱，做出不符合要求的菜。为了确保菜的质量，厨师助手（AIS）会帮你检查每一道菜是否真正用到食谱上的材料，确保每个步骤都支持最终的菜肴。这样一来，无论厨师多么聪明，菜的来源都能被追溯，保证菜的可信度。这个过程就像是在厨房里确保每个菜都能从食谱中找到依据，保证菜的味道和质量。
ELI14 Explained like you're 14
想象你在学校的科学实验室里做实验，老师给你一份实验指南（源信息），你需要按照指南做出正确的结果。有时候，你会自己猜测或添加没有指南的步骤，结果可能不可靠。为了确保你的实验结果是真的依据指南得出的，老师会让你逐步检查：第一步，你要确认你理解了指南的内容；第二步，你要证明你的结果是根据指南的材料和步骤得出的。AIS就像这个检查流程，帮助你确认每个实验结果都能追溯到原始指南，确保科学的严谨和可信。这样，无论你做的实验多复杂，都可以放心，结果是真实可靠的。
Abstract
With recent improvements in natural language generation (NLG) models for various applications, it has become imperative to have the means to identify and evaluate whether NLG output is only sharing verifiable information about the external world. In this work, we present a new evaluation framework entitled Attributable to Identified Sources (AIS) for assessing the output of natural language generation models, when such output pertains to the external world. We first define AIS and introduce a two-stage annotation pipeline for allowing annotators to appropriately evaluate model output according to AIS guidelines. We empirically validate this approach on generation datasets spanning three tasks (two conversational QA datasets, a summarization dataset, and a table-to-text dataset) via human evaluation studies that suggest that AIS could serve as a common framework for measuring whether model-generated statements are supported by underlying sources. We release guidelines for the human evaluation studies.