Only Say What You Know: Calibration-Aware Generation for Long-Form Factuality
Decoupling Exploration and Commitment with Calibration-Aware Generation boosts factuality by up to 13% in long-form outputs.
Key Findings
Methodology
This paper introduces the Exploration-Commitment Decoupling paradigm, which separates knowledge exploration from final answer commitment. The core framework, Calibration-Aware Generation (CAG), enables models to generate intermediate reasoning with explicit reliability estimates and then selectively incorporate only trustworthy content into the final output. The process involves: 1) calibrated exploration—producing reasoning steps with associated reliability signals; 2) selective commitment—filtering and prioritizing reliable reasoning when constructing the final answer. Extensive experiments across five benchmarks and multiple models show up to 13% improvement in factuality metrics, with a 37% reduction in decoding time, demonstrating both effectiveness and efficiency.
Key Results
- Across five long-form factuality benchmarks, CAG achieves up to 13% higher VeriScore, with the average increasing from 67.79% to 79.80%. The approach significantly reduces hallucinations while maintaining informativeness. In diverse models like Qwen-14B and Llama-3-8B, the method consistently outperforms baselines, especially in knowledge-intensive QA and dialogue tasks.
- Ablation studies confirm that both calibrated exploration and selective commitment are essential; removing either reduces performance by at least 5%. The framework also generalizes well to retrieval-augmented scenarios, maintaining superior factuality and efficiency.
- The approach improves factual support rate, reduces hallucinations, and accelerates decoding, making it suitable for real-world deployment where reliability is critical.
Significance
This work addresses the fundamental challenge of factuality in long-text generation by introducing a principled framework that enhances model self-awareness. By explicitly modeling the reliability of intermediate reasoning, it effectively suppresses hallucinations, leading to more trustworthy AI systems. The approach bridges the gap between exploration and commitment, enabling models to explore knowledge with awareness and answer cautiously. Its broad applicability across tasks and models signifies a major step toward trustworthy AI, with potential impacts on automated content creation, knowledge-intensive QA, and conversational agents. The method’s efficiency gains further facilitate deployment in resource-constrained environments, advancing the development of reliable, self-aware generative systems.
Technical Contribution
The primary technical innovation lies in the formalization of the Exploration-Commitment Decoupling paradigm, which introduces step-level reliability estimation into the reasoning process. The framework employs a structured supervision scheme, leveraging factuality scores (e.g., VeriScore) to discretize reliability signals and guide the generation process. During training, the model learns to produce calibrated reasoning trajectories and to perform selective answer construction based on reliability signals. Additionally, a novel answer projection mechanism ensures that only information from trustworthy reasoning steps influences the final output. The approach is model-agnostic and compatible with various architectures, enabling broad adoption. Theoretical analysis indicates improved calibration and robustness, with empirical results validating the effectiveness of the method.
Novelty
This work is the first to explicitly decouple exploration and commitment in long-form generation, integrating reliability estimation directly into the reasoning process. Unlike prior methods that rely solely on post-hoc filtering or reward-based optimization, this approach embeds calibration signals during inference, providing models with self-awareness. The discretization of reliability scores and the structured supervision scheme further distinguish this work, offering a transparent and interpretable mechanism for factuality control. These innovations collectively advance the state-of-the-art in trustworthy generative modeling.
Limitations
- The reliance on external verification signals like VeriScore may limit applicability in domains lacking high-quality evidence sources. In complex or ambiguous scenarios, reliability estimation can still misjudge content correctness.
- Computational overhead increases due to additional reliability estimation and filtering steps, especially in multi-turn or multimodal settings.
- The approach assumes availability of annotated factuality scores for training, which may not be feasible in all applications. Future work should explore self-supervised reliability estimation and efficiency improvements.
Future Work
Future directions include developing self-supervised reliability signals to reduce dependence on external annotations, optimizing the filtering mechanism for faster inference, and extending the framework to multimodal data such as images and videos. Additionally, integrating reinforcement learning to adaptively tune reliability thresholds and exploring broader applications in real-time systems will be valuable. Enhancing the interpretability and user control over the reliability signals can further improve trustworthiness and transparency of AI systems.
AI Executive Summary
Long-form text generation has made remarkable progress with large reasoning models like GPT-5 and Qwen, yet the issue of hallucinations—plausible but factually incorrect content—remains a significant barrier to deployment in real-world applications. Traditional solutions such as reinforcement learning with reward signals or post-hoc filtering have limitations, primarily because they treat the entire reasoning process as a monolithic entity, lacking fine-grained control over the reliability of intermediate steps.
This paper introduces a novel paradigm called Exploration-Commitment Decoupling, which fundamentally rethinks how models handle knowledge exploration and answer commitment. The core idea is to enable models to explore multiple reasoning paths with awareness of their reliability, and then selectively commit to only the most trustworthy information when generating the final answer. This is operationalized through the Calibration-Aware Generation (CAG) framework, which equips models with the ability to generate intermediate reasoning steps annotated with explicit reliability signals. During answer construction, the model filters and prioritizes content based on these signals, effectively reducing hallucinations while maintaining informativeness.
Extensive experiments across five benchmarks demonstrate that CAG improves factuality metrics by up to 13%, with a notable 37% reduction in decoding time. The approach generalizes well across different models, tasks, and scenarios, including knowledge-intensive question answering and retrieval-augmented generation. Ablation studies confirm that both calibrated exploration and selective commitment are crucial for performance gains.
This work significantly advances the development of trustworthy AI systems by embedding self-awareness into the generation process. Its broad applicability and efficiency improvements suggest promising future directions, such as self-supervised reliability estimation, multimodal integration, and deployment in real-time applications. Overall, the research offers a principled, effective solution to the longstanding challenge of factuality in long-form AI generation, paving the way for more reliable and transparent AI systems.
Deep Analysis
Background
近年来，深度学习推动大型预训练模型（如GPT系列、Qwen系列）在自然语言处理领域取得突破。早期工作如GPT-3、T5等通过大规模预训练实现多任务适应，但在长文本生成中仍存在“幻觉”问题，即模型输出虽合理但缺乏事实依据。为缓解此问题，学界提出了事实校准、检索增强等技术，例如VeriScore指标用于评估内容支持比例，强化学习（RL）被用来优化输出质量。然而，这些方法多关注整体性能，缺乏对推理中各环节可靠性的细粒度建模，导致虚假信息在长文本中逐步累积，影响模型可信度。
Core Problem
长文本生成中的核心难题在于如何有效控制推理过程中的信息质量。传统模型未能区分不同推理步骤的可靠性，导致不可靠内容直接传播到最终答案，形成“幻觉”。在知识密集型任务中，错误推理会引发偏差，严重影响应用效果。解决方案需要在模型内部引入可靠性评估机制，实现对推理内容的动态筛选和调节，从而提升整体事实性和用户信任。这一挑战在于如何在保持信息丰富的同时，避免虚假内容的扩散。
Innovation
本文的创新点在于提出探索-承诺解耦（Exploration-Commitment Decoupling）范式，将推理探索与输出承诺两个环节解耦，赋予模型在推理时自主评估内容可靠性的能力。具体包括：1）在中间推理中引入可靠性信号，生成带有支持度的推理内容；2）在最终输出时，根据可靠性信号筛选信息，优先使用可信内容。此机制突破了传统只在后处理或奖励优化中引入校准的限制，实现了模型在推理阶段的自我认知，显著提升事实性指标。结合结构化监督和答案重投影，确保输出的真实性和连贯性。
Methodology
- �� 设计探索-承诺解耦架构，将推理步骤（r）与可靠性（c）分离，形成(r, c)的结构化表示。
- �� 在推理过程中，模型生成每个步骤的内容和对应的可靠性信号，利用VeriScore等指标进行标注。
- �� 采用离散化策略，将连续的可靠性分数映射到有限标签（如“可靠”/“不可靠”），增强模型的可解释性。
- �� 在答案生成阶段，根据可靠性筛选推理内容，优先保留可信信息，抑制不可靠部分。
- �� 训练采用结构化监督（Structured Supervision），同时优化推理的可靠性和答案的正确性。
- �� 引入答案重投影机制，确保最终输出仅依赖可信推理内容，减少虚假信息。
Experiments
采用五个长文本事实性基准（AlpacaFact、Biography、FactBench、Factory、FAVA），涵盖不同任务和模型（如Qwen-4B、8B、14B，Llama-3-8B）。比较基线包括奖励优化和后处理筛选。指标主要为VeriScore，评估内容支持比例。训练中利用GPT-5作为验证模型，采用结构化监督和离散可靠性标签，训练时间为2轮，批次大小为4。评估时结合消融实验，验证校准信号和选择性承诺的贡献。
Results
CAG在所有基准上优于传统方法，最高提升13%的事实性指标，VeriScore从67.79%提升至79.80%。模型在知识问答和对话场景中表现出良好泛化能力，特别是在Qwen-14B模型上，事实支持率提升超过8%。消融实验显示，去除可靠性校准或选择性承诺会导致性能下降至少5%。结合检索增强（RAG）场景，模型依然表现优越，验证其多场景适应性。
Applications
该方法适用于自动问答、智能助手、内容生成等场景，尤其在知识密集型任务中表现优异。通过引入可靠性评估机制，模型能自主筛选信息，减少虚假内容，提升用户信任。未来可结合多模态信息，扩展到图像、视频等多源数据的事实校准，推动可信AI的落地。
Limitations & Outlook
当前方法依赖外部验证指标（如VeriScore）进行可靠性标注，受限于验证数据的覆盖范围。在复杂或模糊场景中，模型仍可能误判可靠性，导致虚假信息未被筛除。此外，模型在大规模应用时计算成本较高，推理速度有待优化。未来应开发更高效的校准机制和多模态融合策略，以实现更广泛的应用。
Plain Language Accessible to non-experts
想象你在做一道复杂的菜，需要不断尝试不同的调料和方法。每次尝试后，你会记下哪些调料效果好，哪些不好，然后再决定下一步用哪些。这个过程就像模型在生成长文本时不断探索不同的内容，但它会根据之前的“尝试”判断哪些信息更可靠，最后只把最靠谱的部分放到菜里。这样做可以避免做出“怪味”菜，也能做出更好吃的。这个方法让模型像个有经验的厨师，知道哪些信息可以信赖，哪些要小心处理，确保输出既丰富又准确。
ELI14 Explained like you're 14
想象你在写一篇长文章，比如写一篇关于历史人物的故事。你会先查很多资料，挑选你觉得最靠谱的细节，然后写出来。可是，有时候你会记错一些细节，导致故事不太真实。现在，这个新方法就像给你一个“可靠度”标签，告诉你哪些资料更靠谱，哪些不太确定。写完后，你会只用那些“靠谱”的细节，避免用不确定的内容。这样，你写的故事就更真实，也更让人相信。它让模型像个聪明的作家，知道怎么挑选信息，写出更可信的故事。
Glossary
Exploration-Commitment Decoupling
A paradigm that separates the reasoning exploration phase from the final answer commitment, allowing models to evaluate the reliability of intermediate steps. Technically, it involves generating reasoning with associated reliability signals and filtering based on these signals.
The core concept introduced in this paper to improve factuality in long-form generation.
Calibration-Aware Generation
A generation process where models produce intermediate reasoning with explicit reliability estimates, enabling self-assessment and selective answer construction. It employs structured supervision and discretized reliability labels.
The main framework implementing the decoupling paradigm.
VeriScore
An evaluation metric that measures the proportion of claims supported by external evidence, reflecting the factual support of generated content. It guides training and assessment of factuality.
Used to supervise reliability estimation and evaluate factuality improvements.
Open Questions Unanswered questions from this research
- 1 如何在缺乏高质量验证数据的领域实现有效的可靠性估计，减少对外部验证指标的依赖。
- 2 多模态信息融合中的可靠性校准机制设计，确保不同数据源的内容一致性和可信度。
- 3 模型在大规模部署中的计算效率优化，尤其是在多轮推理和多任务场景下的性能提升。
Applications
Immediate Applications
知识问答系统
利用校准感知生成提升问答的事实性和可信度，减少虚假信息，增强用户信任。
内容生成与编辑
在新闻、报告等内容生成中筛选可靠信息，确保输出内容的真实性和准确性。
Long-term Vision
自主可信AI系统
实现具有自我校准和自我验证能力的AI，推动可信自动化决策和内容创作。
Abstract
Large Reasoning Models achieve strong performance on complex tasks but remain prone to hallucinations, particularly in long-form generation where errors compound across reasoning steps. Existing approaches to improving factuality, including abstention and factuality-driven optimization, follow a \emph{coupled exploration-commitment} paradigm, in which intermediate reasoning is unconditionally propagated to the final output, limiting fine-grained control over information selection and integration. In this paper, we propose an \textbf{Exploration-Commitment Decoupling} paradigm that disentangles knowledge exploration from final commitment, enabling models to explore with awareness while answering cautiously. We instantiate the paradigm with \textbf{Calibration-Aware Generation (CAG)}, a framework that equips models with end-to-end, calibration-aware generation capabilities, by augmenting intermediate reasoning with calibrated reliability estimates and prioritizing reliable content in final outputs. Across five long-form factuality benchmarks and multiple model families, CAG improves factuality by up to 13%, while reducing decoding time by up to 37%. Overall, our work highlights decoupling as a principled approach for more reliable long-form generation, offering directions for trustworthy and self-aware generative systems.