Dual-Confidence Contrastive Decoding for Retrieval-Augmented Generation
DCCD combines document and token confidence to improve conflict resolution in multi-document QA, outperforming baselines on DRQA.
Key Findings
Methodology
This paper introduces Dual-Confidence Contrastive Decoding (DCCD), a training-free approach that integrates document-level confidence (assessing if a document suffices for answering) and token-level confidence (predicting the certainty of next-token). DCCD employs a support probe for rapid document confidence estimation and Dirichlet distribution for token uncertainty. It dynamically adjusts the decoding process by selecting positive and negative document-conditioned streams based on combined confidence scores, scaling the contrastive term by their confidence margin. Extensive experiments on the newly proposed DRQA benchmark and standard QA datasets demonstrate DCCD's superior performance over existing contrastive and full-context decoding methods, especially in conflict-rich enterprise scenarios.
Key Results
- On DRQA, DCCD improves top-5 accuracy to 53.10% with Qwen3.5-9B, surpassing traditional contrastive methods by approximately 6 percentage points. Across datasets like Natural Questions and TriviaQA, DCCD outperforms CAD, DVD, and COCOA by 3-5%, showing robustness in conflict scenarios.
- In noisy and outdated evidence environments, DCCD effectively suppresses misleading information, leading to more accurate and reliable answers. Its ability to identify and leverage high-confidence evidence sources results in significant gains in complex multi-source QA tasks.
- The approach demonstrates consistent improvements across different model sizes and datasets, validating the generality and scalability of the dual-confidence mechanism.
Significance
This work addresses a critical challenge in multi-source knowledge integration—internal evidence conflict—by introducing a source-aware, confidence-gated decoding strategy. It enhances the robustness of large language models in enterprise and real-world applications, where conflicting, noisy, or stale evidence is common. The method's training-free nature facilitates deployment in diverse environments, reducing the need for costly fine-tuning. By providing a standardized benchmark (DRQA) for conflict resolution, the study paves the way for future research in trustworthy AI, advancing the development of more reliable, explainable, and context-aware NLP systems.
Technical Contribution
The core innovation lies in integrating dual confidence signals—document-level and token-level—into a contrastive decoding framework. This fusion allows dynamic, context-sensitive evidence selection without additional training. The support probe mechanism provides rapid, approximate assessment of evidence sufficiency, while Dirichlet-based token uncertainty captures local prediction confidence. The combined score guides the selection of positive and negative evidence streams, scaling the contrastive term to amplify trustworthy evidence and suppress conflicting signals. This approach fundamentally differs from prior methods like DVD or COCOA by explicitly modeling source reliability and local prediction certainty, resulting in improved accuracy and robustness.
Novelty
This research pioneers the combination of document-level and token-level confidence within a contrastive decoding paradigm, specifically targeting internal evidence conflicts in multi-document QA. Unlike previous methods that treat retrieved evidence as a monolithic context, DCCD dynamically evaluates and adjusts evidence influence at each decoding step. Its training-free design and dual-signal integration represent a significant step forward in source-aware, conflict-sensitive generation, setting a new standard for robustness in retrieval-augmented NLP.
Limitations
- The effectiveness of confidence estimation depends on the pre-trained model's capacity, potentially limiting performance in highly noisy or adversarial environments. The support probe's simplicity may not capture complex evidence nuances, leading to occasional misjudgments.
- Computational overhead increases with the number of documents and the complexity of confidence calculations, which could hinder real-time deployment at large scale.
- The method's reliance on heuristic scaling of the contrastive term may require task-specific tuning, and its performance in multi-modal or multilingual settings remains to be explored.
Future Work
Future directions include integrating adaptive learning for confidence estimation, extending the framework to multi-modal data (images, audio), and developing more efficient algorithms for large-scale, real-time applications. Additionally, exploring self-supervised training strategies to refine confidence measures and expanding the benchmark suite to include more diverse, real-world scenarios will further enhance the robustness and applicability of the approach.
AI Executive Summary
In the era of information overload, enterprises and researchers face the challenge of synthesizing accurate knowledge from vast, heterogeneous sources. Traditional large language models (LLMs) rely heavily on internal parameters, which often become outdated or incomplete, especially when faced with conflicting evidence from multiple documents. Retrieval-augmented generation (RAG) offers a promising solution by incorporating external evidence, but it introduces new complexities: how to handle internal conflicts within retrieved data. Existing contrastive decoding methods primarily focus on resolving conflicts between the model's memory and the retrieved context, yet they fall short when evidence within the retrieved set itself contradicts each other.
This paper addresses this gap by proposing Dual-Confidence Contrastive Decoding (DCCD), a novel, training-free decoding strategy that leverages two levels of confidence signals. The first, document-level confidence, estimates whether a retrieved document is sufficient for answering the question. The second, token-level confidence, assesses whether the document supports a confident prediction of the next token. These signals are combined to dynamically select positive and negative evidence streams during decoding, scaling the contrastive term by their confidence margin. The approach effectively amplifies trustworthy evidence while suppressing conflicting or noisy information.
The authors introduce DRQA, a benchmark derived from enterprise deep-research scenarios, designed to evaluate models' ability to resolve internal evidence conflicts. Experiments across DRQA and standard QA datasets demonstrate that DCCD outperforms existing full-context and contrastive decoding baselines, with significant improvements on DRQA. The results highlight the importance of source-aware, confidence-gated decoding mechanisms in complex, real-world environments where evidence is often heterogeneous and conflicting.
Overall, this work advances the state-of-the-art in retrieval-augmented question answering by providing a robust, scalable, and explainable decoding strategy. It opens new avenues for deploying reliable AI systems in enterprise settings, where accurate and trustworthy information synthesis is critical. Future research may focus on extending these confidence mechanisms to multi-modal data, optimizing computational efficiency, and further refining the evaluation benchmarks to encompass broader real-world challenges.
Deep Analysis
Background
随着大规模预训练模型的普及，检索增强生成（RAG）逐渐成为提升知识覆盖和实用性的核心技术。早期的研究如Karpukhin等（2020）提出的DPR，通过密集检索结合生成模型，显著改善了开放域问答性能。Lewis等（2020）引入的RAG框架，将检索和生成结合，解决了模型知识更新滞后问题。然而，现有方法多关注单一知识源的可靠性，忽视多源证据中的内部冲突和噪声问题。近年来，针对证据冲突的研究逐步增多，如Wang等（2025）提出的对比解码，但多依赖微调或复杂训练，难以在企业环境中快速部署。企业场景中的数据具有私有性、时效性和多样性，证据之间常存在矛盾，亟需源感知和置信度调节机制。本研究基于此背景，旨在提出一种无需训练、结合多层置信度的解码策略，以提升多文档问答中的冲突处理能力。
Core Problem
多文档问答中，检索到的证据常包含冲突信息、噪声甚至过时数据，导致模型难以判断应采纳哪个证据。传统解码策略如贪婪或束搜索无法有效区分可信与不可信的证据源，特别是在证据内部存在矛盾时，模型容易受到干扰，生成不准确或误导性答案。这一问题在企业环境尤为突出，企业信息具有私有性、时效性和多样性，证据之间的冲突更为复杂。解决该问题的关键在于引入源感知的置信度机制，动态调节证据的影响力，从而在冲突环境中保持答案的准确性和可信度。
Innovation
本研究的创新点在于提出双重置信度机制，结合文档级置信度（判断证据是否足够）与词级置信度（评估预测的确定性），实现源感知的动态调节。具体包括：• 支持探针（support probe）快速评估每个文档的置信度；• 利用Dirichlet分布估算每个词的置信度；• 设计无训练的对比解码策略，将两个信号融合，调节生成中的证据选择。这种机制不同于传统单一信号或微调方法，提供了更细粒度的证据筛选和调节能力，显著提升多源环境下的问答性能。
Methodology
- �� 输入：问题x和检索到的文档集合D；• 通过支持探针πsup(x, di)快速评估每个文档的置信度qi；• 利用Dirichlet分布估算每个词的置信度ci(t)，衡量预测的确定性；• 计算双重置信度s(i,t)=qi+ci(t)，融合两个信号；• 在每个解码步骤，选择最高和最低s(i,t)的文档作为正负源，调节对比项的规模ϵt；• 最终生成的概率分布为：z_t=z_fullt+ϵt(zi+t−zi−t)，在全上下文基础上引入源感知的调节机制，增强可信证据，抑制冲突信息。
Experiments
采用DRQA、Natural Questions、TriviaQA、PopQA和RetrievalQA五个多文档问答数据集，比较DCCD与CAD、DVD、COCOA等解码策略。模型使用Qwen3.5-2B、Qwen3.5-9B和Phi-3-medium，指标包括Top-k准确率和LLM-as-a-judge的语义正确性。实验设置包括不同检索文档数（Top-5、Top-10），调节对比参数，进行消融分析验证信号融合效果。评估重点在于冲突环境下的问答准确率和鲁棒性。
Results
DCCD在DRQA中Top-5准确率提升至53.10%，比传统方法最高提升约6个百分点。在标准问答数据集上，平均性能优于CAD、DVD等，提升幅度达3-5%。在存在证据冲突的场景中，DCCD表现出更强的抗干扰能力，显著减少误导性答案。多模型、多数据集验证其广泛适用性，验证了双重置信度机制的有效性。
Applications
该方法适用于企业知识管理、自动问答、信息整合等场景，尤其在多源、多模态信息环境中表现优越。可结合企业内部知识库、实时数据流，实现高效冲突检测与答案生成，提升企业决策效率。未来还可拓展到多模态、多语言和大规模知识图谱中，推动智能企业系统的升级。
Limitations & Outlook
当前方法在极端噪声或严重偏离事实的证据环境下仍可能受影响，置信度估算依赖预训练模型能力。支持探针设计参数敏感，需针对不同任务微调。大规模知识库的实时处理存在计算成本挑战，未来需优化效率和自适应能力。
Plain Language Accessible to non-experts
想象你在厨房做饭，手边有很多食材。有些食材新鲜、适合做菜，有些则可能变质或不相关。你需要判断哪些食材能做出美味佳肴，同时避免放入变质的东西。有时候不同的食材会互相冲突，比如一个说“这是新鲜的”，另一个又说“已经变质”。这就像模型在处理多个信息源时，面对矛盾和噪声。DCCD就像一个聪明的厨师，能根据每个食材的“可信度”决定用哪个，避免被误导，做出最好的菜肴。它会快速判断哪些资料值得相信，哪些要忽略，然后在做菜时优先用可信的食材，确保最终的菜肴既美味又可靠。
ELI14 Explained like you're 14
想象你在学校里，有很多朋友告诉你不同的事情。有的朋友说“今天是晴天”，另一个朋友说“今天下雨了”。你要决定相信谁，才能安排好今天的活动。这个过程就像模型在回答问题时，要从很多信息中挑出最靠谱的证据。DCCD就像一个聪明的朋友，能快速判断哪个消息更可信，然后在回答时优先用这些信息。它还会看每个消息的细节，判断内容是否很确定，就像你判断朋友说的话是不是很确定一样。这样一来，即使有人说错话或消息有误，模型也能聪明地避开误导，给出正确的答案。这个方法让机器变得更聪明，能在复杂、信息混杂的环境中做出更靠谱的判断。
Glossary
Retrieval-Augmented Generation (检索增强生成)
结合外部知识检索和生成模型，提升回答的准确性和时效性。技术上通过检索相关文档，再用生成模型整合信息。
论文中用以描述模型从多个文档中获取信息，解决知识冲突问题。
对比解码 (Contrastive Decoding)
一种无需训练的解码策略，通过对比不同条件下的模型输出，增强生成的相关性和可信度。
用以调节模型在多源信息中的生成行为，减少冲突干扰。
置信度 (Confidence)
模型对某一证据或预测的确信程度，反映其可靠性。包括文档级置信度和词级置信度。
用于筛选可信证据，调节生成过程中的证据影响力。
Dirichlet分布 (Dirichlet Distribution)
一种多变量概率分布，用于估算预测不确定性，特别适合多类别概率分布的建模。
在论文中用来估算词级预测的置信度。
DRQA (Deep Research Question Answering)
基于企业深度研究场景构建的多文档问答基准，含有合成的企业私有事实，强调证据冲突的检测。
评估模型在复杂、多源、冲突环境中的表现。
Open Questions Unanswered questions from this research
- 1 如何在极端噪声环境中进一步提升置信度估算的鲁棒性？当前方法在多模态、多语言场景下的适应性如何？
Applications
Immediate Applications
企业知识问答系统
结合企业内部文档和实时数据，利用DCCD提升自动问答的准确性与可信度，减少误导信息，支持决策。
多源信息整合平台
在多模态、多渠道信息环境中，自动筛选可信证据，优化信息融合流程，提升信息检索和推理能力。
Long-term Vision
智能企业知识管理
实现企业知识的动态更新与冲突检测，构建自适应、可信赖的知识库，推动企业智能化转型。
Abstract
Retrieval-augmented generation (RAG) increasingly requires models to answer questions from multiple retrieved documents, where only some sources are relevant and the retrieved bundle may contain stale, noisy, or conflicting evidence. Existing contrastive decoding methods primarily focus on resolving conflicts between the model's internal memory and the retrieved context. In contrast, we study the complementary problem of intra-context conflict in multi-document RAG. To evaluate this setting, we introduce DRQA, a factual-conflict question answering benchmark derived from enterprise deep-research scenarios, where answers are grounded in synthetic enterprise-specific facts that are designed not to be recoverable from the model's internal memory. We further propose Dual-Confidence Contrastive Decoding (DCCD), a training-free decoding method that combines document-level confidence, which estimates whether a document appears sufficient for answering the question, with token-level confidence, which estimates whether that document supports a confident next-token prediction. DCCD selects positive and negative document-conditioned streams using these dual-confidence signals and scales a document-level contrast by their confidence margin. Across DRQA and standard multi-document QA benchmarks, DCCD achieves the best average performance among full-context and contrastive decoding baselines, with the largest gains on DRQA. These results highlight the importance of source-aware, confidence-gated decoding when retrieved evidence is internally conflicting.