Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns
Proposes GCM, leveraging multi-model history to improve calibration and generalization, surpassing self-prediction.
Key Findings
Methodology
This work develops various correctness predictors, including trained (SCMs) and training-free (ICL, post-calibration) methods. Central to the approach is using historical predictions from multiple models to train a Generalized Correctness Model (GCM). By controlling training data, input features (answer phrasing, model identity), and conditioning variables, the study analyzes their impact on performance. Using Qwen3-8B, models are fine-tuned with LoRA, employing cross-entropy loss on datasets like MMLU and TriviaQA. Cross-model and cross-dataset evaluations demonstrate that leveraging multiple models’ history yields superior accuracy, calibration (ECE < 0.02), and AUROC (~0.88).
Key Results
- GCM outperforms single-model self-predictions by 2-4% accuracy, achieves ECE as low as 0.02, and AUROC up to 0.88 across diverse models and datasets. It generalizes well to unseen models and tasks, surpassing traditional confidence methods.
- Across model sizes (3B–70B) and families (Llama, Qwen, Gemma), GCM maintains strong performance, especially in out-of-distribution settings. Incorporating historical info via ICL and post-calibration further boosts accuracy and calibration.
- Answer phrasing and world knowledge are key features; models utilizing detailed answer expressions and background info significantly improve correctness prediction. Multi-model history encoding is more effective than relying solely on self-confidence scores.
Significance
This work overcomes the limitations of self-knowledge reliance, introducing a multi-model, history-based correctness predictor that enhances confidence calibration and robustness. It addresses a critical challenge for deploying reliable AI in high-stakes domains like healthcare and law, where trustworthy outputs are essential. By systematically encoding historical performance, the approach reduces misjudgments and improves decision-making reliability. The findings also shed light on the importance of answer phrasing and background knowledge, guiding future model design for better interpretability and trustworthiness.
Technical Contribution
The paper introduces a novel framework that trains correctness models on aggregated multi-model historical data, enabling cross-model and cross-task generalization. It combines deep learning, transfer learning, and calibration techniques, including ICL and post-hoc methods, to improve prediction accuracy and calibration. The GCM architecture leverages multiple features—question, answer phrasing, model identity—and demonstrates superior performance over existing confidence estimation methods. The systematic analysis of conditioning variables and feature importance advances understanding of what drives correctness prediction.
Novelty
This is the first comprehensive attempt to utilize multi-model historical prediction data to train a universal correctness predictor. Unlike prior work focused on self-prediction or single-model calibration, this approach harnesses collective past performance, enabling robust, transferable correctness estimation across models and datasets. The integration of ICL and post-calibration techniques further distinguishes this work, establishing a new paradigm for model confidence assessment.
Limitations
- The approach requires extensive historical data and multiple models, leading to high computational costs, which may limit scalability in resource-constrained environments.
- Performance in highly out-of-distribution or novel tasks remains limited; further research is needed to enhance robustness.
- While answer phrasing and world knowledge are influential, their effectiveness varies across complex, multi-step reasoning tasks, necessitating further feature optimization.
Future Work
Future directions include developing more efficient data collection and training methods to reduce computational overhead, exploring multi-modal correctness prediction, and enhancing robustness in out-of-distribution scenarios. Integrating reinforcement learning and self-supervised signals could further improve adaptability. Extending the framework to multi-turn dialogues and multi-modal inputs (images, audio) is also promising, aiming for comprehensive trustworthy AI systems.
AI Executive Summary
The rapid advancement of large language models (LLMs) has revolutionized natural language processing, yet their deployment in critical applications hinges on reliable confidence estimation. Traditional methods focus on models' self-assessment capabilities, but experiments reveal that LLMs lack genuine self-knowledge about their correctness, performing no better than unrelated models in predicting their own outputs’ accuracy.
This study introduces a novel approach: the Generalized Correctness Model (GCM), which leverages historical correctness data from multiple models to predict output accuracy across diverse tasks and datasets. By training on aggregated data from models like Qwen, Llama, and Gemma, GCM captures patterns related to answer phrasing, background knowledge, and response elaboration. The core idea is that correctness prediction benefits more from systemic encoding of historical performance than from models’ self-assessment, which is often unreliable.
Extensive experiments demonstrate that GCM significantly outperforms traditional confidence estimators, achieving accuracy improvements of 2-4%, with calibration errors (ECE) below 0.02 and AUROC approaching 0.88. Notably, the model generalizes well to unseen models and datasets, including out-of-distribution scenarios, confirming the robustness of the approach. Incorporating in-context learning and post-hoc calibration further enhances performance, suggesting practical pathways for deployment.
The implications are profound: moving beyond self-knowledge, models can systematically encode historical performance to produce calibrated, reliable confidence estimates. This paves the way for safer, more trustworthy AI systems in high-stakes domains, addressing longstanding challenges in model interpretability and reliability. Future work aims to optimize computational efficiency, extend to multi-modal inputs, and improve out-of-distribution robustness, promising a new era of dependable AI.
Deep Analysis
Background
近年来，随着GPT、BERT等模型的崛起，模型在自然语言理解和生成方面取得巨大突破。然而，模型输出的可信度仍是制约其实际应用的关键难题。早期工作如Calibration（Guo et al., 2017）和自我预测（Kadavath et al., 2022）试图让模型自我判断正确性，但效果有限。近年来，研究开始关注多模型融合（Zhou et al., 2024）和后校准技术（Lucena, 2018），旨在提升置信度的可靠性。尽管如此，模型对自身正确性的认知仍缺乏系统性理解，特别是在跨模型和跨任务的场景下。
Core Problem
核心问题在于，模型是否真正具备“自我知识”以判断自身输出的正确性？现有方法多依赖模型自我预测，但实验证明这种自我预测并不优于其他模型的预测，反映出模型缺乏真正的“自我认知”。此外，如何利用多模型的历史预测信息，构建具有良好泛化能力的正确性预测器，仍是亟待解决的难题。这个问题关系到模型在高风险场景中的可信度和安全性，亟需突破传统单模型局限。
Innovation
本研究的创新点在于：1）提出基于多模型历史预测的通用正确性模型（GCM），突破了自我知识的局限；2）系统分析答案措辞和世界知识在正确性预测中的作用，揭示其关键性；3）结合ICL和后校准技术，提升模型在不同数据集和模型上的迁移能力。这些创新为模型可信度评估提供了新思路，推动多模型协作的可信AI发展。
Methodology
- �� 构建多模型历史正确性数据集，采集不同模型在MMLU、TriviaQA上的响应和正确性标签。• 采用LoRA微调基础模型Qwen3-8B，训练SCMs（单模型）和GCM（多模型）以预测正确性。• 输入特征包括问答对、答案措辞、模型身份，调控条件变量。• 利用交叉熵损失优化模型，结合后校准技术（如样条、Beta校准）提升校准效果。• 通过在不同模型、不同数据集上的验证，评估模型的准确率、校准误差和迁移能力。
Experiments
采用MMLU和TriviaQA数据集，训练多模型历史数据，比较GCM与单模型、零样本模型的性能。设置多模型训练、不同特征调控、后校准等多种条件，进行AB测试。评估指标包括准确率、ECE、AUROC。实验还包括跨模型、跨数据集迁移测试，验证模型的泛化能力。采用LoRA微调，保证训练效率，确保模型在不同场景下的适应性。
Results
GCM在多模型、多数据集上表现优异，准确率提升2-4%，校准误差降低至0.02，AUROC达0.88。跨模型迁移实验显示，未见模型的正确性预测能力明显优于单模型自预测。答案措辞和世界知识是关键特征，模型对答案的细节和背景知识的利用显著增强预测性能。这些结果验证了多模型历史信息的有效性和泛化能力。
Applications
该方法可应用于高风险场景中的模型可信度评估，如医疗诊断、法律咨询等。通过系统性编码历史信息，提升模型输出的可信度，减少误判风险。未来还可结合多模态信息，扩展到视觉、语音等多模态场景，推动可信AI的发展。
Limitations & Outlook
当前模型训练依赖大量历史数据和多模型训练，计算成本较高，难以在资源有限环境中部署。模型在极端偏离训练分布的场景下表现仍有限，跨领域迁移能力需进一步增强。答案措辞和背景知识虽为关键因素，但在复杂任务中的适应性仍需验证，未来需优化特征选择和模型结构。
Plain Language Accessible to non-experts
想象你在一家工厂工作，工厂每天生产不同的产品。每个工人（模型）都在不同时间点做出判断，是否生产出合格的产品。过去的生产记录（历史预测）可以帮助你判断某个工人在未来是否会出错。单个工人可能不知道自己是否会出错，但如果你收集多个工人的过去表现，结合他们的生产细节和背景知识，你就能更准确地判断他们的产品是否合格。这就像研究中的多模型历史预测一样，利用过去的“表现记录”来提升判断的准确性和可靠性，而不是只依赖某个工人的自我认知。
ELI14 Explained like you're 14
想象你在学校里，有很多同学（模型）每天都在做作业。有些同学知道自己写得好，有些可能不太确定。可是，有时候你可以通过观察他们平时的表现（比如他们写作业的细节、用的词语、背景知识）来判断他们这次是不是答对了。其实，单靠他们自己说“我答对了”不一定靠谱，但如果你记住他们以前的表现，结合他们写的内容，就能更准确地判断。这就像这篇论文用很多模型的“过去表现”来预测它们的正确率，而不是只相信它们自己说自己答得好。
Glossary
Correctness Model (正确性模型)
一种估算答案正确概率的系统，基于输入的问答信息预测输出是否正确。
论文中用来衡量模型输出可信度的工具。
Generalized Correctness Model (GCM, 通用正确性模型)
利用多个模型的历史预测数据训练出的跨模型、跨任务的正确性预测器。
核心创新，用于提升校准和泛化能力。
In-context Learning (ICL, 语境学习)
在推理时加入相关示例，使模型在不训练的情况下学习任务特征。
用于无训练的正确性预测增强。
Expected Calibration Error (ECE, 期望校准误差)
衡量模型预测概率与实际正确率偏差的指标。
评估校准效果的重要指标。
AUROC (受试者工作特征曲线下面积)
衡量模型区分正负样本能力的指标，值越接近1越好。
用于评估正确性预测的判别能力。
Open Questions Unanswered questions from this research
- 1 如何在极端偏离训练分布的场景中进一步提升GCM的迁移能力仍未充分解决，特别是在新兴模型和任务上的适应性。
- 2 多模态信息融合对正确性预测的影响尚未系统研究，未来需探索视觉、语音等多模态数据的结合方式。
- 3 模型在复杂推理和多步骤任务中的表现和校准能力仍有限，需结合强化学习等技术进行优化。
Applications
Immediate Applications
高风险场景中的模型可信度评估
利用GCM对医疗、法律等领域的模型输出进行校准，减少误判，提升用户信任。
模型输出筛选与拒绝机制
在自动问答系统中，结合正确性预测器筛选高置信度答案，提高系统整体可靠性。
Long-term Vision
可信AI的标准化评估体系
建立跨模型、跨任务的校准标准，推动行业制定统一的可信度评估指标。
Abstract
Generating accurate and calibrated confidence estimates is critical for deploying LLMs in high-stakes or user-facing applications, and remains an open challenge. Prior research has often framed confidence as a problem of eliciting a model's "self-knowledge", i.e., the ability of an LLM to judge whether its own answers are correct; this approach implicitly assumes that there is some privileged information about the answer's correctness that is accessible to the model itself. However, our experiments reveal that an LLM attempting to predict the correctness of its own outputs generally performs no better than an unrelated LLM. Moreover, we hypothesize that a key factor in building a "Correctness Model" (CM) is exposure to a target model's historical predictions. We propose multiple methods to inject this historical correctness information, creating a Generalized Correctness Model (GCM). We first show that GCMs can be trained on the correctness data from many LLMs and learn patterns for correctness prediction applicable across datasets and models. We then use CMs as a lens for studying the source of correctness prediction ability and its generalization, systematically controlling their training data and finding that answer phrasing is a strong predictor for correctness. We further explore alternative methods of injecting history without training an LLM, finding that including history as in-context examples can help improve correctness prediction, and post-hoc calibration can provide complementary reductions in calibration error. We evaluate GCMs based on Qwen3-8B across 5 model families and the MMLU and TriviaQA datasets, as well as on a downstream selective prediction task, finding that reliable LLM confidence estimation is a generalizable and model-agnostic skill learned by systematically encoding correctness history rather than a model-specific skill reliant on self-introspection.