Distinguishing the Knowable from the Unknowable with Language Models

The evolution of large language models (LLMs) such as GPT, BERT, and LLaMA has significantly advanced NLP tasks. Prior work on uncertainty quantification includes Bayesian neural networks, ensemble methods, and calibration techniques, but these often face scalability and computational challenges. Recent efforts have explored internal model representations, like probing hidden states for confidence signals, yet these are limited to specific tasks or models. The need for scalable, domain-agnostic, and label-free methods remains pressing, especially for generative tasks prone to hallucinations and unreliable outputs. This research builds on these foundations, aiming to uncover intrinsic uncertainty signals within models, enabling more interpretable and safer AI systems.

A key challenge in deploying LLMs is distinguishing between knowledge-based (epistemic) uncertainty and inherent data randomness (aleatoric). Existing calibration methods often cannot separate these sources, leading to overconfidence or underconfidence. Moreover, in free-form text generation, the absence of explicit ground-truth probabilities complicates uncertainty estimation. The core problem is developing methods that can reliably identify when a model’s uncertainty reflects a genuine lack of knowledge versus natural ambiguity, without relying on extensive labeled data or fine-tuning. This distinction is crucial for reducing hallucinations and improving trustworthiness.

The paper introduces a novel framework combining model comparison and internal representation analysis. It trains linear probes on frozen embeddings to predict the confidence of larger models, using the assumption that bigger models are less epistemically uncertain. Additionally, it proposes an unsupervised ICLT method, which analyzes the model’s repetition behavior during multiple inference passes to infer uncertainty. These approaches are innovative because they do not require labeled data, leverage existing model states, and demonstrate transferability across domains. This contrasts with traditional Bayesian or ensemble approaches that are computationally intensive and less scalable.

• �� Extract intermediate layer embeddings from pre-trained models like LLaMA and Pythia.
• �� Train linear and nonlinear probes on these embeddings to predict the large model’s confidence, using labels generated by model comparison.
• �� Generate labels based on the large model’s predictive entropy, with low entropy indicating high confidence.
• �� Validate probes across multiple datasets, including Wikipedia, code, and multilingual texts, ensuring transferability.
• �� Develop the ICLT: for a given prompt, generate multiple next-token candidates, prepend each to the prompt, and observe the model’s repetition behavior.
• �� Measure the degree of token repetition or entropy change to infer whether the model’s uncertainty is epistemic.
• �� Evaluate using metrics like AUC and accuracy, compare with baselines such as entropy thresholds and random guessing.

Experiments utilize open-source models (LLaMA, Llama 2, Pythia) on datasets like Wikipedia, Pile, and EuroParl. The probes are trained with early stopping, hyperparameters tuned for optimal performance. The evaluation includes in-distribution and out-of-distribution tests, cross-model transfer, and ablation studies on embedding layers. The unsupervised ICLT is tested by analyzing the repetition patterns across multiple inference runs. Baselines include entropy-based heuristics and random classifiers, demonstrating the robustness and superiority of the proposed methods. Results consistently show high AUROC (>0.9) for supervised probes and significant detection capability for ICLT.

Supervised linear probes achieve AUROC > 0.9 in predicting large model confidence, with transferability across datasets and models, including out-of-distribution scenarios (AUROC > 0.8). The unsupervised ICLT method also performs well, indicating that internal repetition behavior correlates with epistemic uncertainty. Deeper layer embeddings encode more reliable signals, and the methods generalize across domains like code and multilingual texts. These results confirm that internal model states contain meaningful uncertainty representations, accessible without supervision.

This work enables more reliable AI deployment by identifying uncertain tokens during generation, reducing hallucinations, and improving fact-checking. It can be integrated into real-time systems for adaptive response filtering, confidence-based reranking, or human-in-the-loop interventions. The methods also support model debugging, safety testing, and transparency efforts, fostering trust in AI outputs. Long-term, these techniques could underpin self-aware models capable of explaining their confidence levels, facilitating safer AI in critical applications like healthcare, legal, and autonomous systems.

The approach depends on the assumption that larger models are less epistemically uncertain, which may not always hold due to their own internal uncertainties. It primarily targets token-level uncertainty, neglecting broader semantic or sequence-level ambiguities. Performance may degrade in highly noisy or adversarial scenarios. Additionally, reliance on existing models as proxies introduces bias, and the methods may need adaptation for multimodal or multi-task settings. Future work should address these limitations by incorporating multi-layer, multi-modal signals and refining the interpretability of internal representations.