SIMBA UQ: Similarity-Based Aggregation for Uncertainty Quantification in Large Language Models

Recent advances in pretraining have propelled LLMs like GPT, BERT, and LLaMA to achieve remarkable performance across NLP tasks. However, their outputs often lack reliable confidence estimates, which are essential for safety-critical applications. White-box methods leveraging internal probabilities or activations have shown promise but face limitations due to model opacity and evolving architectures. Black-box techniques, focusing on output sample analysis, have gained traction, including consistency-based approaches and semantic similarity measures. Despite progress, challenges remain in calibrating confidence scores, especially for complex, multi-turn, or structured outputs like SQL queries. Existing methods often depend on extensive fine-tuning or assumptions about semantic similarity, limiting their generality. This context motivates the development of a unified, flexible framework capable of robust uncertainty quantification without internal model access.

The core challenge is to accurately assess the correctness of LLM outputs in a black-box setting, where internal model information is unavailable. Existing techniques either rely on verbalized confidence, which struggles with complex outputs, or on semantic similarity, which may not correlate well with correctness in structured tasks. Additionally, many methods require large labeled datasets for training, which is costly and impractical. The need for a scalable, task-agnostic, and minimally supervised approach to reliably estimate confidence scores remains unmet. Addressing this gap is crucial for deploying LLMs in real-world scenarios where trustworthiness and safety are paramount.

The paper introduces the SIMBA framework, a novel approach that leverages sample similarities to estimate confidence. Its key innovations include: 1) formalizing confidence as a similarity aggregation problem, 2) developing Bayesian and supervised classification strategies that require only limited labeled data, 3) demonstrating the framework's versatility across diverse tasks and models. Unlike prior methods focused solely on semantic similarity or internal probabilities, this approach emphasizes a functional relation between sample consistency and correctness, enabling effective calibration in complex, real-world settings. The framework's simplicity, flexibility, and empirical robustness mark a significant advancement in black-box UQ.

• �� Generate multiple samples per input using temperature sampling, ensuring response diversity.
• �� Compute pairwise similarities between samples using metrics like Jaccard or ROUGE.
• �� Aggregate similarities to estimate confidence: • Simple average: mean similarity as confidence.
• �� Bayesian update: model similarity distributions conditioned on correctness, compute posterior probabilities.
• �� Classification: train a random forest classifier using similarity features and optional generative scores.
• �� Use limited labeled data to fine-tune confidence models, ensuring minimal supervision.
• �� Validate across datasets, tasks, and models, adjusting hyperparameters for optimal calibration.
• �� Evaluate using ACE, ATS, and AUROC metrics, comparing against baselines like spectral clustering and verbalized confidence methods.

The experimental setup involves datasets like CoQA, TriviaQA, XSum, SamSum, Spider, and BIRD, covering QA, summarization, and SQL tasks. Models include LLaMA 3.3 70B, Granite 8B, and Codellama 34B. For each, multiple samples are generated at temperatures from 0.25 to 1.5. The key evaluation metrics are ACE for calibration, ATS for selection accuracy, and AUROC for predictive power. Baselines include avg. log prob, spectral clustering, and verbalized confidence. The training involves limited labeled data for confidence model fine-tuning, with ablation studies to assess feature contributions. Results consistently show superior calibration and accuracy, validating the framework's effectiveness.

Across all datasets, the proposed similarity-based methods outperform baselines, reducing ACE error by over 20% and achieving AUROC scores above 0.9. On CoQA, ACE dropped from 0.272 to 0.043, and AUROC increased from 0.72 to 0.91, demonstrating excellent confidence calibration. Combining similarity features with generative scores in classifiers further improved results, especially for structured outputs like SQL. The methods maintained robustness across different models and response lengths, confirming their general applicability. Ablation results indicated that integrating multiple features yields the best performance, highlighting the importance of multi-modal information for reliable UQ.

This framework enables deployment in real-world NLP systems requiring high-confidence outputs, such as customer service chatbots, automated content moderation, and database query systems. Its black-box nature allows easy integration without internal model modifications, suitable for diverse architectures and tasks. The approach can be extended to multimodal scenarios, combining text, images, and audio, to assess confidence across data types. It also facilitates active learning by identifying uncertain samples for annotation, reducing labeling costs. Long-term, it supports the development of trustworthy AI systems with calibrated, reliable outputs, fostering user trust and safety.

The effectiveness depends on the quality and diversity of generated samples; poor sampling reduces confidence accuracy. In out-of-distribution scenarios, calibration may degrade, requiring further robustness enhancements. The supervised classifier approach still needs some labeled data, which may be costly in certain domains. The choice of similarity metric influences performance, necessitating task-specific tuning. Future work should address these limitations by exploring unsupervised methods, advanced similarity measures, and robustness under distribution shifts.