RISA: Response Inspection and Selective Actions for Refusal Calibration in Large Language Models
RISA enhances LLM safety by calibrating refusal behavior at inference time.
Key Findings
Methodology
RISA is an inference-time framework that inspects initial responses and selectively corrects refusal errors without updating the base model. It first uses fixed rules to assign refusal scores for clear cases. For unmatched cases, it derives a refusal score from the final-layer prompt hidden state using a calibrated linear probe. RISA separately calibrates the probe score, representation-support boundary, and action thresholds to adapt to different base models.
Key Results
- RISA improves refusal reliability across three LLMs and multiple benchmarks while maintaining model utility.
- Experimental results show significant improvement in refusal reliability with minimal impact on overall model utility.
- RISA's intervention significantly reduces refusal error rates, especially in handling sensitive prompts.
Significance
RISA offers a practical solution for refusal calibration in LLMs. By performing selective interventions at inference time, RISA enhances refusal behavior reliability without altering model parameters. This approach not only improves model safety but also maintains its utility in handling benign prompts, addressing issues of over-refusal and under-refusal.
Technical Contribution
RISA's technical contribution lies in its inference-time calibration mechanism, achieving precise refusal calibration through a combination of fixed rules and hidden state probes. Unlike existing methods, RISA requires no model parameter updates, providing a flexible and efficient refusal calibration method.
Novelty
RISA is the first to achieve refusal calibration at inference time by combining rules and probes. Compared to traditional training-time calibration methods, RISA offers a more efficient solution without additional computational resources.
Limitations
- RISA may misjudge extremely complex prompts due to limited coverage of fixed rules and probes.
- Calibration effectiveness may vary when the base model's representation space changes significantly.
Future Work
Future research directions include expanding RISA's rules and probes to cover a wider range of prompt types and exploring ways to enhance calibration accuracy without increasing computational complexity.
AI Executive Summary
Large language models often face refusal errors when handling user prompts, leading to over-refusal or under-refusal. Existing methods primarily rely on training-time safety alignment, which requires substantial computational resources. RISA proposes an inference-time calibration framework that inspects initial responses and selectively corrects refusal errors, avoiding updates to base model parameters.
The core of RISA lies in combining fixed rules and hidden state probes to assign refusal scores for clear and unmatched cases, respectively. By calibrating probe scores, representation-support boundaries, and action thresholds, RISA adapts to different base models, offering a flexible refusal calibration solution.
Experimental results demonstrate that RISA significantly improves refusal reliability across multiple benchmarks while maintaining overall model utility. Although limitations exist in handling complex prompts, RISA provides a practical and efficient solution for enhancing LLM safety.
Deep Analysis
Background
Large language models play a crucial role in various real-world scenarios, but their refusal behavior reliability remains a challenge. Existing training-time alignment methods can improve refusal reliability but require substantial computational resources and training time. Inference-time calibration methods are gaining attention for their flexibility and efficiency.
Core Problem
Large language models may exhibit over-refusal or under-refusal when rejecting harmful prompts. Over-refusal prevents users from obtaining useful information, while under-refusal may expose harmful information. Improving refusal behavior reliability without updating model parameters is a pressing issue.
Innovation
RISA achieves inference-time refusal calibration by combining fixed rules and hidden state probes. Fixed rules handle clear cases, while hidden state probes address unmatched cases. By separately calibrating probe scores, representation-support boundaries, and action thresholds, RISA adapts to different base models.
Methodology
- �� Use fixed rules to assign refusal scores for clear cases.
- �� For unmatched cases, derive refusal scores from the final-layer prompt hidden state using a hidden state probe.
- �� Separately calibrate probe scores, representation-support boundaries, and action thresholds to adapt to different base models.
- �� At runtime, combine prompt scores with initial refusal status and intervene only when necessary.
Experiments
Experiments were conducted on three large language models and multiple benchmarks to evaluate RISA's performance in refusal reliability and model utility. Benchmarks included sensitive and harmful prompts, with results showing significant improvement in refusal reliability.
Results
RISA significantly improves refusal reliability while maintaining overall model utility. Experimental results show a notable reduction in refusal error rates, particularly in handling complex prompts.
Applications
RISA can be directly applied to scenarios requiring improved refusal reliability, such as online chatbots and intelligent assistants. By performing selective interventions at inference time, RISA enhances safety without affecting overall model utility.
Limitations & Outlook
RISA may misjudge extremely complex prompts due to limited coverage of fixed rules and probes. Additionally, calibration effectiveness may vary when the base model's representation space changes significantly.
Plain Language Accessible to non-experts
Imagine an intelligent assistant that needs to distinguish between harmful and harmless questions. RISA acts like a smart filter for this assistant. It first uses simple rules to judge if a question is harmful, much like we use common sense to judge if a question is appropriate. If these rules can't decide, RISA delves into the hidden information of the question, like an experienced teacher who can sense a student's true intentions from their expressions and tone. This way, RISA ensures the assistant only answers safe questions without changing the assistant itself.
ELI14 Explained like you're 14
Hey there! Imagine you're playing a super cool game where you're the guardian of a castle, protecting it from bad guys. RISA is like your secret weapon that helps you figure out who's bad and who's good. First, it uses some simple rules to quickly decide, just like you use your eyes to spot enemies. If those rules aren't enough, RISA uses its super sensing ability to dive deep into the enemy's intentions, just like you use your heart to feel what the enemy is thinking. This way, you make sure only the good guys get in, and the bad guys stay out!
Glossary
Large Language Model (LLM)
A large neural network model capable of understanding and generating natural language.
Used in the paper to process user prompts and generate responses.
Refusal Calibration
Adjusting a model's refusal behavior to improve its safety and reliability.
Achieved through inference-time calibration in RISA.
Inference-time Calibration
Adjusting model behavior through selective interventions during inference without updating model parameters.
RISA uses inference-time calibration to enhance refusal reliability.
Fixed Rules
A predefined set of rules used to identify clear cases.
RISA uses fixed rules to assign refusal scores for clear cases.
Hidden State Probe
A tool that extracts information from a model's hidden state to determine prompt intent.
RISA uses hidden state probes to handle unmatched cases.
Open Questions Unanswered questions from this research
- 1 How to expand RISA's rules and probes to cover a wider range of prompt types without increasing computational complexity.
Applications
Immediate Applications
Online Chatbots
RISA can be used to improve the refusal reliability of online chatbots, ensuring safety when handling user prompts.
Long-term Vision
Intelligent Assistants
In the future, RISA can be applied to intelligent assistants to enhance refusal behavior reliability without affecting overall utility.
Abstract
Reliable refusal behavior requires Large Language Models (LLMs) to reject harmful prompts with only answering benign ones. Incorrect refusal behavior can either expose users to harmful responses or prevent users from obtaining useful answers. Training-time alignment improves refusal behavior by updating model parameters with safety data, but requires additional computation and training. In contrast, inference-time alignment aims to modify LLM behavior during inference without updating the underlying model parameters. Existing inference-time methods mainly rely on in-context safety prompting, activation steering, or decoding control. However, most of them intervene without first determining whether the initial response is already appropriate, potentially altering a correct refusal or a useful answer. Effective selective intervention therefore requires identifying prompt intent beyond sensitive keywords, covering semantic variations that fixed rules may miss, and adapting the verifier to different base models. To address these challenges, we propose Response Inspection and Selective Actions (RISA), an inference-time framework that inspects the initial response and selectively corrects refusal errors without updating the base model. RISA first uses fixed contextual rules to assign refusal scores to clear cases. For unmatched cases, it derives a refusal score from the final-layer prompt hidden state using a calibrated linear probe. To adapt to different base models, RISA separately calibrates the probe score, representation-support boundary, and action thresholds. At runtime, RISA combines the prompt score with the initial refusal status and applies an action policy to intervene only when necessary. Experimental results demonstrate that RISA improves refusal reliability while largely preserving model utility, offering a practical solution for response-aware refusal calibration in LLMs.