What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics
Proposes a training-free method using intermediate-layer entropy trajectories to detect jailbreak prompts, achieving AUROC up to 0.94 across Llama, Qwen, and Gemma.
Key Findings
Methodology
The study analyzes token-level predictive entropy trajectories across model layers using the logit lens, extracting static (mean, variance) and dynamic (Kendall τ, Spearmanρ, monotonicity) features. Emphasizing the importance of dynamic trends, the approach identifies structured entropy changes associated with jailbreak prompts. Experiments on Llama-3.1-8B, Qwen-3-8B, and Gemma-7B validate the effectiveness across multiple adversarial benchmarks, showing that intermediate layers harbor the strongest signals, with performance peaking at 50-85% depth and degrading at the final layer.
Key Results
- Dynamic trend features such as monotonicity and rank correlation consistently achieve high AUROC (~0.94), outperform static features (mean entropy AUROC ~0.67-0.89). These features are robust across models, with AUROC values nearly identical (~0.80) for Kendall τ and Spearmanρ, indicating stable structural signals. The detection performance peaks at intermediate layers, with a sharp decline at the final layer, suggesting that the core structural differences are suppressed or reorganized before output.
- Across architectures, the rank-based features show high cross-model consistency, with AUROC differences less than 0.02, confirming their transferability. The experiments demonstrate that the entropy dynamics encode a stable and architecture-agnostic signal for identifying jailbreak prompts, with no additional training required. Static entropy measures, however, vary significantly across models, highlighting their limited generalization.
- The results underscore the importance of dynamic, structure-based features over static statistics, revealing that the most discriminative signals are concentrated in the middle layers, which are crucial for understanding internal model behavior related to safety. This insight opens new avenues for internal model interpretability and security.
Significance
This work advances understanding of internal model signals related to safety, moving beyond output-based detection to internal entropy dynamics. It offers a novel, training-free approach that leverages the structured nature of intermediate representations, providing a robust, architecture-agnostic tool for real-time jailbreak detection. Such insights are vital for deploying large models safely in industry, enabling proactive defense mechanisms that do not rely on retraining or fine-tuning. The findings also deepen theoretical understanding of how harmful behaviors are encoded within model representations, guiding future research in model interpretability and robustness.
Technical Contribution
The paper introduces a novel framework for analyzing internal model signals via token-level entropy trajectories, emphasizing dynamic trend features like Kendall τ, Spearmanρ, and monotonicity. It demonstrates that these features are more stable and transferable than static entropy measures, and that the discriminative signal is concentrated in intermediate layers. The approach is model-agnostic, requiring no additional training, and leverages the logit lens for efficient extraction. The empirical validation across multiple architectures and benchmarks confirms its robustness, offering a new interpretability and security tool for large language models.
Novelty
This is the first comprehensive study to leverage intermediate-layer entropy dynamics for jailbreak detection, emphasizing the importance of structured, dynamic features over static measures. Unlike prior work focusing on output or prompt features, this approach reveals that the core signals are embedded within the model’s internal representations, especially in middle layers. The use of rank-based trend measures as stable, transferable indicators constitutes a significant innovation, opening new directions for model interpretability and security.
Limitations
- The method relies on internal activation signals, which may vary with different training regimes, architectures, or fine-tuning procedures, potentially limiting generalization. Its effectiveness in real-world, complex scenarios remains to be validated.
- In highly sophisticated or adaptive adversarial settings, structural signals might be suppressed or masked, reducing detection accuracy. The approach also assumes the presence of detectable entropy trends, which may not hold in all cases.
- Computational costs are minimal for inference but may increase with larger models or higher probe layer sampling, and the method currently focuses on static models without adaptive or online updating capabilities.
Future Work
Future research could integrate multi-modal signals or multi-layer fusion to enhance robustness. Developing adaptive algorithms that dynamically select optimal probe layers based on context, and extending the framework to real-time deployment scenarios, are promising directions. Additionally, exploring training strategies that embed entropy trend signals into model parameters could further improve detection capabilities, fostering safer AI systems.
AI Executive Summary
The rapid deployment of large language models (LLMs) has revolutionized natural language processing, yet their safety remains a critical concern. Adversaries craft prompts—so-called jailbreaks—that bypass safety mechanisms, leading to potentially harmful outputs. Existing defenses often rely on external filters or output analysis, which are limited by their surface-level perspective and lack internal interpretability.
This study introduces a novel, training-free approach focusing on the internal representations of models. By analyzing the entropy trajectories of token predictions across different layers, the authors reveal that structured, dynamic patterns emerge predominantly in intermediate layers when models process jailbreak prompts. Using the logit lens, they project hidden states into probability distributions, extracting features that capture how uncertainty evolves as tokens are processed.
The core insight is that while static entropy measures are inconsistent across models, dynamic trend features—such as monotonicity and rank correlation—are highly effective. Experiments across Llama-3.1-8B, Qwen-3-8B, and Gemma-7B demonstrate that these features achieve AUROC scores up to 0.94, consistently distinguishing jailbreak prompts from benign ones without any additional training. The detection signal peaks at middle layers (50-85% depth) and diminishes at the output layer, indicating that the structural markers of harmful intent are most prominent before final processing.
This work offers a significant leap in understanding how harmful behaviors are internally encoded, providing a robust, architecture-agnostic tool for real-time security monitoring. Its implications extend to safer deployment of AI, guiding future research towards internal interpretability and proactive defense mechanisms. Despite some limitations in complex scenarios, this approach paves the way for more transparent and resilient AI systems, fostering trust and safety in AI applications.
Deep Dive
Abstract
Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training. While most defenses operate at the prompt or output level, it remains unclear how harmful intent is encoded within the model's internal representations. We investigate this question by analyzing token-level predictive entropy trajectories across layers of a frozen LLM using the logit lens. We find that static aggregate statistics of prompt-level entropy (e.g., mean, variance) carry little discriminative signal, whereas features capturing how entropy evolves across token positions, such as monotonic rank-based trend scores, are substantially more informative. Importantly, this signal is not uniform across model depth: it is concentrated in intermediate layers and degrades at the final layer, indicating that jailbreak-relevant structure is most pronounced in mid-network representations rather than at the output head. Across multiple models (Llama, Qwen, Gemma) and adversarial benchmarks, these entropy dynamics provide architecture-consistent separation without additional training. Together, our findings show that jailbreak behavior is reflected in structured intermediate uncertainty dynamics, clarifying both which entropy-derived features encode harmful intent and where in the network that signal is most pronounced.