What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models
Introduces Internal Compliance Score (ICS), a training-free activation readout that reveals rule blindness in compliance detectors, validated across multiple benchmarks and models.
Key Findings
Methodology
This paper develops a training-free activation readout called Internal Compliance Score (ICS), which leverages a linear projection of internal model activations calibrated with ten labeled pairs. The approach involves computing the mean activation vectors for adherent and violating texts, deriving a normalized difference vector representing the rule, and projecting new samples onto this vector to produce a compliance score. This method circumvents the need for training classifiers, enabling rapid, scalable auditing across multiple models and benchmarks. The authors compare ICS against various guard models (e.g., Llama Guard 3, Qwen3Guard) and activation probes, demonstrating that ICS achieves AUROC scores of 0.952 in in-domain tests and maintains robust performance (average AUROC 0.728) in cross-domain settings. They further employ a purpose-built cross-rule counterfactual benchmark to systematically expose the rule blindness phenomenon, where detection accuracy remains unchanged despite rule modifications, indicating superficial reliance on surface features rather than rule content.
Key Results
- In in-domain evaluation, ICS attains an AUROC of 0.952, outperforming traditional guard models such as Llama Guard 3 (AUROC 0.749) and Qwen3Guard (AUROC 0.908). Cross-domain transfer tests show an average AUROC of 0.728, surpassing lexical baselines like TF-IDF (0.728), illustrating strong generalization. The counterfactual experiments confirm that deleting or swapping rules affects detection scores by less than 0.003, validating the rule blindness issue. In practical deployment, ICS improves the pass rate of candidate response ranking by 5.2 percentage points on the IFEval benchmark, although white-box adversarial attacks can diminish this advantage, highlighting the need for further robustness enhancements.
- The systematic cross-rule and scenario manipulations reveal that most compliance detectors, including policy-conditioned guards, primarily respond to superficial cues or rule polarity rather than the actual rule content. This exposes a fundamental flaw: detectors are often rule-agnostic, risking false assurances in real-world regulatory compliance. The experiments also demonstrate that ICS's low-cost, calibration-once approach can be effectively applied across multiple models and domains, providing a scalable solution for large-scale compliance auditing.
- Furthermore, the authors show that ICS can be re-calibrated efficiently after model fine-tuning, maintaining high AUROC scores, and that it can be used to rank candidate responses effectively. The comparison with trained guard models indicates that ICS not only matches but often exceeds their detection performance in zero-shot and cross-domain scenarios, making it a promising tool for real-world deployment where cost and adaptability are critical. However, the presence of white-box attacks underscores the importance of developing complementary robustness strategies.
Significance
This research critically exposes the limitations of current compliance detection methods, which often rely on superficial cues rather than true rule understanding. The discovery of rule blindness has profound implications for legal and ethical AI deployment, especially in sensitive sectors like finance, healthcare, and content moderation. The proposed ICS method offers a low-cost, scalable, and adaptable alternative that can be deployed in real-time monitoring systems. By systematically validating the rule content’s weak influence on detection outcomes, the work urges the community to rethink the design of rule-conditioned models and detectors, emphasizing the need for more robust, explainable, and rule-aware approaches. This paradigm shift could significantly enhance the reliability and trustworthiness of AI systems in regulated environments.
Technical Contribution
The paper introduces a novel, training-free activation readout—Internal Compliance Score (ICS)—which employs a simple yet effective difference-of-means linear projection calibrated with minimal labeled data. Unlike traditional classifiers that require extensive training, ICS directly reads the model’s internal activations, making it computationally inexpensive and easily adaptable to new models or rules. The authors also develop a systematic cross-rule counterfactual benchmark, revealing the pervasive rule blindness phenomenon across various guard architectures and robustness checks. This benchmark involves crossing rules with different scenarios to test whether detection relies on superficial features or genuine rule understanding. The combination of a low-cost, calibration-based approach with rigorous counterfactual validation advances the state-of-the-art in scalable, rule-sensitive compliance monitoring.
Novelty
This work is the first to demonstrate that existing compliance detectors, including policy-conditioned guards and activation probes, are largely rule-blind, responding primarily to surface cues rather than rules themselves. The introduction of ICS as a training-free, calibration-based activation readout represents a significant departure from prior methods that depend on trained classifiers or complex neural networks. The systematic use of cross-rule counterfactual benchmarks to expose rule blindness is a novel evaluation paradigm, providing a rigorous and scalable way to assess the true rule sensitivity of compliance detectors. These innovations collectively push the field toward more robust, interpretable, and cost-effective compliance monitoring solutions.
Limitations
- Despite its advantages, ICS’s rule insensitivity means it cannot reliably detect subtle rule violations or rule micro-tuning, which may be critical in high-stakes environments. Its effectiveness diminishes if adversaries understand and exploit its reliance on superficial cues.
- The method’s calibration depends on a small labeled set, which, while inexpensive, may not fully capture complex or evolving rules, potentially leading to false negatives or positives in dynamic regulatory contexts.
- The experiments primarily focus on specific models and benchmarks; broader validation across diverse architectures, languages, and real-world scenarios remains necessary to confirm general applicability and robustness.
Future Work
未来的研究将致力于增强ICS对规则内容的敏感性，探索多模态信息融合和上下文理解技术，以提升检测的鲁棒性和可解释性。同时，构建更全面的反事实基准体系，系统性地检测和缓解规则盲问题。未来还应结合人类专家知识，设计更复杂的反事实变换，提升检测器对规则微调和微变化的敏感度。此外，结合对抗训练和多模态信息，将有助于增强模型在实际部署中的安全性和可靠性，推动合规检测技术迈向更高的水平。
AI Executive Summary
在人工智能技术飞速发展的今天，确保模型在实际应用中遵守法律法规和伦理准则成为关键挑战。传统的合规检测方法多依赖训练有素的守卫模型，这些模型成本高昂，难以快速适应法规变更或模型微调，限制了其在大规模、多场景环境中的应用效果。本文提出了一种低成本、无需训练的激活读取机制——内部合规评分（ICS），旨在解决当前检测器普遍存在的“规则盲”问题。
ICS通过分析模型内部激活状态，利用差异化线性投影，快速评估模型输出是否符合特定规则。该方法只需少量标注样本进行校准，便能在多模型、多基准场景中实现高效检测，AUROC最高达0.952，显著优于传统守卫模型。同时，作者设计了交叉规则的反事实基准，系统性地揭示了检测器对规则内容的敏感度不足，验证了规则盲现象。这一发现提醒行业关注检测器的本质局限，推动模型监控技术的革新。
在广泛的实验中，ICS展现出优异的迁移能力和低成本优势，能够在未见分布上保持良好性能，验证了其在实际部署中的潜力。尽管如此，ICS也存在规则内容敏感度不足、易被白盒对抗攻击等局限。未来，作者建议结合多模态信息和更复杂的反事实设计，提升检测的鲁棒性和解释性，为模型安全监控提供更可靠的技术基础。这项工作不仅丰富了模型监控的理论体系，也为行业实践提供了新思路，推动AI安全向更高层次迈进。
Deep Dive
Abstract
Regulatory compliance monitoring in deployed language models is increasingly implemented as a legal and audit control, checking model outputs against written rules spanning data protection, healthcare, financial regulation, and platform policy. Such monitoring is meaningful only if a detector's verdict depends on the stated rule rather than on surface features of the scenario. We show this condition fails across the current class of compliance detectors, a failure we call rule blindness. Deleting, permuting, or substituting the governing rule leaves detection accuracy unchanged for every guard and activation probe we test, including a policy-conditioned guard that correctly cites the governing clause yet barely changes its verdict when that clause is swapped for its permissive counterpart. A purpose-built benchmark crossing two rules with two scenarios, so that neither alone predicts the label, confirms the failure under a design no prior benchmark rules out, and shows that step by step reasoning, not any fast detector we test, is what escapes it. Auditing at scale requires a retraining-free detector, so we introduce the Internal Compliance Score (ICS): a training-free activation readout calibrated from ten labelled pairs and scored by a single projection. We hold ICS to the same scrutiny as the guards it audits: a pre-registered criterion for beating trivial baselines is not met, and a bag-of-words model matches its pooled generalisation exactly. It remains useful because it is inexpensive, letting us audit four deployed guard models, an 8B zero-shot judge, and thirteen benchmarks, and it raises the mechanically verified pass rate when used to rank candidate responses, though an adaptive white-box attack removes this gain. We release the counterfactual protocol and crossed-rule benchmark so rule blindness can be tested in future probe and guard claims.
References (20)
Training-Free Policy Violation Detection via Activation-Space Whitening in LLMs
Oren Rachmil, Roy Betser, Itay Gershon et al.
OmniCompliance-100K: A Multi-Domain, Rule-Grounded, Real-World Safety Compliance Dataset
Wenbin Hu, Huihao Jing, Haochen Shi et al.
Refusal in Language Models Is Mediated by a Single Direction
Andy Arditi, Oscar Obeso, Aaquib Syed et al.
GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis
Yueqi Xie, Minghong Fang, Renjie Pi et al.
Representation Engineering: A Top-Down Approach to AI Transparency
Andy Zou, Long Phan, Sarah Chen et al.
Programming Refusal with Conditional Activation Steering
Bruce Lee, Inkit Padhi, K. Ramamurthy et al.
TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law
Zheng Hui, Yijiang River Dong, Ehsan Shareghi et al.
DynaGuard: A Dynamic Guardian Model With User-Defined Policies
Monte Hoover, Vatsal Baherwani, Neel Jain et al.
LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails
Nanxi Li, Zhengyue Zhao, Chaowei Xiao
The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicators
Dominik J. Schwarz
FlexGuard: Continuous Risk Scoring for Strictness-Adaptive LLM Content Moderation
Zhihao Ding, Jinming Li, Zeren Lu et al.
CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems
Jingbo Yang, Guanyu Yao, Bairu Hou et al.
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
Samuel Marks, Max Tegmark
Regulation (EU) 2016/679 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation – GDPR)
Marta Otto
Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitors
Max McGuinness, Alex Serrano, Luke Bailey et al.
Linear probes rely on textual evidence: Results from leakage mitigation studies in language models
Gerard Boxó, Aman Neelappa, Shivam Raval
Auditing LLM Benchmarks with Item Response Theory
Sander Land, D. Bikel
The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
Wenbo Pan, Zhichao Liu, Qiguang Chen et al.
Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs
Alexander Panfilov, Evgenii Kortukov, Kristina Nikoli'c et al.