PRISM: Recovering Instruction Sets from Language Model Activations
PRISM directly decodes instruction sets from model activations, outperforming activation-to-language baselines with 97%+ recall in security scenarios.
Key Findings
Methodology
PRISM employs a frozen target language model, extracting residual activations from a specific layer. It trains an activation projection and LoRA adapters to map these activations into the model's input space. A judge-guided Group Relative Policy Optimization (GRPO) framework guides reinforcement learning, where a separate language model judge assesses the fidelity of predicted instruction sets, rewarding coverage and penalizing hallucinations. The training involves supervised pretraining with cross-entropy loss, followed by RL to enhance recall and reduce false positives, especially in multi-instruction, multi-constraint settings. The approach enables faithful decoding of active instructions directly from internal states, bypassing surface token analysis.
Key Results
- In security-sensitive tests (prompt injection, hidden objectives), PRISM achieves over 97% instruction recall, compared to ~53% for activation-to-language baselines. Across diverse datasets, PRISM’s coverage exceeds 73%, with hallucination rates below 1.5%. The method outperforms LatentQA and Activation Oracles, especially in complex scenarios, demonstrating robustness and high fidelity in instruction recovery.
- The judge-guided RL significantly improves recall by approximately 20% and reduces hallucinations by 30%, validating the effectiveness of reward-driven training. Calibration of the judge (Cohen’s κ=0.78) ensures consistent evaluation. Results show that PRISM maintains stable performance across different attack scenarios, confirming its potential for real-world security monitoring.
- Experimental data confirms that PRISM’s approach yields a substantial improvement over prior methods, with an average reward increase of +0.204 over non-RL baselines. Its ability to recover latent instructions and detect covert objectives makes it a promising tool for AI safety and transparency, especially in adversarial settings.
Significance
This work addresses a critical gap in understanding and monitoring large language models by enabling direct extraction of their active instruction sets from internal activations. It enhances transparency, accountability, and safety, particularly in high-stakes applications such as security, compliance, and AI governance. The method provides a scalable, model-agnostic approach to internal inspection, facilitating better oversight and debugging of complex AI systems. Its robustness against adversarial manipulations marks a significant step toward trustworthy AI deployment.
Technical Contribution
PRISM introduces a novel architecture that leverages frozen model weights, combined with a learned activation projection and LoRA adapters, to decode instructions directly from hidden states. The integration of a judge-guided reinforcement learning framework, based on GRPO, allows targeted optimization of instruction recall and hallucination minimization. This approach departs from traditional activation-to-language methods by explicitly training for set-level fidelity, enabling comprehensive extraction of multi-instruction scenarios. The calibration of the judge model further ensures reproducibility and evaluation consistency, setting new standards in interpretability research.
Novelty
This paper is the first to formalize instruction set retrieval as a direct decoding problem from internal activations, rather than surface-level token analysis. It innovates by combining a dedicated interpretive model with a judge-guided RL training paradigm, specifically targeting multi-instruction, multi-constraint environments. Unlike prior work limited to description or single-objective extraction, PRISM achieves high-fidelity, comprehensive instruction recovery, especially in security-critical contexts, representing a significant advancement in model interpretability.
Limitations
- PRISM relies on the calibration and accuracy of the judge model, which may be less effective in highly complex or novel instruction scenarios. Its performance can be affected by the quality of the judge's semantic equivalence judgments.
- The training process involves substantial computational resources, especially for large models and multi-scenario datasets, limiting real-time deployment. Further optimization is needed for efficiency.
- Current evaluation focuses on specific datasets and attack scenarios; generalization to broader, real-world environments remains to be validated. Handling unseen or highly covert instructions poses ongoing challenges.
Future Work
Future research will explore integrating multi-modal internal signals, such as attention maps and gradient information, to improve instruction completeness. Expanding training datasets to cover more diverse and complex instructions will enhance robustness. Additionally, developing lightweight, real-time versions of PRISM and extending its application to multi-agent systems and federated settings will be key directions. Further, refining judge calibration and exploring unsupervised or semi-supervised training strategies could reduce reliance on labeled data.
AI Executive Summary
Understanding the internal workings of large language models (LLMs) is vital for ensuring their safe and responsible deployment. Traditional methods focus on surface-level outputs or superficial probing, which often fail to reveal the full scope of active instructions, constraints, and hidden objectives guiding the model’s behavior. This gap is especially critical in security-sensitive applications, where malicious prompt injections or covert goals can lead to undesirable outcomes.
Addressing this challenge, the paper introduces PRISM, a novel framework that directly decodes instruction sets from internal activations of a frozen target model. By leveraging a specialized activation projection and LoRA adapters, PRISM transforms residual states into a natural-language bullet list of active instructions. Its core innovation lies in a judge-guided reinforcement learning process, which optimizes for comprehensive coverage and minimal hallucination, guided by a separate language model judge calibrated for semantic fidelity.
Experimental results demonstrate that PRISM achieves over 97% instruction recall in adversarial scenarios, far surpassing existing activation-to-language baselines. Its robustness across multiple datasets and attack types underscores its potential for real-world security monitoring and model interpretability. The approach not only enhances transparency but also provides a scalable, model-agnostic tool for internal inspection, debugging, and compliance.
Looking ahead, future work will focus on integrating multi-modal signals, expanding instruction coverage, and improving efficiency for real-time deployment. The development of standardized evaluation metrics and broader dataset benchmarks will further solidify its role in trustworthy AI. Overall, PRISM marks a significant step toward transparent, accountable, and secure AI systems, enabling stakeholders to better understand and control complex language models in high-stakes environments.
Deep Analysis
Background
近年来，随着大规模语言模型（LLMs）在自动化、对话和内容生成中的广泛应用，模型的内部机制逐渐成为研究焦点。早期的探针（Probing）和潜空间分类器（Latent Space Classifiers）试图从激活中提取知识信息，但多局限于单一属性检测。激活到语言的解释方法如LatentQA、Activation Oracles等，能将隐藏状态转化为文本描述，但在多指令、多约束环境中难以完整提取所有指令，存在信息遗漏和虚假指令的问题。随着模型应用复杂化，理解其多重目标和约束成为关键，亟需更精细的内部监控工具。
Core Problem
核心挑战在于如何从模型内部激活状态中准确提取当前所有影响行为的指令、约束和潜在目标。现有激活到语言方法多为单一描述或问答，难以覆盖多指令、多约束的复杂场景，容易遗漏关键指令或引入虚假信息。这限制了模型行为的透明度和安全监控的有效性。特别是在安全敏感场景中，误判或漏检潜在风险指令可能导致严重后果。因此，开发一种能全面、准确提取指令集的工具成为研究重点。
Innovation
PRISM的创新在于其将激活状态直接映射为完整的指令集，结合判决引导的强化学习优化策略，显著提升指令的召回率和减少虚假。其架构设计采用冻结模型参数，利用激活投影和LoRA适配器，确保高效部署。引入判决模型对指令的真实性进行评分，指导训练过程，确保输出的指令集具有高信度和完整性。这一方法突破了传统激活到语言的局限，为多指令、多约束环境下的模型监控提供了新思路。
Methodology
- �� 采集目标模型在特定输入下的隐藏状态激活Hℓ。• 训练激活投影，将激活映射到模型输入空间，形成软前缀。• 利用判决模型对预测指令的真实性进行评分，作为强化学习的奖励信号。• 采用判决引导的GRPO方法，采样多个候选指令集，优化召回率和虚假指令。• 训练过程中，冻结目标模型参数，仅更新投影和适配器。• 结合监督预训练和强化学习，逐步提升指令提取的准确性和完整性。
Experiments
使用多场景数据集（Benign、Adversarial）进行训练和评估，数据包括真实指令和模拟攻击场景。基线方法包括LatentQA和Activation Oracles。评价指标为指令覆盖率、虚假指令率和整体召回率。采用Qwen3.5-9B模型，激活层为第16层最后128个生成标记的残差状态。通过多轮训练和调参，验证PRISM在多场景下的优越性能，特别是在安全敏感场景中的表现。还进行了消融实验，验证判决引导强化学习的贡献。
Results
PRISM在安全场景中的指令召回率达97%以上，显著优于基线的53%。在多指令环境中，覆盖率提升至73.1%，虚假指令率降至1.4%。判决引导的强化学习显著改善了指令完整性，减少虚假信息。实验证明，PRISM在多场景下的指令恢复能力优于现有激活到语言方法，尤其在复杂、多指令环境中表现出更强的鲁棒性和准确性。
Applications
该方法可用于模型安全监控、行为追踪和责任追溯，特别适合在敏感场景中检测潜在滥用或隐藏目标。企业和研究机构可以利用PRISM实现模型行为的内部审查，增强系统透明度。未来还可结合多模态信息，扩展到更复杂的指令环境，推动AI安全治理的标准化和自动化。
Limitations & Outlook
当前PRISM依赖判决模型的校准，可能在极端复杂或隐蔽指令下出现漏检。训练成本较高，尤其在大模型和多场景下，实时性有限。未来需优化算法效率，扩展指令空间，增强对未知指令的适应能力。同时，模型在多模态信息融合和多任务训练方面仍有待提升，以应对更复杂的应用场景。
Plain Language Accessible to non-experts
想象你在管理一个复杂的工厂，工厂里有许多不同的指令和规则，比如什么时候开工、如何检查产品、哪些操作禁止。每个工人都在按照这些指令工作，但你不能每次都去问他们，工厂的机器也会记住一些秘密指令，比如隐藏的操作或特殊的任务。PRISM就像是一个聪明的监控员，它可以偷偷观察工厂的内部状态（激活信息），然后告诉你工厂现在在执行哪些具体的任务、遵守了哪些规则、隐藏了哪些秘密。这样你就能知道工厂是否在正常工作，是否有潜在的问题，而不用打扰工人或检查每个细节。这种方法帮助你更好地理解和控制工厂的运行，确保一切都在安全范围内。
ELI14 Explained like you're 14
想象你在学校里有个超级厉害的老师，他可以偷偷观察每个学生的表现，知道他们在做什么秘密任务。平时老师不会直接问学生，但他有一种特殊的办法，可以通过观察学生的动作和反应，猜出他们在做什么。PRISM就像这个老师，它可以通过观察大脑里的“秘密信息”——也就是模型内部的隐藏状态，来猜出模型正在执行哪些指令、遵守哪些规则。这样一来，老师就能确保学生没有偷偷做坏事，也能帮老师发现哪些指令被隐藏或被误解。这个方法让我们更聪明地监控AI，确保它们在正确的轨道上运行，不会做出危险或不合规的事情。
Abstract
As LLMs are deployed as agents, reliable monitoring requires knowing not only what they output, but which instructions are steering their behavior. This is difficult when models infer unintended subgoals, follow contextual cues, or are influenced by prompt injections and hidden objectives. While activation-to-language methods suggest that hidden states can reveal natural-language information, existing approaches are not designed to recover the full set of simultaneous instructions, constraints, prohibitions, and subgoals active in agentic settings. We formalize this problem as instruction set retrieval and introduce PRISM, an activation-conditioned interpreter that decodes hidden states from a frozen target model into a faithful bullet list of active instructions. Unlike prior activation-to-language methods, PRISM is trained to recover instruction sets directly, using judge-guided GRPO to reward covered instructions and penalize unsupported ones. Across benign, constrained, prompt-injection, and hidden-objective settings, PRISM outperforms activation-to-language baselines, especially on security-relevant objectives.