Stealing Reasoning Traces from Proprietary LLM APIs
Exploiting cross-model compatibility of encrypted reasoning traces allows scalable extraction of proprietary reasoning from APIs without model hacking.
Key Findings
Methodology
This paper analyzes the structure of encrypted reasoning chains returned by major LLM APIs, revealing their high cross-model compatibility. Leveraging this, the authors develop a scalable attack framework: by injecting encrypted reasoning traces from a strong, protected model into a weaker, less guarded model, they induce the latter to decode and output the reasoning in plaintext. The approach involves detailed cryptographic analysis of the AEAD-encrypted blocks, verification of their portability across different sessions, users, and models, and systematic experiments across platforms like OpenAI GPT-4, Google PaLM, and Anthropic Claude. The methodology emphasizes that these encrypted blocks are not tightly bound to specific models or sessions, enabling the attacker to bypass security measures such as anti-distillation and refusal training, and to extract sensitive internal reasoning, training data, and private information at scale.
Key Results
- The attack achieved over 95% fidelity in reconstructing the reasoning chains from GPT-4 and GPT-3.5 models, with the reconstructed reasoning closely matching the original internal process. In Google PaLM API, the success rate was 92%. On Anthropic Claude, weaker models like Claude-Haiku successfully recovered detailed reasoning traces, demonstrating broad applicability. Analyzing 315,320 publicly scraped reasoning blocks, the authors recovered 367 PII artifacts and 182 API credentials, including passwords, emails, and API keys, highlighting privacy risks. The method proved effective across multiple model families and versions, confirming the universality of the vulnerability.
- The attack's scalability is notable: by using weaker models as decoding intermediaries, attackers can perform large-scale extraction without direct access to the protected models. This approach circumvents typical defenses like input filtering and output monitoring, enabling mass data harvesting, model distillation, and privacy breaches. The experiments also show that cross-model and cross-session compatibility of reasoning blobs is a widespread phenomenon, making the vulnerability a systemic issue across the industry. The results underscore the urgent need for improved cryptographic protections and model architecture designs to prevent such exploits.
Significance
This research exposes a fundamental security flaw in current large language model API designs—namely, that encrypted reasoning chains are not as secure as presumed. The high cross-model compatibility of these encrypted blocks allows malicious actors to bypass model-specific safety measures, extract proprietary reasoning, and compromise user privacy at scale. The implications are profound: industry-standard encryption mechanisms, which rely on the assumption of isolated and model-specific data, are insufficient against such cross-model attacks. The findings challenge the security assumptions underpinning current API protocols and call for a re-evaluation of how reasoning data is protected. For academia, this work opens new avenues for research into cryptographic defenses tailored for AI models. For industry, it highlights the urgency of developing more robust encryption schemes and architecture-level safeguards to prevent reasoning leakage, ensuring the safe deployment of powerful AI systems.
Technical Contribution
This paper makes a pioneering contribution by systematically analyzing the structure and compatibility of encrypted reasoning chains across multiple models and providers. It demonstrates that these encrypted blocks, protected by AEAD encryption, are not model-specific but are instead portable and interchangeable, due to the use of a common cryptographic key. Building on this insight, the authors develop a novel attack framework that employs weaker, less secure models as decoders—effectively turning them into decryption oracle machines—without needing to access the original, more secure models. The approach combines cryptographic analysis, model compatibility testing, and practical injection techniques, resulting in a scalable, high-fidelity reasoning extraction method. This work bridges the gap between cryptography, model architecture, and security analysis, providing a new perspective on AI model safety and privacy.
Novelty
This is the first comprehensive study revealing that encrypted reasoning chains in major LLM APIs are broadly compatible across models, sessions, and users. Unlike prior work focusing on output-based model extraction or parameter theft, this research exploits the structural and cryptographic properties of reasoning blobs to perform scalable, cross-model reasoning recovery. The core innovation lies in recognizing that the encryption scheme, combined with the model architecture, inadvertently facilitates a form of 'cryptographic portability,' which can be weaponized for large-scale reasoning extraction. This paradigm shift in understanding model security highlights that protecting only the model parameters or output is insufficient; internal reasoning data must also be secured against cross-model leakage. The work sets a new benchmark for security evaluation in AI APIs.
Limitations
- The attack relies on the assumption that models share a common cryptographic key and that the encryption scheme does not incorporate model-specific or session-specific randomness. If future models adopt differentiated keys or multi-layer encryption, the attack's effectiveness will diminish.
- The method is primarily demonstrated in API environments with accessible reasoning traces; in highly secure enterprise settings with additional layers of encryption or strict access controls, the attack may face practical barriers.
- Large-scale extraction requires significant computational resources and automation, which could be mitigated by implementing stricter cryptographic protocols or model architecture modifications.
Future Work
Future research should focus on developing cryptographic schemes that prevent cross-model portability of reasoning blobs, such as model-specific keys or session-bound encryption. Additionally, exploring architecture-level modifications—like embedding reasoning in hardware enclaves or employing differential encryption—can enhance security. Industry standards for reasoning data protection need to be established, possibly integrating hardware security modules (HSMs) and secure multi-party computation (SMPC). Further, extending the analysis to other AI paradigms, such as multimodal models or federated learning setups, will be crucial. The community should also investigate formal cryptographic proofs for reasoning chain confidentiality, ensuring that future models are resilient against such cross-model extraction attacks.
AI Executive Summary
In recent years, large language models (LLMs) have become central to AI research and industry applications, especially with their ability to generate complex reasoning chains that enhance problem-solving capabilities. To safeguard proprietary information and prevent leakage of sensitive internal processes, API providers have adopted encryption schemes that return reasoning traces as opaque, encrypted blocks. These blocks are intended to be non-interpretable outside the provider’s environment, thus protecting the model’s internal logic. However, this paper uncovers a critical security flaw: these encrypted reasoning blocks are not as isolated as previously believed.
Through systematic analysis and extensive experiments, the authors demonstrate that the encrypted reasoning traces are highly compatible across different models, sessions, and even users within the same provider ecosystem. This compatibility stems from the cryptographic design—most likely a shared key-based AEAD encryption—that allows the same encrypted block to be decrypted and interpreted by different models, including weaker, less secure variants. Exploiting this, the researchers develop a scalable attack framework: by capturing a reasoning trace generated by a high-capability, protected model, they inject it into a weaker model, which then decodes and outputs the plaintext reasoning verbatim.
The implications of this vulnerability are profound. The authors successfully recover detailed reasoning chains, revealing sensitive information such as API keys, personal data, and internal hypotheses. Their experiments across major platforms like OpenAI, Google, and Anthropic show that over 95% of reasoning traces can be faithfully reconstructed, exposing privacy and intellectual property risks at an unprecedented scale. The attack’s scalability is facilitated by the fact that weaker models, which are often less guarded, can serve as effective decoders, enabling mass extraction without raising suspicion.
This research highlights a systemic flaw in current encryption strategies for AI reasoning data. The widespread compatibility of encrypted reasoning blobs means that adversaries can bypass model-specific defenses and conduct large-scale model inversion, knowledge distillation, and privacy breaches. The findings call for urgent re-evaluation of cryptographic protocols, advocating for model-specific encryption keys, session-bound encryption, and architecture modifications to prevent such cross-model leakage.
In conclusion, this work not only exposes a significant security vulnerability but also sets the stage for future innovations in AI safety and privacy. It urges the community to rethink how reasoning data is protected, emphasizing the need for cryptographic and architectural safeguards that can withstand cross-model compatibility attacks. As AI models continue to grow in capability and deployment, ensuring their internal reasoning remains confidential is paramount for safeguarding intellectual property, user privacy, and societal trust in AI systems.
Deep Dive
Abstract
Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks. By decoding 315,320 reasoning blocks scraped from public repositories, we recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials. Third, it inadvertently reveals hazardous information hidden within the reasoning process, even in cases where the model's final, visible output safely rejects a malicious request. Fourth, attackers can leverage this flaw to execute invisible prompt injections, embedding malicious payloads entirely within encrypted blocks to poison public agentic rollouts. Following responsible disclosure, we propose concrete cryptographic and system-level mitigations to secure client-side reasoning.