Enterprise AI Security  ·  August 2026

Stolen LLM Reasoning Blocks
Contain Credentials and PII
: 315,000 Already in Public Repos

A new paper from ELLIS Institute Tübingen and the Max Planck Institute documents a structural flaw in encrypted chain-of-thought: blocks are replayable across sessions, users, and models. 315,320 have been scraped from public repositories. 182 contain API keys or passwords. 367 contain PII. Here is what attackers do with them.

Arjun Jaggi  ·  8 min read  ·  Security & Governance
315K Blocks in public repos
182 Credential artifacts found
367 PII artifacts found
3 Major providers affected

What the Paper Actually Found

Panfilov et al. (ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems) published a paper in August 2026 documenting a property of encrypted chain-of-thought (CoT) reasoning blocks that most practitioners had not considered: they are interchangeable.[1]

The architecture is straightforward. When a frontier model reasons before responding, the provider encrypts that reasoning trace and returns it alongside the final answer. The intent is to let the provider cache and reuse computation. The flaw is that the encryption scheme does not bind the block to the session, the user, or even the specific model that generated it. A block created by a premium frontier model can be injected into an API call targeting a smaller sibling model, and the sibling decodes and continues from that reasoning state as if it produced the block itself.

Key Finding

Encrypted reasoning blocks are not session-scoped, user-scoped, or model-scoped. They are replayable across all three dimensions within a provider's ecosystem, by design, without any bypass of the encryption layer.[1]

The researchers scraped 315,320 such blocks from public GitHub repositories, Hugging Face datasets, and developer forums. Within that corpus: 182 blocks contained credentials (API keys, passwords, service tokens), and 367 contained PII artifacts (names, emails, internal identifiers). These numbers come from automated scanning of plaintext-decodable content, blocks that providers had already decrypted or that developers had inadvertently logged.

Three major providers are affected: Anthropic, OpenAI, and Google. No architectural fixes have been issued as of the paper's publication date.

Why Encryption Does Not Protect You Here

The instinct is to assume encryption means safety. In this case, that instinct fails because the threat model is different from the one encryption was designed for.

Encryption protects data in transit from a third party who cannot hold the key. Here, the attacker is not a third party. They are a legitimate API customer who can submit any properly structured API call. The provider's infrastructure decrypts the block and feeds its content to the model as part of a normal inference request. The attacker does not need to break the encryption, they need only to replay the ciphertext through the provider's own decryption pipeline.

Structural Gap

Encrypted CoT blocks cannot be validated as belonging to a specific session or user by the provider's own API, because no such binding was built into the scheme. Replay is a property of the design, not a bug in the implementation.[1]

Prompt injection via indirect input has been documented since Greshake et al. (2023).[2] The reasoning block replay attack extends that threat surface into the reasoning layer, a layer that many organizations now treat as opaque infrastructure rather than attack surface.

Fig. 1: Attack Surface by Data Type in Scraped Blocks
Data from Panfilov et al., arXiv:2608.09867. Left: total corpus scraped from public repos. Right: sensitive blocks within the corpus (right panel shown at 0-400 scale for readability; 549 sensitive blocks total out of 315,320). "Clean" blocks contained no sensitive artifacts in automated scanning.

Four Attack Vectors Enterprises Must Model

The paper establishes the capability. The operational question for security and AI teams is: what does an adversary actually do with 315,000 blocks?

1. Credential Harvesting

Blocks logged to shared repositories or observability pipelines contain API keys and passwords that were present in the reasoning context. Automated scanning of public repos yields immediate value without any model interaction.

2. System Prompt Reconstruction

Enterprise system prompts are proprietary IP. When a model reasons about its instructions, those instructions appear in the reasoning trace. An attacker who can decode or replay a block gains read access to system prompts the organization considers confidential.

3. Model Capability Distillation

Replaying a frontier model's reasoning block into a cheaper sibling effectively transfers the frontier's reasoning capability for that specific task. This is capability distillation without training data, at API call cost.

4. Agent State Manipulation

In multi-step agentic workflows, reasoning blocks carry intermediate state. Injecting a crafted or stolen block at step N of an agent run can redirect subsequent tool calls, data writes, and external API interactions.

Why Your Existing Security Stack Misses This

OWASP's LLM Top 10 (2025 edition) covers prompt injection, insecure output handling, and sensitive information disclosure.[3] Reasoning block replay sits at the intersection of all three, but none of the existing controls were designed for the encrypted-intermediate-state threat model.

DLP tools scan content at rest and in transit, but they do not inspect the ciphertext of a reasoning block in an API response. SIEM rules are written for known log patterns, not for opaque base64 fields in JSON payloads. API gateway rate limiting does not distinguish between a legitimate call and a replay call, both look identical at the network layer. Observability pipelines that log full API request/response pairs are unintentionally creating the corpus that attackers need.

The Real Problem

Most enterprise AI logging configurations were designed when reasoning traces were plaintext and clearly visible. Encrypted blocks changed the threat model; the logging configurations did not change to match. Every full-response log is a potential credential and PII exposure vector.

Three Controls That Work Now

Architectural fixes from providers will take time. These controls are implementable before a provider patch ships.

Control What It Does Implementation
Strip encrypted blocks from logs Eliminates the primary exposure vector. If encrypted blocks never appear in your observability stack, they cannot be scraped from logs or repositories. Add a log sanitizer that removes fields matching the provider-specific encrypted block schema before writing to any log sink. Regex on field names is sufficient for the current block format.
Audit API call structures for replay patterns Detects replay attacks in progress by identifying API calls where an encrypted reasoning block appears in the request but the conversation history is shallow or absent. Parse API call bodies in your gateway layer. Flag requests that include a non-empty encrypted reasoning field with fewer than 2 prior turns. Alert and rate-limit before forwarding.
Exclude sensitive context from reasoning scope Prevents credentials and PII from entering the reasoning trace in the first place. If the data is not in the context window, it cannot appear in the block. Restructure system prompts and tool outputs to pass credentials via environment injection at the infrastructure layer, not as text in the model context. Redact PII before it reaches the model.
Implementation: Log Sanitizer for Encrypted Reasoning Blocks
// Strip encrypted reasoning blocks before writing to any log sink.
// Works for Anthropic (thinking blocks) and OpenAI o-series (reasoning_content).

function sanitizeApiResponseForLogging(response) {
  const safe = JSON.parse(JSON.stringify(response)); // deep clone

  // Anthropic: remove encrypted thinking blocks from content array
  if (safe.content && Array.isArray(safe.content)) {
    safe.content = safe.content.filter(block => block.type !== 'thinking');
  }

  // OpenAI o-series: remove reasoning_content from each choice
  if (safe.choices && Array.isArray(safe.choices)) {
    safe.choices.forEach(choice => {
      delete choice.reasoning_content;
      if (choice.message) delete choice.message.reasoning_content;
    });
  }

  // Google: remove thought_signature from parts
  if (safe.candidates) {
    safe.candidates.forEach(c => {
      if (c.content?.parts) {
        c.content.parts = c.content.parts.filter(p => !p.thought_signature);
      }
    });
  }

  return safe;
}

// Usage: wrap every response before sending to your observability stack
// logger.info(sanitizeApiResponseForLogging(apiResponse));
Fig. 2: Control Effectiveness vs. Implementation Effort
Directional illustration only, not derived from empirical benchmarks. Effectiveness represents estimated risk reduction relative to unmitigated baseline; effort represents implementation complexity for a team with existing API gateway access.

Executive Checklist

What a CISO or Chief AI Officer needs to validate before the next quarterly review:

What This Means for the AI Security Stack

The reasoning block replay finding is a structural signal, not an isolated vulnerability. It reveals that the encryption boundary between provider infrastructure and enterprise application was designed for performance, not for security. That boundary will be stressed again as providers add more opaque intermediate state to the inference pipeline.

The precedent from traditional security is instructive: JWT tokens were similarly designed for stateless session management and later exploited via replay, algorithm confusion, and key confusion attacks. The pattern is the same: a cryptographic primitive designed for one purpose gets loaded with security assumptions it was never designed to carry.

Enterprises that treat the reasoning layer as a black box that someone else secures will be exposed at the next architectural boundary. The organizations that map the threat model of each new inference primitive as it ships, and ask "what is this not binding, and what can an adversary do with that?" will be two quarters ahead of the incident.

Practitioner Note

The 315,320 blocks already in public repositories are not recoverable. The control surface that matters now is forward-looking: every block that enters your logging pipeline after today is within your control. Start there.

References

  1. [1] A. Panfilov, D. Schmotz, I. Shumailov, L. Beurer-Kellner, J. Schaeffer, A. Prabhu, J. Geiping, M. Andriushchenko, "Stealing Reasoning Traces from Proprietary LLM APIs," ELLIS Institute Tübingen and Max Planck Institute for Intelligent Systems, arXiv:2608.09867, 2026.
  2. [2] K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, M. Fritz, "Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection," arXiv:2302.12173, 2023.
  3. [3] OWASP Foundation, "OWASP Top 10 for Large Language Model Applications," Version 2025, owasp.org, 2025.

Excited about AI, innovation, and growth?

Start a conversation