Anthropic Self-Assessment: Claude Grades Its Own Conversations for Misuse at Under 2% Extra Compute

Short answer: Anthropic has described a safety technique called self-assessment, in which Claude reviews the safety of each conversation immediately after replying. In Anthropic's October 16, 2026 post, the team reports that on Claude Fable 5 it flags 78% of the conversations a stronger prompted classifier flags as potentially harmful, and 94% of the high-priority ones, at a 1% false-positive rate, while adding under 2% to inference compute. It has run in production since June, and Anthropic is clear that it is an additional layer, not a wall.

Trade coverage of the post appeared on October 29, which is why it is trending now. The figures below come from Anthropic's own post unless marked otherwise.

TL;DR

Agents that act on your behalf need rules. See how explainx helps secure your agents →

How self-assessment works

A normal chat alternates between user and Claude. After each Claude response, the system appends a short system turn, for illustration something like "Please reflect on the safety of this conversation." Claude's reply to that instruction is the assessment. Those last two turns are not shown to the user and are not carried into the rest of the conversation.

The verdict follows a fixed JSON schema. When nothing is wrong, it is just a flagged false. When Claude flags a conversation it also gives a reason, a category and subpolicy from Anthropic's internal taxonomy (cyber harms, CBRNE weapons, child safety and so on), whether it refused, and a priority from 1 to 10 that tells a human reviewer how urgently to look. To get a continuous score, Anthropic reads the probability Claude assigns to the word true in the flagged field and thresholds it, which is how it picks the 1% false-positive operating point.

The assessment covers the entire conversation, including requests Claude refused, and it can flag harms it was never specifically trained on, such as prompt injection attempts.

Why the KV cache makes this cheap

When a model finishes a reply, it already holds the whole conversation in its key-value cache, the stored intermediate computations from reading the text. A separate classifier must re-read the conversation plus the policy documents and then reason about them. Self-assessment reuses the cache, so it needs only the short instruction and roughly ten output tokens.

Anthropic's comparison points:

• Its first Constitutional Classifiers in early 2025 added 23.7% to inference cost.
• OpenAI has reported safety reasoning using as much as 16% of total compute in some launches, which Anthropic cites.
• Self-assessment adds under 2% over Anthropic's production API traffic mix.

For monitoring at scale, that gap decides whether you can watch most traffic or only a sample. Anthropic says the technique runs across a large fraction of Claude's global traffic.

The training recipe

Anthropic trains self-assessment in two stages, both distilling a more expensive teacher. The teacher is a prompted classifier based on Claude Mythos Preview that has the full internal policy documents in its prompt and walks a reasoning checklist.

1. Supervised fine-tuning on synthetic conversations, each followed by the reflection instruction and the teacher's verdict. The loss is masked on everything except the assessment, so Claude learns to judge compliant-with-harm responses without learning to produce them.
2. Reinforcement learning in environments that prefill synthetic harmful and benign conversations and score each sampled assessment on six verifiable rewards: valid JSON, matching flag, category, subpolicy, refused label, and a close priority score.

The training data is fully synthetic, generated by a pipeline similar to the one used for Constitutional Classifiers: 32 themes per subpolicy and 32 scenarios per theme, giving 1,024 harmful and 1,024 hard-negative benign scenarios per subpolicy. Half the conversations are in one of nine non-English languages, and some harmful ones use obfuscation such as homoglyphs, base64 or rot13.

Does it get better as models get better?

That is the main bet. Because the monitor is the same model that had the conversation, its judgment should improve with capability. Anthropic trained self-assessment into four model sizes (supervised fine-tuning only), and reports detection improving consistently from smallest to largest. AlphaSignal's reading of the figures puts recall at a 1% false-positive rate rising from about 16% to about 96%, with AUC going from 0.862 to 0.995. Anthropic notes those models are not comparable with the production Fable 5 numbers.

Generalization is the other claim. Removing a whole harm category from training did not substantially cut recall: per AlphaSignal, dropping fraud examples took recall on synthetic fraud conversations from 92.8% to 90.0%, and dropping cyber examples took it from 92.7% to 90.3%. That suggests the monitor draws on Claude's broader understanding of harm and can notice misuse nobody labeled in advance.

What it found in production

Anthropic says that since June self-assessment has revealed large-scale misuse other methods did not identify, helped improve its other classifiers, and served as an extra layer in its defense-in-depth strategy. It describes three uses of the logs:

• Flags are aggregated by organization nightly in a growing set of harm areas, and organizations with several high-priority flags are escalated to an enforcement team for human review.
• Flags are compared with other classifiers' output to find harmful conversations those classifiers missed.
• Flags that fall outside the taxonomy are clustered to look for new kinds of misuse.

Anthropic's discussion section gives examples: in August, over 100,000 child-exploitation flags from a single API customer, which it says were reported to NCMEC; in September, an operation that used Claude Code to run an automated romance scam against dozens of victims, after which the accounts were banned; and, from clustering flags outside the taxonomy, prompt injection, illicit distillation attempts and agentic actions the user had not authorized. Anthropic says about a third of flags do not map cleanly to its existing taxonomy.

The honest limits

A folder with a small green flag, representing flagged conversations queued for human review in Anthropic self-assessment

Anthropic is direct about the weaknesses. Self-assessment "does not catch every harmful conversation, and it is not robust enough to stop a dedicated attacker." AlphaSignal reports that as a blocking classifier it performed poorly, with a 54% attack success rate versus 25% for a prompted classifier, but works well as an offline monitor. That is exactly how Anthropic uses it: wide coverage first, higher-precision tools second, real-time safeguards for blocking.

Other caveats worth keeping in mind:

• The 78% and 94% figures measure agreement with a teacher classifier, not independently verified harm.
• A monitor that is the model itself can share the model's blind spots.
• The labels come from a Claude-based teacher, so the loop is internal. Outside audits would strengthen the claims.
• Assessments are retained under a data-handling policy described in the post; privacy-minded customers should read it.

How this compares with other labs' monitoring

Monitoring is becoming a visible arms race. OpenAI described a four-layer system for monitoring frontier model misalignment. Google's Gemini 4 Argon safety report leaned on activation probes and a misalignment monitor. Startups such as Goodfire sell inside-out monitors built on activation probes. Probes are cheap but only score a harm fixed in advance. Prompted classifiers are flexible but costly. Self-assessment tries to get the cheapness of the first and the flexibility of the second, and Anthropic presents it as one layer among several.

It also fits Anthropic's wider safeguards push, such as the midterm election safeguards work, and arrives while regulators are scrutinizing lab safety practices, including EU AI Office probes and an FTC inquiry.

What builders should take from it

• If you use the Claude API, your traffic may be assessed by this layer. Anthropic's usage policy still applies, and organizations with repeated high-priority flags can be escalated for review.
• If you build your own monitoring, the pattern is reusable: ask the model to assess its own transcript with a structured schema, reuse the cache to keep cost low, and treat the output as a triage signal for humans rather than a block.
• Do not rely on one layer. A self-monitor will not stop a determined attacker. Pair it with permission boundaries and runtime controls; AgentBeam is the agent security platform from the explainx.ai team that stops AI agents before they take dangerous actions.
• Ask for evidence. Recall against a teacher at 1% false positives is useful, but your own domain may behave differently.

Related reading

• OpenAI's four-layer misalignment monitor
• Gemini 4 Argon safety report: activation probes
• Goodfire inside-out monitors
• Anthropic midterm safeguards
• EU AI Office probes of OpenAI and Anthropic
• FTC probe into AI safety
• Contractor's Claude Code agent on a DOD network (reported)

Sources: Anthropic safeguards post, AlphaSignal summary.

Figures are as published by Anthropic on October 16, 2026 and reported on October 29, 2026; they may change as the technique evolves.