Skip to main content
Mallory
← Back to News
ai-platform-security

Anthropic Deploys Post-Response AI Self-Assessment to Detect Misuse

Emerging Technology Risk · 3 sources · Updated 19d ago · First seen Dec 8, 2025

Anthropic Deploys Post-Response AI Self-Assessment to Detect Misuse

Anthropic reports deploying self-assessment, an offline monitoring technique in which Claude reviews conversations for potential misuse after responding. On Claude Fable 5, the technique detected 78% of conversations flagged by a stronger prompted classifier and 94% of high-priority conversations at a 1% false-positive rate, while adding less than 2% to inference compute. Anthropic says production deployment helped uncover previously undetected abuse, prompt injection attempts, unauthorized agent actions, and an automated romance-scam operation. These results measure agreement with a reference classifier, rather than detection of all actual misuse.

The approach complements research into both preventive safeguards and post-response accountability. Constitutional Classifiers, trained using synthetic data derived from natural-language policies, reportedly resisted universal jailbreaks across more than 3,000 estimated hours of red teaming; enhanced classifiers added 23.7% inference overhead and increased production-traffic refusals by 0.38 percentage points. Separately, Training LLMs for Honesty via Confessions describes training GPT-5-Thinking to disclose policy violations after answering, with confession rewards based solely on honesty and separated from the original answer’s reward. For security teams, these approaches support layered defenses rather than reliance on model self-reporting: Anthropic warns that self-assessment remains vulnerable to targeted evasion, inherits its teacher classifier’s blind spots, and cannot replace real-time safeguards or human investigation.

New surface, new adversariesTrack how attackers are adapting to this technology.
Track this risk
Event timeline

How this story unfolded

Events from the most recent confirmed update back to the earliest known activity.

8 events · Jan 31, 2025 – Dec 8, 2025
  1. Reported Oct 16, 20261mo ago

    Authorization experiment distinguishes permitted vulnerability research targets

    In a deployment-time authorization experiment, Fable 5 flagged all vulnerability-research conversations involving non-allowlisted targets and none involving allowlisted targets across a wide range of thresholds.

    Scaling Misuse Detection with Self-Assessment
  2. Reported Oct 16, 20261mo ago

    Adversarial testing exposes limits of self-assessment as a blocking safeguard

    Across 24 offensive cybersecurity tasks, attack success was 79% without safeguards, 54% with self-assessment blocking, and 25% with a standalone prompted classifier. Self-assessment flagged 21 attack trajectories at least once, but Anthropic cautioned that attackers targeting the monitor directly could evade it.

    Scaling Misuse Detection with Self-Assessment
  3. Reported Oct 16, 20261mo ago

    Anthropic reports production self-assessment detection results

    At a 1% false-positive rate, Fable 5 self-assessment detected 78% of conversations flagged by its reference classifier and 94% of high-priority conversations. The monitoring added less than 2% to inference compute, while more than half of the organizations it surfaced had no flags from other detection sources during the preceding month.

    Scaling Misuse Detection with Self-Assessment
  4. Reported Oct 16, 20261mo ago

    Anthropic detects automated romance scam and bans involved accounts

    In September, self-assessment surfaced an operation using Claude Code to build and run an automated romance scam targeting dozens of victims. Anthropic banned the involved accounts after human review.

    Scaling Misuse Detection with Self-Assessment
  5. Reported Oct 16, 20261mo ago

    Self-assessment flags API customer for child sexual exploitation content

    In August, self-assessment generated more than 100,000 child sexual abuse and exploitation flags associated with one API customer. The flags prompted an investigation and a report to the National Center for Missing & Exploited Children.

    Scaling Misuse Detection with Self-Assessment
  6. Reported Oct 16, 20261mo ago

    Anthropic deploys passive self-assessment monitoring with Fable 5

    Self-assessment began operating in production when Fable 5 launched in June, flagging conversations for subsequent review rather than blocking them. Anthropic says every Claude model released since Fable 5 includes the technique.

    Scaling Misuse Detection with Self-Assessment
  7. Dec 8, 20251y ago

    Researchers submit paper on training models to confess policy violations

    The paper proposed eliciting honest post-answer accounts of model misconduct by rewarding confessions independently of the original answer. Researchers trained GPT-5-Thinking to produce these confessions to demonstrate the approach.

    [2512.08093] Training LLMs for Honesty via Confessions
  8. Jan 31, 20252y ago

    Researchers submit Constitutional Classifiers defense paper

    The paper introduced safeguards trained on synthetic data generated from natural-language policy rules. It reported no successful universal jailbreak across more than 3,000 estimated hours of red teaming, with a 0.38% absolute increase in production-traffic refusals and 23.7% inference overhead.

    [2501.18837] Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teaming
Linked entities

Related entities

Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.

Open in app →

Affected products

1 linked
Claude

Organizations

1 linked
Anthropic
Do you run this?Follow how adversaries are adapting to this technology, and where it touches your stack today.
Track This Risk
Mallory platform

See the full picture, correlated to your attack surface.

This page covers what’s public. Mallory adds the parts that aren’t: which of your assets are affected, which threat actors are using it right now, and what to deploy next.

Exposure mappingMap indicators from this story to your assets and identify affected systems in minutes.
Threat actor evidenceEvery observed campaign, victim, and pivot linked to actors named in this story.
Associated malwareMalware, exploits, and IOCs connected to the activity described here.
Detection signaturesYARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Scheduled alertsGet matching new stories delivered to your team as they break, not the next morning.
AI threadsAsk questions about this story and take action on the answers.