Anthropic published a research post on September 17, 2026 proposing three specific measurements for tracking how AI development is actually progressing inside a frontier lab — and shared numbers from inside its own operation as a working example. The headline figure: Claude now "leads" 26% of Anthropic's AI research and development work, up from under 1% in February 2026. This is Anthropic's follow-through on Dario Amodei's "Pace the Frontier" essay from earlier this month — moving from a commitment to embed outside evaluators toward the actual measurements those evaluators would check.
TL;DR
| Question | Answer |
|---|---|
| What are the three measurements? | (1) How much AI R&D is done by AI, (2) how well AI agents are overseen, (3) how compute is allocated between safety and everything else |
| The headline number | Claude "leads" (AL4) 26% of Anthropic's AI R&D work, up from under 1% in February 2026 |
| Is Claude fully autonomous on any of it? | No — Anthropic states Claude is not operating fully autonomously (AL5) for any measured subset |
| How many internal agents does Anthropic run? | ~30,000 at any one time on its main internal platform, with 100% of actions passing through a monitor |
| What share of compute goes to safety? | ~6% of AI R&D compute; ~12% of AI-driven AI R&D compute specifically |
| Is this independently verified? | No, not yet — Anthropic says it plans to embed third-party evaluators to check these exact metrics going forward |
| Why does this matter? | It's a concrete, publishable measurement template other frontier labs could adopt, addressing the "how would we even know" question in AI pacing debates |
The R&D Automation Index: how "26%" was actually built
The number itself is less interesting than the methodology behind it, because the methodology is the part meant to be reusable by other labs. Anthropic couldn't manually catalogue every AI R&D task at the company, so it built the list bottom-up: for each week in July 2026, it randomly sampled 20% of staff across departments in the model R&D loop, and had a Claude research agent review each sampled person's Slack and internal documentation to list what they worked on. That produced roughly 15,000 granular tasks, which Claude then organized into a hierarchical tree with 542 nodes — 378 of them leaf categories like "eval platform defect diagnosis and fixes" or "RL sandbox egress and network policy." That tree is frozen as the fixed basket of work every future measurement compares against.
For each node, a Claude research agent gathered evidence on how that work gets done and by whom, and a separate Claude judge assigned one of six automation levels — a scale adapted from Epoch AI, running AL0 (no AI involvement) to AL5 (AI operates fully autonomously, no human in the loop). AL3 ("collaborates") means AI does large chunks of work under close human direction; AL4 ("leads") means AI completes most of a task end-to-end from a high-level prompt while a human supervises rather than stays actively engaged. Each node's weight in the final index comes from how much person-time Anthropic staff reported spending on it — so busier categories count for more, rather than Anthropic subjectively ranking what matters.
The August 2026 result: Claude leads 26% of R&D work, and the share at or above "AI collaborates" is above 90%. Nothing is measured at AL5. Anthropic reports its judge model agreed with human raters on exact automation level 59% of the time, and was within one level of human ratings 97% of the time — a real methodology disclosure that lets a reader gauge how much noise sits in the headline number, rather than presenting 26% as a clean, uncontested figure.
Overseeing 30,000 agents
The second measurement addresses a different worry: as more work shifts from "AI collaborates" toward "AI leads," agents make more consequential decisions with less continuous human attention. Anthropic reports approximately 30,000 agents doing research and engineering work at any one time on its most-used internal platform, constrained by two kinds of monitors:
| Online monitors | Offline monitors | |
|---|---|---|
| Purpose | Block dangerous, irreversible actions in real time | Catch dangerous behaviors that develop slowly and can be reversed |
| Coverage | 100% of actions, checked before execution | 100% of actions, reviewed after the fact |
| Review latency | Seconds; humans review blocked actions within a week | ~100,000 transcripts flagged weekly; ~50 escalate to human review |
| Rate over August 2026 | 0.002% of over a billion decisions blocked (about 1 in 47,000) | 1-2 transcripts per thousand flagged for further review |
Two design choices underpin this: every agent gets an individual identity persisted across model upgrades (so its action history stays continuous and auditable even as the underlying model changes), and agents communicate through a shared, open messaging system rather than privately — letting agents correct each other's errors and letting behavioral monitoring trace communication between agents rather than just within one agent's own transcript. This is the operational-scale version of the identity and cross-agent-communication pattern explainx.ai has covered in Claude Code's own cross-session messaging feature — the same design idea, applied at 30,000-agent scale with safety monitoring wrapped around it.
Compute allocation: 6% to safety
The third measurement is the bluntest and, by Anthropic's own account, the least precise: what share of AI R&D compute goes specifically to safety work. Over one examined week in July 2026, about 6% of compute allocated to AI R&D went to safety research, and about 12% of compute allocated to AI-driven AI R&D specifically went to safety. Anthropic classified workloads using a Claude-built classifier that read run metadata and code (for training/eval runs) or session transcripts (for research-agent inference), sampling about 14% of roughly 10,000 weekly runs, weighted toward the highest-compute runs.
Anthropic is upfront that this measurement undercounts safety work in specific ways: a token that advanced capabilities as much as safety wasn't counted as safety spend, and the figures exclude a separate, comparable amount of compute spent on safeguards classifiers. The company frames the value of the number less as an absolute figure and more as "a straightforward mechanism to compare like with like, across developers and over time" — assuming other labs adopt a similar published methodology, which is the open question this whole exercise is really testing.
Honest limitations
- These are Anthropic's own numbers, about Anthropic's own systems, using Anthropic's own Claude models as both the tool that gathered evidence and the judge that rated it — a conflict of interest the post itself names directly, since "the judge model could make the same kinds of errors as the model it is checking."
- No independent third-party verification exists yet for any of the three measurements. Anthropic states it plans to embed outside evaluators with internal-team-level access to check these numbers going forward, but that's a stated intention, not a completed audit.
- The R&D automation index measures a frozen basket of tasks — a rising index tells you existing work is being automated, but doesn't by itself show whether entirely new categories of work are emerging that humans have simply shifted onto (Anthropic ran a separate check for this and found no rise in "novel" tasks between January and July 2026, but that check has the same self-measurement limitation as everything else here).
- Compute allocation measures spending, not effort — a more efficient safety classifier lowers the safety compute share even if safety work itself hasn't decreased, which the post acknowledges directly.
Why the aggregation weighting matters
One methodological detail worth dwelling on, because it's the kind of thing that determines whether the 26% figure is meaningful or gameable: the weight each of the 542 task-tree nodes carries in the final index comes from person-time, not from Anthropic's own judgment about which categories matter most. If ten engineers each split their week across four tasks, each of those forty task-slots gets weighted by how much of that week it consumed — a category everyone touches lightly counts less than one a smaller group works on full-time. This is a deliberate design choice to avoid the obvious failure mode of a self-reported automation index: a lab quietly weighting the index toward categories where automation is already high, to inflate the headline number. It doesn't eliminate the possibility of gaming — the initial category tree itself, and which 20% of staff got sampled each week, are still choices Anthropic controls — but it does mean the number isn't simply "whichever categories we pick, weighted however we like."
What this means for builders and the pacing debate
This post is best read alongside explainx.ai's earlier coverage asking whether "pace the frontier" is a genuine safety commitment or a plateau narrative — these measurements are Anthropic's attempt to move that debate from rhetoric to checkable numbers, even while publishing them unilaterally and without independent audit yet. The most consequential number for anyone tracking recursive self-improvement isn't the 26% headline — it's that Anthropic reports zero measured work at AL5 (fully autonomous, no human in the loop). Whether that stays true as the 26% figure climbs from its "under 1% in February" starting point is the number worth watching next, not this one snapshot.
Related on explainx.ai
- Dario Amodei wants to "pace the frontier" — the actual plan
- What is an embedded evaluator? AI safety, explained
- Is "pace the frontier" safety or a plateau narrative?
- What is recursive self-improvement in AI?
- Anthropic's CEO on METR evaluator salaries ($687K)
- Claude Code cross-session messaging: sessions can now talk to each other
- Emad Mostaque, Elon Musk, and Altman react to "pace the frontier"
- Official source: Anthropic — Measurements for understanding the pace of AI development
This post is sourced to Anthropic's own September 17, 2026 research post and its methodological appendix. All figures — the 26% automation index, the 30,000-agent oversight statistics, and the compute-to-safety ratios — are Anthropic's self-reported measurements of its own internal systems; no independent third-party audit was available at time of writing.
