Embedded third-party evaluators with "employee-like access" inside Anthropic
“First, we committed to embed external evaluators inside Anthropic with employee-like access, similar to a food inspector, and we recommended that other companies across the world do the same.”
Context: Dario Amodei at the UN Security Council (23 Sep 2026), restating the commitment in his September 2026 essay "We Must Pace the Frontier", which says "Anthropic is unilaterally committing to this step now" and names METR as an example evaluator. No deadline was given; the 31 Mar 2027 horizon is ours.
How we judge it
Our interpretation: correct if by 31 Mar 2027 Anthropic and at least one named independent evaluator (e.g. METR) publicly confirm that an embedded third-party evaluation team has ongoing, employee-like access inside Anthropic (to models and training pipelines); wrong if no such arrangement is confirmed by then.
Resolution criteria are our interpretation of the claim, written before grading. Methodology
- Said
- Resolve by
- Due in 171 days
- Verdict
- Pending
- Stated probability
- None stated
- Specificity
- 2 of 3 (dated, some interpretation)
Evidence & grader’s note
- What changes for
- Government & policy
Share this scorecard
Post on X LinkedIn Share image (PNG)
Embed / Cite
The embedded card is live: it shows the current verdict, with the date the data was read. If a verdict is corrected later, the card changes too, so cite the page and its date for a fixed record.