"Probably in 2025" an AI that can be a mid-level engineer
“Probably in 2025, we at Meta, as well as the other companies that are basically working on this, are going to have an AI that can effectively be a sort of mid-level engineer that you have at your company that can write code”
Context: Said on The Joe Rogan Experience (January 2025); quoted by Fortune on Jan 24, 2025. Episode #2255 released Jan 10, 2025 (podcast listings).
How we judge it
Our interpretation: by 31 Dec 2025, Meta or a peer deploys an AI agent doing the work of a mid-level software engineer end-to-end.
Resolution criteria are our interpretation of the claim, written before grading. Methodology
- Said
- Resolve by
- Overdue by 282 days
- Verdict
- Open
- Stated probability
- None stated
- Specificity
- 1 of 3 (vague or hedged)
Evidence & grader’s note
- METR on X: Claude Opus 4.5 50%-time horizon about 4 hrs 49 mins on its software tasks (Dec 2025)
- METR on X: Opus 4.5 80%-time horizon 27 minutes (Dec 2025)
- METR: experienced developers took 19% longer with early-2025 AI tools (Jul 2025)
- Reuters (via Yahoo Finance): Zuckerberg says AI agents progressing slower than expected (Jul 2, 2026)
Due for grading. Evidence collected so far: in Dec 2025 METR put Claude Opus 4.5, its highest published result at the time, at a 50% time horizon of about 4 hours 49 minutes on its software tasks (27 minutes at 80% reliability); in July 2025 METR found that experienced developers using early-2025 AI tools took 19% longer on their tasks; in July 2026 Zuckerberg told Meta staff that AI agents had not progressed as quickly as he had expected. Editor review of the verdict is pending.
