METR report: Anthropic's AI models improving about twice as fast as a year ago
For the chief risk officer at a bank using Anthropic models
The AI safety nonprofit METR published a report on Thursday saying Anthropic's AI models are improving roughly twice as quickly as a year ago, following an 11-week review inside the company, according to The Verge.12
METR said it is confident the acceleration did not come from more computing power, and that the majority of it as of October "is due to AI," according to The Verge's account of the report.12 METR based this on Anthropic internal data, employee interviews and Slack polls, according to The Verge and AP; the report is described as published Thursday, and the sources do not link to the METR document itself.12
Benchmark scores suggest "that Anthropic has made roughly two years of historical progress over the last year," METR wrote, according to The Verge.1
METR said Anthropic finished training three models it has not released: Model 2 in July, Model 3 in August-September and Model 4 in September-October, according to The Verge.1
Anthropic redacted portions of the public report, and METR wrote on X: "For the public version, Anthropic redacted a majority of the evidence we found most informative."1
Anthropic co-founder Sam McCandlish said on X that sharing exactly what is driving the speedup is commercially sensitive and risks handing out a roadmap for others to go faster, according to The Verge, which also reported that he said the company plans to publish its own report on the speed of AI progress "in the coming weeks."12
If a model vendor's capabilities change this fast, a bank's annual vendor review may cover a product that has already been replaced. Tying model risk and third-party reviews to each new release, not the calendar, would match the pace METR describes, and the redactions mean banks cannot verify the underlying evidence themselves.
McCandlish said Anthropic plans to publish its own report on the speed of AI progress "in the coming weeks," along with new safety steps.1
Our take. Your annual review of an AI vendor checks a model once a year; METR says Anthropic made two years of progress in the last one and finished three unreleased models in six months. Tie revalidation to releases, not the calendar.
Action. Model risk: Add a trigger to revalidate any Anthropic-based model at each vendor model release, not on the annual cycle.
What our records say
Anthropic sells Claude to banks directly and through cloud marketplaces, and in 2026 deepened work with Goldman Sachs and FIS while JPMorgan withdrew Claude for Hong Kong staff (Jun 18, 2026).12 Anthropic sells banks Claude seats and API access directly and through cloud marketplaces, and in 2026 added embedded engineering with Goldman Sachs and FIS, restricted cyber-defence access to its Mythos model for a few large banks, and a services firm co-founded with Goldman Sachs.1
From our records
- Claude (Anthropic)vendor · banks using it (21) Goldman, JPMorgan, BankUnited · alternatives Microsoft Copilot, ChatGPT Enterprise, Gemini Enterprise