SemiAnalysis: best AI model scores 12% on GDPx, zero on finance and legal
For the cto of a bank weighing agent vendors
SemiAnalysis released GDPx on Nov. 3, an agentic benchmark that tested 13 frontier AI models across 206 tasks in 11 professional domains, and the top-ranked model, Anthropic's Claude Opus 5.5, scored 12.0%, according to a TechTimes report.1
SemiAnalysis authors Max Kan, Andrew Megalaa and Dylan Patel built GDPx to replace OpenAI's GDPval, which requires each task to fit inside a single prompt, according to the TechTimes report on the release.1 GDPx instead places models in simulated workspaces with real tools, files and APIs, built by 14 partner startups, with a default limit of eight hours per task.1
Finance and legal tasks returned a binary score of zero for every model, so SemiAnalysis switched those two domains to partial credit to measure relative capability, according to TechTimes.1
The finance environment, built by Living Context, reconstructs a real $880 million private-equity acquisition of a snack manufacturer across roughly 4,000 anonymized files.1
The legal environment, built by Chakra Labs, is a Series B financing redline task.1
Gemini 4 Carbon ranked second at 10.9% and GPT-6 Astra third at 9.6%; SemiAnalysis says Opus 5.5 and GPT-6 Astra are held back by safety classifiers in cybersecurity, biology and alignment, not in finance or legal.1
Vendor benchmark scores built on single-prompt tests may overstate how well AI agents handle deal-room finance and legal work, so banks should test vendor claims in their own document environments before committing to deployments. This is based on one report of SemiAnalysis's own benchmark, not independent replication.
Our take. Vendor decks quote benchmark scores; on GDPx, every model scored zero on finance and legal work done the way analysts do it. Approve agent autonomy in those functions only after a vendor passes your own task, not theirs.
Action. Technology risk: Require vendors to run agent pilots on your own finance and legal files, graded pass or fail, before granting autonomy.
What our records say
Anthropic sells banks Claude seats and API access directly and through cloud marketplaces, and in 2026 added embedded engineering with Goldman Sachs and FIS plus restricted Mythos access for a few large banks.1 Its record shows broad adoption, with a retreat in Hong Kong: JPMorgan's Hong Kong staff could no longer select Claude models over language in Anthropic's usage terms, following a similar move by Goldman Sachs (Jun 18, 2026).2
From our records
- Claude (Anthropic)vendor · banks using it (21) Goldman, JPMorgan, BankUnited · alternatives Microsoft Copilot, ChatGPT Enterprise, Gemini Enterprise
- ChatGPT Enterprise (OpenAI)vendor · banks using it (14) Morgan Stanley, BNY, Customers Bank · alternatives Microsoft Copilot, Claude, Gemini Enterprise
- Gemini Enterprise (Google Cloud)vendor · banks using it (9) Wells Fargo, BNY, Citi · alternatives Microsoft Copilot, ChatGPT Enterprise, Claude