Anthropic gives six METR researchers employee-level access to assess its models

Anthropic said in a blog post on Oct. 15 that it has started giving employee-like access to a team from METR, a nonprofit that evaluates frontier AI models, so METR can assess its risks and practices.2

Anthropic said it wants to be publicly accountable but cannot share all safety information publicly, because some of it could help malicious actors or is commercially sensitive, so it is giving third-party evaluators deep access instead.2 Anthropic said last month it committed to stronger independent assessment and signed the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators.2

Anthropic said it onboarded six METR employees with company hardware, office space, and access to internal tools and systems roughly on par with what its own risk assessment teams have.2

METR will publish its analysis of Anthropic's risk levels, incidents or practices, and Anthropic said it cannot veto publication but can require redaction of information in narrow, predefined categories such as customer data, attorney-client privileged information and model architecture information.2

Anthropic said METR may characterize the redactions publicly and say whether it believes they were justified, as long as it does not reveal the redacted information.2

METR is the only evaluator at the third tier of Anthropic's three-tier system; first-tier evaluators such as SecureBio need Anthropic's approval to publish, and second-tier evaluators can publish without approval, according to Anthropic and TechCrunch.12