Anthropic said on 15 October it had given six METR evaluators access to its systems broadly comparable to that of its own risk assessment teams, Reuters reported. The nonprofit can assess the company’s advanced models, safety practices and risk management independently.
Key points
- Six METR employees have company hardware, office space and access to Anthropic’s internal tools.
- METR can publish assessments without prior approval, subject to limited redactions.
- The evaluators can publicly say whether they consider a redaction justified, without revealing the withheld material.
- Anthropic describes the arrangement as the most extensive of three access tiers it is developing.
Six METR evaluators gain internal access
Anthropic has provided the embedded METR team with company hardware and office space, as well as access to internal tools and systems. The comparison with Anthropic’s own risk assessment teams concerns the access available to the evaluators, rather than their employment status.
Anthropic said its employees are encouraged to share information with the team much as they would with a colleague working on alignment risk, Unite.AI reported. The company identified exceptions for customer information, legally privileged material and certain details of model architecture.
Anthropic said it takes precautions with architecture information because evaluators may also work with other companies and could inadvertently disclose it. The company considers that information generally less relevant to risk assessment. It said it would usually seek to share architecture details when they were highly relevant, subject to explicit internal approval.
The arrangement covers a small team working under enforceable agreements. Other METR staff, including researchers involved in separate projects with Anthropic, are kept apart from the embedded team’s work. Anthropic described the access as a way for qualified evaluators operating under its security standards to examine its practices closely.
METR can contest Anthropic’s redactions
METR has the right to publish independent assessments of Anthropic’s risks, incidents and safety practices without the company’s prior approval. Anthropic can require limited redactions for categories including customer data, privileged information and certain model architecture details, Reuters reported.
Those redactions do not prevent METR from publicly describing the restrictions or saying whether it considers them justified, provided it does not reveal the withheld information. Anthropic cannot veto publication under the arrangement, Unite.AI reported.
Researchers and evaluators had raised concerns that AI labs could exert too much influence over independent reviews by controlling access to sensitive information and restricting publication, Reuters reported. METR’s publication rights cover its analysis of the company’s risk levels, incidents and practices.
Anthropic develops three evaluator access tiers
The METR arrangement follows Anthropic’s September 2026 commitment to strengthen independent, third-party assessment of its models and safety practices. Anthropic said it had worked with METR on several reviews of its Risk Reports over the preceding year.
Anthropic signed the White House Accord on Super Intelligence, under which participating companies pledged to work with independent external auditors or evaluators. The company said some information relevant to its safety work could be useful to malicious actors or was commercially sensitive, including details of compute and training.
Anthropic said evaluators of frontier models need expertise in assessing their capabilities, access to information beyond the latest models, and freedom to publish observations. It also identified independence as a requirement, including the absence of equity in the company being assessed or financial or operational dependence on it.
Anthropic is developing three tiers of evaluator access. In the first, it controls access for narrowly scoped investigations and must approve publication. In the second, evaluators have broader access and can publish without approval, although Anthropic can require the removal of specified categories of information. The METR arrangement is the third and most extensive tier, with employee-like access and publication rights.
