Anthropic gives six METR evaluators internal access and publication rights
Six METR researchers have received Anthropic hardware, office space and internal access comparable to the company’s risk teams. The outside evaluator can publish assessments without an Anthropic veto, while narrowly defined sensitive information may be removed. METR can publicly challenge those redactions. The arrangement implements a stronger access tier following earlier work reviewing Anthropic’s risk reports.
Artificial Intelligence··Morning
Six outside researchers receive internal tools
Anthropic has embedded six employees of METR, a nonprofit that evaluates advanced AI systems, with company hardware, office space and access to internal tools. Announced on October 15, the arrangement gives the team access broadly comparable to Anthropic’s own risk assessment staff. Company employees are encouraged to share information with them as they would with colleagues working on alignment risk, subject to defined exclusions.[1], [2]
Customer information, legally privileged communications and model architecture information fall within the excluded categories. Anthropic says architecture details highly relevant to risk assessment may generally be shared after explicit internal approval. The engagement covers a small team under enforceable agreements. Other METR researchers, including people collaborating with Anthropic on separate projects, are kept apart from the embedded team’s work.[1], [2]
METR may publish without a company veto
METR can publish assessments of Anthropic’s risk levels, incidents and practices without the company vetoing publication. Anthropic retains the right to require removal of information within narrow categories specified in advance. METR may describe those removals publicly and state whether it considers them justified, while keeping the protected information confidential. Publication rights therefore accompany the researchers’ broader access to the company’s systems.[1], [2]
The arrangement adds a third access tier
Anthropic is developing three evaluator access tiers. The first allows narrowly scoped investigations with company-selected access and publication requiring company approval. The second gives evaluators more information and freedom to publish without advance approval, while preserving specified redaction rights. METR’s third-tier arrangement adds employee-like access to publication terms similar to the second tier. Anthropic says it hopes to embed several such teams over time.[1], [2]
The new access follows METR’s earlier work reviewing Anthropic’s risk reports over the past year and the company’s September commitment to stronger independent assessment. Anthropic describes expertise, independence, sufficient information and freedom to publish as necessary for this work. It says some training and safeguard information cannot be released openly because it is sensitive or could assist malicious actors. The embedded team’s work is being introduced voluntarily. It says evaluators should hold no company equity and should not depend on it financially or operationally.[1], [2]
