Anthropic recommends broader tests for AI safety evaluations
Anthropic has published recommendations for designing, running and reporting AI safety evaluations. The guidance separates refusal of harmful requests from assistance with legitimate ones and describes longer conversation simulations. It also recommends checking automated graders against human reviewers, defining analysis plans before seeing results, and publishing uncertainty and unchanged findings alongside improvements.
Artificial Intelligence··Midday
Safety tests extend beyond single refusals
Anthropic has issued recommendations for evaluating AI safety, covering test design, execution and reporting.[1], [2]
The company separates a model’s ability to complete a task from whether its behavior is appropriate in a given situation. It recommends distinct tests for refusing harmful assistance and answering legitimate requests. For dual-use fields such as biology and cybersecurity, topic-specific grading rules define what assistance is appropriate.[1]
Long conversations get simulated users
In multi-turn simulations, the opening user message stays fixed. A separate model then generates later messages according to specified goals, tactics, tone and circumstances, while a grader evaluates behavior across the conversation. The guidance also addresses training-data contamination and shortcuts that let a model pass without demonstrating the intended behavior. A failed grader should flag affected examples and issue warnings instead of silently returning scores.[1]
Grading and analysis plans need checks
Human reviewers should label examples without seeing model grades, according to the recommendations. Agreement with the grader is corrected for chance and compared with agreement among humans. Analysis plans and improvement thresholds are set before inspecting results. To support reproduction, researchers publish model versions, prompts, sampling settings, grader details and dates. Reports include uncertainty intervals and unchanged category results alongside improvements. These are Anthropic’s methodological recommendations, rather than independent safety certification of its models.[1]
Related columns
For more information on this topic, you can read the related columns.