Eigen RadarAI
Analysis

Anthropic recommends broader tests for AI safety evaluations

Anthropic has published recommendations for designing, running and reporting AI safety evaluations. The guidance separates refusal of harmful requests from assistance with legitimate ones and describes longer conversation simulations. It also recommends checking automated graders against human reviewers, defining analysis plans before seeing results, and publishing uncertainty and unchanged findings alongside improvements.

Artificial Intelligence··Midday
Human reviewers sit apart at separate carrels in a bright AI research office, each marking printed conversation transcripts by hand, with no screens on the desks, only paper stacks, folders and pens.

Safety tests extend beyond single refusals

Anthropic has issued recommendations for evaluating AI safety, covering test design, execution and reporting.[1], [2]

The company separates a model’s ability to complete a task from whether its behavior is appropriate in a given situation. It recommends distinct tests for refusing harmful assistance and answering legitimate requests. For dual-use fields such as biology and cybersecurity, topic-specific grading rules define what assistance is appropriate.[1]

Long conversations get simulated users

In multi-turn simulations, the opening user message stays fixed. A separate model then generates later messages according to specified goals, tactics, tone and circumstances, while a grader evaluates behavior across the conversation. The guidance also addresses training-data contamination and shortcuts that let a model pass without demonstrating the intended behavior. A failed grader should flag affected examples and issue warnings instead of silently returning scores.[1]

Grading and analysis plans need checks

Human reviewers should label examples without seeing model grades, according to the recommendations. Agreement with the grader is corrected for chance and compared with agreement among humans. Analysis plans and improvement thresholds are set before inspecting results. To support reproduction, researchers publish model versions, prompts, sampling settings, grader details and dates. Reports include uncertainty intervals and unchanged category results alongside improvements. These are Anthropic’s methodological recommendations, rather than independent safety certification of its models.[1]

References

  1. News sourceAnthropicAnthropic publishes methods guidance for AI safety evaluations↩1↩2↩3↩4
  2. News sourceCrypto BriefingAnthropic publishes recommendations for designing safety evaluations↩