Eigen RadarAI
Analysis

Role prompts shift clinical AI decisions despite unchanged case details

A peer-reviewed experiment across 20 open-weight language models found that short descriptions of a physician’s decision style changed their willingness to recommend action. Affirmative responses ranged from 36.9 per cent to 46.4 per cent while clinical information stayed fixed. The study measures decision tendencies rather than medical correctness, using simulated emergency cases and de-identified discharge summaries.

Artificial Intelligence··Morning
On a table in a research room beside a hospital emergency department: nine closed case folders with different coloured tabs, a stethoscope, face-down discharge summaries and a local workstation tower; an empty emergency bay behind glass.

Fixed case details still produce different decisions

Short instructions about a physician’s decision style changed how often 20 language models recommended action in a peer-reviewed experiment. Without a role description, affirmative answers accounted for 42.8 per cent of decisions. Across the specified roles, that share ranged from 36.9 per cent to 46.4 per cent, a difference of 9.5 points under unchanged clinical information.[1]

Nine roles combine ethical priorities and thinking styles

Eyal Klang and colleagues paired duty-based, care-based and utilitarian priorities with intuitive, integrative and analytic styles. The experiment used nine role descriptions. Models answered binary questions covering treatment, safety and autonomy as well as follow-up and resource use. Response format and task instructions stayed fixed, and each condition was repeated ten times. The open-weight models ran locally, without tools, retrieval, memory or self-correction.[1]

Two clinical collections show similar directions of change

The material combined 2500 simulated emergency cases with MIMIC-IV-Note clinical texts: 2500 discharge summaries concerning adults, with identifying information removed. The researchers reported 85.7 per cent directional agreement across the collections. Medical fine-tuning and model size offered no consistent protection against framing effects. Published on November 14 in npj Health Systems, the study did not assess whether recommendations followed clinical guidelines or improved patient outcomes.[1]

References

  1. News sourcenpj Health SystemsPeer-reviewed study reports clinical model decisions shift with persona prompts↩1↩2↩3