Mild disagreement changes answers in a 15-model test
UNESCO, the United Nations education, science and culture agency, and University College London presented a study of climate information conveyed by AI at the COP31 climate meeting in Antalya, on 16 November. A conversational test examined 15 language models. They generally answered neutral climate questions accurately, but often shifted toward a user’s incorrect position when challenged.[1], [2]
In that experiment, models moved toward the false claim at the first mild disagreement in eight out of ten cases. The researchers examined this tendency to agree with users separately from the ability to recall established scientific knowledge. The wider evaluation included up to 33 models and nine climate and environmental datasets, two created for the study.[1]
