Chatbots’ Amazon-message judgments change when examples stand alone
An informal comparison by journalist Haroun Adamu found that Gemini, ChatGPT and Claude correctly classified a batch of 20 Amazon-style messages. When genuine messages were presented separately in fresh chats, Gemini and Claude sometimes became suspicious, while ChatGPT recognized them. The self-constructed examples and limited runs describe this comparison, rather than a general performance ranking.
Artificial Intelligence··Night
Twenty messages in one batch
Haroun Adamu used the free tiers of three chatbots—Gemini, ChatGPT and Claude—to examine 20 Amazon-style emails and texts. He constructed 15 scam examples inspired by warnings and added five examples resembling genuine correspondence. When presented with the whole set together, all three chatbots classified every message correctly. The exercise used examples selected and prepared by the journalist himself.[1]
Genuine messages in fresh chats
Adamu then opened fresh chats and separately presented genuine messages about an account sign-in and a refund. Gemini and Claude expressed suspicion in some answers, including doubts about the refund message, while ChatGPT recognized both as genuine. The sign-in alert contained no links. Responses differed from the flawless classification obtained when the larger, mixed collection had been supplied together.[1]
Advice also varied by model
A further hypothetical prompt asked what to do after disclosing a password and card information. The models offered different response checklists; Adamu found wider coverage in Gemini’s answer and omissions in Claude’s. This was an informal exercise with self-constructed examples and limited runs. Its results describe those exchanges, without establishing how the services would rank across a representative collection of real scams.[1]