When the machine cries wolf

Between the 2024-25 school year, 43 percent of US teachers in grades 6 to 12 had started running student writing through an AI detector, according to a Center for Democracy and Technology survey cited by The Verge. Tools like Turnitin, GPTZero and Pangram don't compare text against a database the way plagiarism checkers do; they have their own models guess whether wording, rhythm and sentence structure look machine-made. Turnitin says it wrongly calls human writing AI-generated less than 1 percent of the time, and Pangram claims a rate below 1 in 10,000 -- numbers precise enough to sound like a solved problem.[1]

The precision hasn't matched the fallout. Thierry Rignol sued Yale after a professor used GPTZero to fail him and suspend him for a year; the suit argues detection tools "unfairly target non-native English speakers" like him, echoing a 2023 Stanford finding that detectors flagged non-native speakers' essays more often. An Adelphi University student won a similar case this February. The accusations aren't confined to campus either: Ozzy Osbourne's son Jack accused Verge contributor Kat Tenbarge of using AI on a Rolling Stone piece, citing a detector called Getsolved as evidence, which Tenbarge disputes. Yale, Johns Hopkins, Vanderbilt, Georgetown and MIT have since disabled or restricted the tools; MIT's own guidance states, "AI detectors don't work."[1]

The students nobody flagged

Meanwhile the opposite failure was running for years at a different set of schools. Professor David Song at East Los Angeles College told The Decoder he first noticed the pattern when suspiciously generic Anglo-Saxon names showed up in a history class with a mostly Latino and Asian student body -- enrollments later tied to a scheme that plants fake students at community colleges to draw financial aid, then uses AI to produce their coursework so the fraud stays invisible. No detector caught it; a professor's memory of who is actually in his classroom did.[2]

Both stories share the same hole: nobody has a reliable way to tell, from the outside, whether a specific piece of writing came from a person or a model. Detectors fill that hole with a confident-sounding percentage and end up punishing the wrong people; fraud rings exploit the same gap and get away with it, sometimes for years. It's possible better tooling closes the gap over time -- but until a vendor publishes a false-positive rate and a false-negative rate measured on the same independent sample, instead of just the one that flatters the product, no single number tells you which failure you're more likely to live through.[1], [2]