AI prose study finds language patterns linked to processing difficulty
A study of 14 AI models found frequent use of linguistic constructions associated with processing difficulty, including unusual word combinations and compressed clauses. Comparisons used human writing and reworded versions of generated text. A separate small-model experiment showed that penalizing long outputs can change grammar. The research measured language features rather than readers’ comprehension or reading speed.
Artificial Intelligence··Midday
A language study examines the effort of reading AI prose
Language researchers Thomas Hikaru Clark, Matias B. Martin and Kyle Mahowald investigated linguistic properties of AI-generated text in a study titled Why is AI-generated language so hard to understand?[1], [2]
Their work asks whether model outputs rely on constructions associated with greater processing demands. The subject is the structure of generated language, examined through comparisons and a separate training experiment.[1]
The corpus covered 14 model variants across conversation, coding and essay tasks, although coverage differed by setting: 12 models in chat, nine in coding and 14 in essays. Researchers studied six groups of features associated with processing difficulty. These included distances between grammatically connected words, stacked nouns, unusual word combinations and relative clauses in which optional words were omitted.[1]
Human writing and reworded passages provide comparisons
Human comparison texts came from OpenSubtitles, the subtitle collection; StackExchange, the question-and-answer platform; and student essays. Wikipedia supplied another reference. Researchers also used NLLB-200, a translation model, to translate English into Japanese and back into English. The round trip offered alternative wording for approximately preserved content, allowing the original model output to be compared with a rephrased version.[1]
Leading models more often used unusual word combinations, verbs derived from nouns, inanimate subjects paired with verbs normally associated with living actors, and compressed relative clauses. Some Olmo and Llama variants showed weaker effects. These measurements concern language features linked by earlier research to processing difficulty; the study did not directly measure people’s reading speed or comprehension of the generated passages.[1]
A length penalty changes a small model’s grammar
A separate experiment adapted Qwen2.5-1.5B-Instruct, a small language model, to turn structured event descriptions into sentences. Increasing the penalty for longer outputs increased its preference for shorter grammatical alternatives. Some preferences also appeared in constructions absent from training. Transfer to unrelated grammatical patterns was weaker, appearing only for possessives under one training condition. This establishes a limited example of length penalties affecting grammar, while the suggested explanation for commercial models’ training remains a hypothesis.[1]
