Two reading-time measures point in opposite directions
The paper's central result is a split between two measures that sound interchangeable but are not. Predicted total reading time estimates how long an entire text takes to read and process; predicted mean reading time per word captures the average processing time attached to each word. LLM-generated rewrites scored lower on the first and higher on the second.
The numbers come from 496 human-written articles and their LLM rewrites, each rewrite instructed to preserve the original's information while freely changing wording, sentence structure and organization. LLM texts averaged 115.6 s of predicted total reading time against 117.1 s for human text, a mean relative change of −1.19%. Per word, LLM texts averaged 233.4 ms against 229.9 ms, a change of +1.57%. All reported Human–LLM differences remained significant after correction for multiple comparisons (p adj < .01).
Text length explains how both can be true. LLM rewrites averaged 496.4 words versus 511.2 for the originals, a 2.66% reduction. Fewer words offset the higher per-word cost, so the total falls only slightly. The paper's own reading is that conciseness should not be equated with cognitive ease: cutting words does not necessarily cut the effort each remaining word demands.
Reading time was predicted, not measured
Directly eye-tracking hundreds of article pairs would require many participants and would not scale, so the authors built RoBERTa-RT, a RoBERTa-based model trained on existing human eye-tracking corpora to predict reading time for individual word occurrences. Gaze fixations on a word are summed into an observed reading time, and the model is trained by minimizing mean squared error between observed and predicted times.
Because the target is defined per word, a word split into several subword tokens has those token representations averaged into one vector before an MLP regression head produces the prediction. Texts are processed one paragraph at a time. Training combined data from the OneStop, GECO and MECO eye-tracking datasets, and the model was evaluated on internal and external test sets before being applied to the unseen article pairs.
This design is what makes the comparison possible at scale, and it is also the main limit on what the results mean. The reported times are model predictions for texts no participant read in the experiment. The authors state that further studies with human participants are needed to establish whether these differences translate into measurable changes in cognitive effort and comprehension.
Rewrites use longer, rarer and more diverse words
Four linguistic measures were compared across the pairs, and all four shifted systematically. Mean word length rose from 4.95 to 5.25 characters (+6.06%). Mean word frequency fell 7.61%, meaning more use of uncommon words. Lexical diversity, measured with MTLD, rose 31.83%. Mean surprisal fell 6.51%, meaning words were on average more predictable from their preceding context.
Correlations with the change in predicted per-word reading time show which shifts line up with higher cost. Word-length change correlated positively (ρ = .588), lexical-diversity change positively (ρ = .292), and word-frequency change negatively (ρ = −.405), so larger frequency drops accompanied larger per-word increases. Surprisal behaved differently: its change correlated positively (ρ = .438), meaning bigger surprisal reductions accompanied smaller per-word increases.
The apparent tension between rarer words and greater predictability is resolved by what the two measures describe. Word frequency reflects how common a word is in general language use, while surprisal depends on the specific context. A text can use relatively uncommon words while placing them where they are highly expected. The net result is still a higher predicted per-word reading time for LLM text, so the predictability gain only partly offsets the lexical cost.
Stronger compression comes with a higher per-word cost
The paper also asks whether how much a rewrite shortens its source matters. Using relative change in word count as the compression measure, greater shortening correlated negatively with the change in predicted per-word reading time (ρ = −.458). Pairs where the LLM cut more words tended to show larger per-word increases.
The linguistic correlations follow the same direction. Greater word-count reduction accompanied larger increases in mean word length (ρ = −.303) and lexical diversity (ρ = −.269), and larger decreases in mean word frequency (ρ = .364). It also accompanied smaller reductions in surprisal (ρ = −.362), so the predictability advantage of LLM text weakened as compression grew.
The authors call this a compression–reading cost trade-off. It is a correlation across the 496 pairs, not a demonstrated causal effect of shortening text, and it describes predicted times rather than observed ones. Within those bounds, it sharpens the practical picture: the more aggressively a rewrite condenses, the more the saved length appears to be paid back in per-word processing demand.
Efficiency for the writer is not ease for the reader
The paper frames its contribution as moving Human–LLM comparison beyond linguistic description toward reading effort. Its answer to whether one kind of text is simply easier to read is no: LLM rewrites reduce the amount of material to process while concentrating greater predicted demand on individual words. The authors describe this as a redistribution of reading demands rather than a uniform improvement or decline.
That framing matters for anyone judging machine-generated content by length or conciseness. In this dataset, the shorter text was not the one with lower predicted effort per word, and the total-time advantage was modest at about 1.5 s on average. The paper argues that the efficiency of machine-generated writing should be assessed from the reader's side as well as from content production.
The scope is specific: 496 English article pairs, rewrites prompted to preserve information, and reading times predicted by a model trained on human eye-tracking data. The results do not establish comprehension outcomes, and they do not show that any particular reader or population would experience these differences. They do identify where to look: per-word processing cost, not just word count, is the measure that separates the two text types.
Judge rewrites by per-word cost, not length alone
If one number from this study should guide how LLM rewrites are evaluated, it is predicted reading time per word, not total predicted time or word count. The per-word measure is where the two text types diverge most clearly (+1.57%), while the total-time difference (−1.19%) is small and partly an artifact of shorter texts. The compression analysis adds a threshold-like pattern: the more a rewrite shortens its source, the larger the per-word penalty tends to be.
The linguistic correlations point to the same priority. Word length showed the strongest association with per-word cost (ρ = .588), ahead of surprisal (ρ = .438), word frequency (ρ = −.405) and lexical diversity (ρ = .292). A rewrite that swaps short, common words for longer, rarer ones is the case where the predicted per-word cost rises most, even if the resulting text is shorter and more contextually predictable.
These are associations in predicted times, and the authors' own caveat stands: human-participant studies are needed to test whether the pattern reaches actual cognitive effort and comprehension. Until then, the paper's most useful claim is comparative rather than absolute. LLM rewrites did not make reading uniformly cheaper; they shifted where the predicted cost sits, from the number of words to the difficulty of each one.
Sources used in this answer
The Cognitive Cost of LLM Writing: A Reading-Time Comparison with Human Text
Across 496 paired articles, LLM rewrites were shorter and had slightly lower predicted total reading time (115.6 s vs 117.1 s) but higher predicted reading time per word (233.4 ms vs 229.9 ms), consistent with a trade-off between conciseness and per-word processing cost.