Can punctuation, word frequency, and sentence length really prove AI authorship?
Yes, but not in isolation. These features are powerful when combined. A 2023 study used 20 features—including word frequency (e.g., scientists' use of 'but,' 'however,' 'although') and sentence length—to distinguish human academic writing from ChatGPT with over 99% accuracy [3]. The key is that humans tend to write longer sentences and use more hedging language, while AI texts are more uniform.
Word frequency alone is revealing. A 2025 study found that AI-generated texts have a significantly smaller vocabulary than human-written ones, and the ratio of words that appear only once (hapax legomena) to total different words was a strong classifier [1]. Using just six features—including frequencies of the four most common words and that ratio—the researchers could 'fairly confidently' determine if a text was computer-generated [1].
Sentence length and structure also differ. A 2025 comparative study of 20 essays (10 human, 10 AI) found that human sentences are longer on average, and while both rely heavily on simple sentences, 60% of AI texts had no compound-complex sentences at all, whereas human writing showed more variation [7]. This means AI tends to avoid complex sentence structures, a pattern that can be detected.
How accurate are these linguistic features in real-world detection?
Very accurate when combined with machine learning. A 2023 study using features like syllable count, word length, sentence structure, and punctuation ratios achieved 93% accuracy in detecting AI-generated text [4]. The same study found that stylistic indices like Herdan's C (a measure of vocabulary richness) and Simpson's Index (diversity) were the most influential features [4].
Even higher accuracy is possible. A 2026 study combined linguistic features (including perplexity and readability) with a DistilBERT model, achieving 99.45% accuracy on one dataset and 96.23% on another [2]. A 2025 study using a Bi-GRU model with linguistic features like perplexity, readability, and syntactic complexity reached 98% accuracy on one dataset and 72% on a more challenging one [6]. These results show that while no single feature is perfect, combining them with modern AI models yields extremely reliable detection.
However, performance drops when AI is instructed to evade detection. A 2023 study found that when ChatGPT was asked to write in a way that 'a human would not recognize it as AI,' classification F1-scores fell from over 96% to just over 78% for rephrased texts [5]. This means that sophisticated users can reduce the effectiveness of these features, though detection still outperforms tools like GPTZero by 183.8% in F1-score for basic rephrasing [5].
What are the catches and limitations?
The main limitation is that these features are not absolute—they work best in combination and with context-specific models. For example, the 99% accuracy study [3] was trained specifically on academic science writing; it may not generalize to casual blog posts or fiction. Similarly, the 2025 study on word frequencies found that rare words alone (those not in the top 1,000) were not enough to confidently classify texts [1].
Another catch: AI models are improving rapidly. The 2023 study that achieved 93% accuracy [4] used features like punctuation ratios, but newer AI models (like GPT-4.5 or Gemini) may mimic human punctuation patterns more closely. The 2026 study [2] explicitly notes that state-of-the-art LLMs like GPT-4.5 and DeepSeek produce 'highly sophisticated, human-like text,' making detection harder.
Finally, these methods require a baseline of human writing for comparison. Without knowing the typical word frequencies or sentence lengths of a specific human author, it's harder to prove AI authorship definitively. The evidence is strong for group-level differences (humans vs. AI in general), but less reliable for individual cases, especially if the human writer has an unusual style.
About These Sources
This answer is built on 7 peer-reviewed studies — published from 2023 to 2026, 4 from 2024 or later, 1 in Q1 journals, collectively cited 179 times — selected as the most relevant from 7 studies that passed quality screening, drawn from 58 papers retrieved from a database of over 500 million.
Sources used in this answer
Comparative Statistical Analysis of Word Frequencies in Human-Written and AI-Generated Texts
AI texts have a significantly smaller vocabulary than human texts; the ratio of hapax legomena (words appearing once) to total different words is a strong classifier, and using six features (including frequencies of the four most common words) allows fairly confident classification [1].
Synergizing linguistic features and transformer networks for detecting AI-generated text
Combining linguistic features with a DistilBERT model achieved 99.45% accuracy on the HC3 dataset and 96.23% on the M4GT dataset for distinguishing AI from human text [2].
Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools
Using 20 features including word frequency (e.g., 'but,' 'however') and sentence length, a model distinguished human academic writing from ChatGPT with over 99% accuracy [3].
Detecting and Unmasking AI-Generated Texts through Explainable Artificial Intelligence using Stylistic Features
Features like syllable count, word length, punctuation ratios, and stylistic indices (Herdan's C, Simpson's Index) achieved 93% accuracy in detecting AI-generated text [4].
Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT
Best systems for classifying basic human vs. AI texts achieved F1-scores over 96%, but when AI was instructed to evade detection, scores dropped to just over 78% for rephrased texts [5].
Detecting AI-Generated Text: A Bi-GRU with Linguistic Features Approach
A Bi-GRU model using linguistic features (perplexity, readability, syntactic complexity) achieved 98% accuracy on the DAIGT dataset and 72% on the more challenging Deepfake dataset [6].
SENTENCE STRUCTURE IN HUMAN AND AI-GENERATED TEXTS: A COMPARATIVE STUDY
In a corpus of 20 essays, human sentences were longer on average; 60% of AI texts had no compound-complex sentences, while human writing showed more variation in sentence structure [7].