Measuring the Spread of AI Text Online
Around one-in-ten webpages on .com domains now show signs of being written or edited by artificial intelligence. This marks a significant shift since the release of ChatGPT in November 2022. Researchers at the Pew Research Center analyzed nearly half a million English-language webpages to track how machine-generated content has permeated the internet over the last five years.
AI authorship is not uniform across the web. While .com sites show high levels of AI influence, institutional domains like .edu and .gov remain largely human-authored, with AI indicators present on only about 1% of pages. This divergence suggests that commercial interests are the primary drivers behind the rapid adoption of AI writing tools.
Linguistic Patterns and AI Indicators
AI models are trained on vast datasets of human writing, which leads them to mimic specific linguistic quirks. When researchers analyzed these texts, they identified clear statistical patterns that distinguish machine output from human composition. These indicators include a high frequency of em dashes, the consistent use of Oxford commas, and a repetitive lexicon of specific transition words.
Data shows that since 2023, the use of em dashes in sampled web content has doubled. Similarly, Oxford comma usage rose by 63% during the same period. Phrases often favored by AI models—such as words like "interplay" or "testament"—have more than doubled in frequency across the web. These "tells" allow detection models to flag content that deviates from standard human writing structures.
Methodology and Detection Limits
To perform this study, researchers used the Common Crawl web repository to pull snapshots of the internet dating back to January 2021. They used an open-weight detection model from Pangram to evaluate the likelihood of AI involvement. This model looks for subtle statistical anomalies in word choice and sentence arrangement rather than relying on simple keyword matches.
It is vital to acknowledge that these detection methods are not infallible. Individual documents can be misclassified, and humans often use the same punctuation or phrasing that AI favors. However, when aggregated across hundreds of thousands of pages, the trends become clear. The rise of machine-generated text creates a distinct signature on the internet that grows more pronounced every six months.
The Future of Online Content
As the volume of synthetic text grows, the character of the web continues to change. Algorithms now shape much of the language readers encounter daily, creating an ecosystem where the line between human and machine expression blurs. The data shows that we are currently in a period of rapid transition.
Moving forward, the prevalence of these AI signals will likely remain a focus for researchers and internet users. Tracking this shift is necessary to understand how the internet functions as a source of information. The tools developed by Pew Research and other data scientists provide a baseline for observing how synthetic text alters the quality and authenticity of the digital environment.