The Google Books index, which has become a valuable resource for academics, has started indexing low-quality books, which may affect the Ngram language tracking tool. A website called 404Media found some books in the Google Books index that appear to be automatically generated by AI systems instead of being written by humans.
404Media searched for the phrase “as of my last knowledge update” on Google Books. This is a phrase that is commonly used by chatbots like ChatGPT. When they searched for this, most of the results were books about AI technology. However, some books in the results were unrelated to AI and seemed like they could have been written by bots.
One example book they found was called “Bears, Bulls, and Wolves: Stock Trading for the Twenty-Year-Old”. This book only included basic facts about financial events from Wikipedia and contained the chatbot-like phrase. Other books on topics like Twitter only discussed information that was current as of 2021, when AI systems would have last received training data.
Google Books scans published works from as far back as the 1500s and makes this text searchable. It has become a vital research tool used widely by academics. The popular Ngram viewer also uses data from Google Books to track how language has changed over hundreds of years. Ngram analyzes written works in Google Books to show trends in word and phrase usage over time.
While books recently added to Google Books do not show up in Ngram yet, Google told 404Media that future updates to Ngram’s underlying data could include texts from these automatically generated books. Indexing low-quality AI-written works could introduce inaccurate or meaningless results if analyzed by the powerful Ngram language discovery tool.
