This tool should be able to recognize almost any AI text
Researchers from the USA have developed a new tool that can apparently recognize AI texts with a probability of 99 percent. To do this, they tested the approach using scientific documents as an example. The backgrounds.
ChatGPT makes it possible for everyone to have access to artificial intelligence. However, this does not only have advantages. In the past, the OpenAI language model has often been misused.
Contents
ChatGPT used for fraud and forgery
Criminals used AI to commit crimes in some cases and students let them do their homework. In journalism, too, ChatGPT has occasionally been commissioned to write articles.
“The distinction between human writing and artificial intelligence is now both critical and urgent,” writes a research group from the University of Kansas in a new one science paper. They have developed a new method to distinguish ChatGPT texts from those of human scientists – and very precisely.
AI vs. AI: New tool can correctly recognize 99 percent of AI texts
“With a set of 20 characteristics, we created a model that assigns the author as a human or AI with an accuracy of over 99 percent,” the research paper says.
The experts fed the AI model with stylistic and content-related criteria in which the respective texts differ. They used articles from the science magazine Science and ChatGPT products.
They divided the typical differences between the two types of text into four categories: paragraph length, sentence length differences, punctuation differences, and popular words.
Human vs. ChatGPT: This is how the texts differ from each other
The research shows that the paragraphs produced by ChatGPT are significantly less complex and shorter than the texts from the science magazine. Humans, on the other hand, form much longer sentences and have a preference for ambiguous language.
They also frequently use words such as “but” and “although”. And the research group was also able to identify a penchant for punctuation marks, numbers and proper names when comparing the two texts.
The scientists from Kansas used the XGBoost software library to train their model. To test their performance, the researchers chose a variant of leave-one-out cross-validation.
“The chosen method removed the possibility that any examples from a specific author or a single paper, in the case of ChatGPT, could be used to make it easier to identify other paragraphs within the same paper (by the same author). The model therefore does not rely on having previous examples of work by the human author whose essay is being classified,” the researchers said.
Not the first tool to recognize AI texts
The model created by the Kansas researchers is not the first attempt to recognize AI-generated text. In the past, even ChatGPT’s developer OpenAI has released a tool designed to distinguish AI texts from human creations.
And still other software companies offer various tools designed to accomplish the same task. The hit rate should still be lower than with the new model from Kansas.
Also interesting:
