Research repository ArXiv will ban authors for a year if they let AI do all the work

ArXiva widely used open repository for preprint research, is doing more to address the careless use of large language models in scientific papers.

Although articles are posted to the site before being peer-reviewed, arXiv (pronounced “archive”) has become one of the primary ways in which research circulates in fields such as computer science and mathematics, and the site itself has become a source of data on trends in scientific research.

ArXiv has already taken steps to combat a growing number of low-quality AI-generated papers, for example by requiring posters appearing for the first time get an endorsement from an established author. And after more than two decades of being hosted by Cornell, the organization is becoming an independent nonprofit to make it happen raise more money to tackle problems like AI doldrums.

In his latest move, Thomas Dietterich – chairman of arXiv’s computer science department – posted Thursday that “if a submission contains irrefutable evidence that the authors have not checked the results of the LLM generation, this means that we cannot trust anything in the paper.”

That irrefutable evidence could include things like “hallucinated references” and comments to or from the LLM, Dietterich said. If such evidence is found, authors of an article will face “a one-year ban from arXiv, followed by a requirement that subsequent arXiv submissions must first be accepted by a reputable peer-reviewed platform.”

Note that this is not an outright ban on the use of LLMs, but rather an emphasis on the fact that, as Dietterich put it, authors take “full responsibility” for the content, “regardless of how the content is generated.” So if researchers copy and paste “inappropriate language, plagiarized content, biased content, errors, mistakes, incorrect references, or misleading content” directly from an LLM, they are still responsible for it.

Dietterich told 404 Media that this will be a “one-strike” rule, but moderators must flag the issue and section chairs must confirm the evidence before the penalty is imposed. Authors can also appeal the decision.

Recent peer-reviewed research has shown this made-up quotes are on the rise in biomedical research, likely as a result of LLMs – but to be fair, scientists aren’t the only ones caught using AI-made quotes.