Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals
Probabilistic text generators have been used to produce fake scientific
papers for more than a decade. Such nonsensical papers are easily detected by
both human and machine. Now more complex AI-powered generation techniques
produce texts indistinguishable from that of humans and the generation of
scientific texts from a few keywords has been documented. Our study introduces
the concept of tortured phrases: unexpected weird phrases in lieu of
established ones, such as 'counterfeit consciousness' instead of 'artificial
intelligence.' We combed the literature for tortured phrases and study one
reputable journal where these concentrated en masse. Hypothesising the use of
advanced language models we ran a detector on the abstracts of recent articles
of this journal and on several control sets. The pairwise comparisons reveal a
concentration of abstracts flagged as 'synthetic' in the journal. We also
highlight irregularities in its operation, such as abrupt changes in editorial
timelines. We substantiate our call for investigation by analysing several
individual dubious articles, stressing questionable features: tortured writing
style, citation of non-existent literature, and unacknowledged image reuse.
Surprisingly, some websites offer to rewrite texts for free, generating
gobbledegook full of tortured phrases. We believe some authors used rewritten
texts to pad their manuscripts. We wish to raise the awareness on publications
containing such questionable AI-generated or rewritten texts that passed (poor)
peer review. Deception with synthetic texts threatens the integrity of the
scientific literature.
Authors
Guillaume Cabanac, Cyril Labbé, Alexander Magazinov