The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index; 2 of the 13 sources its claims cite were checked for support.
GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation
Open accessAbstract
Academic journals, archives, and repositories are seeing an increasing number of questionable research papers clearly produced using generative AI. They are often created with widely available, general-purpose AI applications, most likely ChatGPT, and mimic scientific writing. Google Scholar easily locates and lists these questionable papers alongside reputable, quality-controlled research. Our analysis of a selection of questionable GPT-fabricated scientific papers found in Google Scholar shows that many are about applied, often controversial topics susceptible to disinformation: the environment, health, and computing. The resulting enhanced potential for malicious manipulation of society's evidence base, particularly in politically divisive domains, is a growing concern.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper identifies 139 questionable, GPT-fabricated scientific papers in Google Scholar, most in non-indexed journals or working papers and many on policy-relevant topics. It argues these papers permeate scholarly infrastructure and pose risks of evidence hacking and erosion of trust in science.
Key findings
- Of 227 retrieved papers, 139 were classified as undeclared and/or fraudulent GPT use, while 88 were legitimate and/or declared false positives; around 62% did not declare GPT use. Venues included 19 indexed journals, 89 non-indexed journals, 19 student papers, and 12 working papers.
- 57% dealt with policy-relevant subjects: computing 23% (32), environment 19.5% (27), and health 14.5% (20); health and environment papers made up around 34% (47) of the sample, and 66% of those were in non-indexed journals.
- Dissemination was widespread: 20 health-related papers appeared across 20 unique domains and 46 URLs, while 27 environment-related papers appeared across 26 unique domains and 56 URLs; copies appeared on platforms including ResearchGate, ORCiD, Easychair, IEEE, Frontiers, JPTCP, and X/Twitter.
- Google Scholar presents quality-controlled and non-controlled sources on the same interface; only the query “as of my last knowledge update” had more than two search engine result pages, with questionable articles evenly distributed (min: 4, max: 9, mode: 8) and a slightly higher proportion of undeclared use on later pages. Intercoder reliability was Cohen’s kappa 0.806.
Methods
- The authors scraped Google Scholar using the Python library Scholarly for papers containing the ChatGPT-like phrases “as of my last knowledge update” and/or “I don’t have access to real-time data,” retrieving 227 papers.
- Multiple coding by all authors classified papers as suspected fraudulent/undeclared versus legitimate/declared, then categorized fraudulent papers into health, environment, computing, and others; disagreements were resolved, yielding Cohen’s kappa 0.806.
- Paper titles were programmatically searched in Google Search via googlesearch-python, manually verified, and compiled by field; descriptive statistics used pandas, and a word rain/TF-IDF visualization analyzed full texts of environment- and health-related papers.
Limitations
- Stated: the study focused on a selection of papers easily recognizable as fraudulent, used a relatively small sample as a magnifying glass, did not include detailed analysis of which article parts contained fabricated text, and did not investigate SciHub/shadow libraries or Wikipedia references.
- Stated: the authors note they likely see only the tip of the iceberg, with more sophisticated cases and fabricated images already existing.
- Evident: reliance on two telltale ChatGPT phrases likely misses GPT-fabricated papers that do not contain those phrases; Google Search from a local IP and Google Scholar scraping may limit reproducibility and coverage.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
16 Own interpretation · 12 Own result · 9 Own method · 5 Not checked yet · 3 No full text of the sources · 2 No basis given · 1 Supported · 1 Partly supported · accuracy not yet measured
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More than half (57%) of these GPT-fabricated papers concerned policy-relevant subject areas susceptible to influence operations.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This facilitated the identification of papers that likely used generative AI to produce text, resulting in 227 retrieved papers.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finding 1: 139 GPT-fabricated, questionable papers were found and listed as regular results on the Google Scholar results page.”
- Also stated in: § Findings · plain
“We found a total of 139 papers with a suspected deceptive use of ChatGPT or similar LLM applications (see Table 1).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The two common ChatGPT responses used as search terms reliably indicate GPT-fabricated papers.
- Also stated in: § Findings · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Based on that, we can say that the phrases occurred in most sections typically found in scientific publications, including the literature review, methods, conceptual and theoretical frameworks, background, motivation or societal relevance, and even discussion.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Platforms on which identified papers have appeared include ResearchGate, ORCiD, Journal of Population Therapeutics and Clinical Pharmacology (JPTCP), Easychair, Frontiers, the Institute of Electrical and Electronics Engineer (IEEE), and X/Twitter.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results of our descriptive statistical analyses showed that around 62% did not declare the use of GPTs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The authority of quality-controlled research as evidence to support legislation, policy, politics, and other forms of decision-making is undermined by the presence of undeclared GPT-fabricated content in publications professing to be scientific.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The search interface does not offer the possibility to filter the results meaningfully by material type, publication status, or form of quality control, such as limiting the search to peer-reviewed material.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It became clear that not just the text related to the telltale phrases was created by GPT, but that almost all articles in our sample of questionable articles likely contained traces of GPT-fabricated text everywhere.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The joint coding process reliably identified GPT-fabricated text.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, the position of the articles on the search engine results pages was not particularly informative, considering the relatively small number of results in combination with the recency of the publications.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 17
- It may state two separate claims as one. 21
Every source the paper cites for this claim was checked and supports it.
In the paper
“We employed multiple coding (Barbour, 2001) to classify the papers based on their content.”
What the sources say
- Supportedcites Barbour, 2001 source
The “Multiple coding” section describes cross-checking coding strategies and interpretations by independent researchers. The citation supports the method named in the sentence; classifying the papers by content is the citing authors’ description of how they used it. (LLM, conf=0.91)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This was done to single out coding discrepancies and settle coding disagreements, which in turn ensured methodological thoroughness and analytical consensus (see Barbour, 2001).”
What the sources say
- Partly supportedcites Barbour, 2001 sourceExperimental — accuracy not yet measured
The Multiple coding section says discussing coding disagreements can encourage thoroughness and refine coding frameworks. It also says the degree of agreement between researchers is not important, so it does not support analytical consensus as the outcome. The Conclusion adds that multiple coding does not, by itself, ensure rigour. (LLM, conf=0.90)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This enabled the identification of other platforms through which the papers had been spread.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Most of the identified papers exist in multiple copies and have already spread to several archives, repositories, and social media.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The 27 papers dealing with environmental issues can be found across 26 unique domains, accounting for 56 URLs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Out of these, 19 were in indexed journals, 89 were in non-indexed journals, 19 were student papers found in university databases, and 12 were working papers (mostly in preprint databases).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On those, questionable articles with undeclared use of GPTs were evenly distributed across all result pages (min: 4, max: 9, mode: 8), with the proportion of undeclared use being slightly higher on average on later search result pages.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The 20 papers concerning health-related issues are distributed across 20 unique domains, accounting for 46 URLs.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our analysis shows that questionable and potentially manipulative GPT-fabricated papers permeate the research infrastructure and are likely to become a widespread phenomenon.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“It has also been highlighted that Google Scholar, in particular, can be and has been exploited for manipulating the evidence base for politically charged issues and to fuel conspiracy narratives (Tripodi et al., 2023).”
What the sources say
- Not checked — no full text of the sourcecites Tripodi et al., 2023 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our study focused on a selection of papers that were easily recognizable as fraudulent.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This was confirmed during the joint coding, where we read and discussed all articles.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, Google Scholar lacks the transparency and adherence to standards that usually characterize citation databases.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and will have implications for any attempts to retract or remove fraudulent papers from their original publication venues.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Society's methods of evaluating sources and the foundations of media and information literacy are under threat and public trust in science is at risk of further erosion, with far-reaching consequences for society in dealing with information disorders.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“However, as the rise of the so-called anti-vaxx movement during the COVID-19 pandemic and the ongoing obstruction and denial of climate change show, retracting erroneous publications often fuels conspiracies and increases the following of these movements rather than stopping them.”
What the sources say
- Not checked — no full text of the sourcecites Dunlap & Brulle, 2020 source
- Not matchedcites Carrion, 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“First, we jointly assessed whether the paper was suspected of fraudulent use of ChatGPT (or similar) based on how the text was integrated into the papers and whether the paper was presented as original research output or the AI tool's role was acknowledged.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“There are already more sophisticated cases (Dadkhah et al., 2023) as well as cases involving fabricated images (Gu et al., 2022).”
What the sources say
- Not checked — no full text of the sourcecites Dadkhah et al., 2023 source
- Not matchedcites Gu et al., 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the mere possibility of these things happening is a significant risk in its own right that can be strategically exploited and will have ramifications for trust in and perception of science.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Google Scholar's central position in the publicly accessible scholarly communication infrastructure, as well as its lack of standards, transparency, and accountability in terms of inclusion criteria, has potentially serious implications for public trust in science.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Google Scholar lacks standards, transparency, and accountability in terms of inclusion criteria.
- Rests on an assumption: Google Scholar has a central position in the publicly accessible scholarly communication infrastructure.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The multiple coding was conducted jointly by all authors of the present article, who collaboratively coded and cross-checked each other's interpretation of the data simultaneously in a shared spreadsheet file.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“It has been shown that Google Scholar is susceptible to manipulation through citation exploits (Antkare, 2020) and by providing access to fake scientific papers (Dadkhah et al., 2017).”
What the sources say
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We manually verified each search result to filter out false positives-results that were not related to the paper-and then compiled the most prominent URLs by field.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finding 3: Google Scholar presents results from quality-controlled and non-controlled citation databases on the same interface, providing unfiltered access to GPT-fabricated questionable papers.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Papers containing the specific ChatGPT phrases likely used generative AI to produce text.
In the paper
“It is also often touted as a credible source for academic literature and even recommended in library guides, by media and information literacy initiatives, and fact checkers (Tripodi et al., 2023).”
What the sources say
- Not checked — no full text of the sourcecites Tripodi et al., 2023 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, in analyzing the content of the papers, we continued the multiple coding by classifying the fraudulent papers into four categories identified during an initial round of analysis-health, environment, computing, and others-and then determining which subjects were most affected by this issue (see Table 1).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We did not, however, investigate whether copies had spread into SciHub or other shadow libraries, or if they were referenced in Wikipedia.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“However, we have made the data available in the Harvard Dataverse repository.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The resulting enhanced potential for malicious manipulation of society's evidence base, particularly in politically divisive domains, is a growing concern.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: The number of GPT-fabricated papers in academic repositories is increasing.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both concerns are likely to be magnified in the future, increasing the risk of what we suggest calling evidence hacking-the strategic and coordinated malicious manipulation of society's evidence base.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
- Rests on an assumption: The number of GPT-fabricated papers in academic repositories is increasing.
The paper relies on this without support.
The two common ChatGPT responses used as search terms reliably indicate GPT-fabricated papers.
“To understand the occurrence of ChatGPT (co-)authored work in Google Scholar's index, we scraped it for publications, including one of two common ChatGPT responses (see Appendix A) that we encountered on social media and in media reports (DeGeurin, 2024).”
The paper relies on this without support.
The joint coding process reliably identified GPT-fabricated text.
“This was confirmed during the joint coding, where we read and discussed all articles.”
The paper relies on this without support.
The number of GPT-fabricated papers in academic repositories is increasing.
“Academic journals, archives, and repositories are seeing an increasing number of questionable research papers clearly produced using generative AI.”
The paper relies on this without support.
Papers containing the specific ChatGPT phrases likely used generative AI to produce text.
“This facilitated the identification of papers that likely used generative AI to produce text”
The paper relies on this without support.
Google Scholar lacks standards, transparency, and accountability in terms of inclusion criteria.
“as well as its lack of standards, transparency, and accountability in terms of inclusion criteria”
The paper relies on this without support.
Google Scholar has a central position in the publicly accessible scholarly communication infrastructure.
“Google Scholar's central position in the publicly accessible scholarly communication infrastructure”
Citations
Cites
Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?
Barbour, R. S. (2001) · BMJ · doi:10.1136/bmj.322.7294.1115 · Cited by 2,613
Gusenbauer, M., & Haddaway, N. R. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1378 · Cited by 2,379
ChatGPT is fun, but not an author
Thorp, H. H. (2023) · Science · doi:10.1126/science.adg7879 · Cited by 1,437
Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021) · Scientometrics · doi:10.1007/s11192-020-03690-4 · Cited by 1,018
Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24750 · Cited by 919
ChatGPT is shaping the future of medical writing but still requires human judgment
Kitamura, F. C. (2023) · Radiology · doi:10.1148/radiol.230171 · Cited by 317
The role of ChatGPT in scientific communication: writing better scientific review articles.
(2023) · PubMed Central · Cited by 289
Cabanac, G., Labbé, C., & Magazinov, A. (2021) · arXiv · doi:10.48550/arxiv.2107.06751 · Cited by 224 · cited 1×
“The work of ongoing whistleblowing initiatives 5 , recent media reports of journal closures (Subbaraman, 2024), or GPT-related changes in word use and writing style (Cabanac et al., 2021; Stokel-Walker, 2024) suggest that we only see the tip of the iceberg.”
BackgroundWord vectors, reuse, and replicability: Towards a community repository of large-text resources
Fares, M., Kutuzov, A., Oepen, S., & Velldal, E. (2017) · DSpace repository (University of Tartu) · Cited by 138
AI-enabled image fraud in scientific publications
Gu, J., Wang, X., Li, C., Zhao, J., Fu, W., Liang, G., & Qiu, J. (2022) · Patterns · doi:10.1016/j.patter.2022.100511 · Cited by 103
Prevalence of nonsensical algorithmically generated papers in the scientific literature
Cabanac, G., & Labbé, C. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24495 · Cited by 102
Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown
Simon, F. M., Altay, S., & Mercier, H. (2023) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-127 · Cited by 85 · cited 1×
“…intermediaries between science, but also certain types of misinformation, and the public, namely Google Scholar, also in response to the legitimate concerns that the discussion of generative AI and misinformation needs to be more nuanced and empirically substantiated (Simon et al., 2023).”
Background“You need to do your research”: Vaccines, contestable science, and maternal epistemology
Carrion, M. L. (2017) · Public Understanding of Science · doi:10.1177/0963662517728024 · Cited by 71
Exploring the Opportunities and Challenges of ChatGPT in Academic Writing: a Roundtable Discussion
Bom, H.-S. H. (2023) · Nuclear Medicine and Molecular Imaging · doi:10.1007/s13139-023-00809-2 · Cited by 50
Sources and amplifiers of climate change denial
Dunlap, R. E., & Brulle, R. J. (2020) · Research Handbook on Communicating Climate Change · doi:10.4337/9781789900408.00013 · Cited by 49
How journals are fighting back against a wave of questionable images
Jones, N. (2024) · Nature · doi:10.1038/d41586-024-00372-6 · Cited by 39
From word clouds to Word Rain: Revisiting the classic word cloud to visualize climate change texts
Skeppstedt, M., Ahltorp, M., Kucher, K., & Lindström, M. (2024) · Information Visualization · doi:10.1177/14738716241236188 · Cited by 35
‘Do your own research': affordance activation and disinformation spread
Tripodi, F. B., Garcia, L. C., & Marwick, A. E. (2024) · Information, Communication & Society · doi:10.1080/1369118x.2023.2245869 · Cited by 31
Detection of fake papers in the era of artificial intelligence
Dadkhah, M., Oermann, M. H., Hegedüs, M., Raman, R., & Dávid, L. D. (2023) · Diagnosis · doi:10.1515/dx-2023-0090 · Cited by 24
Haider, J., & Åström, F. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23669 · Cited by 23
Self-citationQuestionable papers in citation databases as an issue for literature review
Dadkhah, M., Lagzian, M., & Borchardt, G. (2017) · Journal of Cell Communication and Signaling · doi:10.1007/s12079-016-0370-6 · Cited by 23
Chinn, S., & Hasell, A. (2023) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-117 · Cited by 22
Back to the past: on the shoulders of an academic search engine giant
Martín-Martín, A., Orduna-Malea, E., Ayllón, J. M., & Delgado López-Cózar, E. (2016) · Scientometrics · doi:10.1007/s11192-016-1917-2 · Cited by 18
Correcting the scientific record on abortion and mental health outcomes
Littell, J. H., Abel, K. M., Biggs, M. A., Blum, R. W., Foster, D. G., Haddad, L. B., Major, B., Munk-Olsen, T., Polis, C. B., Robinson, G. E., et al. (2024) · BMJ · doi:10.1136/bmj-2023-076518 · Cited by 14
Munro, R. (2011) · Nick Hern Books · doi:10.5040/9781784602314 · Cited by 13
Cholewiak, S., Ipeirotis, P. G., Silva, V., & Kannawadi, A. (2021) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.5764801 · Cited by 13
Ike Antkare, His Publications, and Those of His Disciples
Antkare, I. (2020) · Gaming the Metrics · doi:10.7551/mitpress/11087.003.0018 · Cited by 9
The pandas development team (2024) · Zenodo · doi:10.5281/zenodo.10957263
Research works printed but not linked: 1
- Google Scholar Help. (n.d.). Inclusion guidelines for webmasters. https://scholar.google.com/intl/en/scholar/inclusion.html
Cited by
Sorted by citations
Malik, M. A., & Amjad, A. I. (2025) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2025.8.1.9 · Cited by 32
Hui, L., & Wu, X. (2025) · AI and Ethics · doi:10.1007/s43681-025-00827-0 · Cited by 19
Giray, L., Sevnarayan, K., Maphoto, K. B., & Wider, W. (2026) · Internet Reference Services Quarterly · doi:10.1080/10875301.2026.2637526 · Cited by 11
Comas‐Forgas, R., Koulouris, A., & Kouis, D. (2025) · Learned Publishing · doi:10.1002/leap.1647 · Cited by 9
Macko, D., Ramakrishnan, A. A., Lucas, J. S., Móro, R., Srba, I., Uchendu, A., & Lee, D. (2026) · Computer · doi:10.1109/mc.2025.3592765 · Cited by 8
Metadata
- Authors
- Haider, J., Söderström, K. R., Ekström, B., & Rödl, M.
- Affiliations
- University of Borås (SE)
- Lund University (SE)
- Swedish University of Agricultural Sciences (SE)
- Venue
- Harvard Kennedy School Misinformation Review
- Published
- 2024-09-03
- ISSN
- 2766-1652 (ISSN-L)
- OpenAlex
- W4402119162
- Semantic Scholar
- 6b377283028611f49f9debde0ada8eef1e87a504
- Cited by
- 70
- References
- 27
- Citations by year
- 2026: 45 · 2025: 20 · 2024: 5
- FWCI
- 31.76
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Harvard Kennedy School Misinformation Review (publishedVersion, cc-by)
- Epsilon Open Archive (Sveriges lantbruksuniversitet biblioteket (Swedish University of Agricultural Sciences)) (acceptedVersion, cc-by)
- Diva portal (Dalarna University Library) (submittedVersion, other-oa)
- Lund University Publications (Lund University) (submittedVersion)
- Funders
- Stiftelsen för Miljöstrategisk Forskning
- Marcus och Amalia Wallenbergs minnesfond (2020.0004)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- generative AI, ChatGPT, Google Scholar, disinformation, evidence manipulation, environmental topics, health topics, AI-generated text detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
