CoChat
Sign Up
PreprintCited by 18Academic integrity and plagiarismAcademic Writing and PublishingMisinformation and Its Impacts

Compound Deception in Elite Peer Review: A Failure Mode Taxonomy of 100 Fabricated Citations at NeurIPS 2025

Open access
Ansari, Samar·2026·arXiv
DOI 10.48550/arxiv.2602.05930arXiv 2602.05930
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) are increasingly used in academic writing workflows, yet they frequently hallucinate by generating citations to sources that do not exist. This study analyzes 100 AI-generated hallucinated citations that appeared in papers accepted by the 2025 Conference on Neural Information Processing Systems (NeurIPS), one of the world's most prestigious AI conferences. Despite review by 3-5 expert researchers per paper, these fabricated citations evaded detection, appearing in 53 published papers (approx. 1% of all accepted papers). We develop a five-category taxonomy that classifies hallucinations by their failure mode: Total Fabrication (66%), Partial Attribute Corruption (27%), Identifier Hijacking (4%), Placeholder Hallucination (2%), and Semantic Hallucination (1%). Our analysis reveals a critical finding: every hallucination (100%) exhibited compound failure modes. The distribution of secondary characteristics was dominated by Semantic Hallucination (63%) and Identifier Hijacking (29%), which often appeared alongside Total Fabrication to create a veneer of plausibility and false verifiability. These compound structures exploit multiple verification heuristics simultaneously, explaining why peer review fails to detect them. The distribution exhibits a bimodal pattern: 92% of contaminated papers contain 1-2 hallucinations (minimal AI use) while 8% contain 4-13 hallucinations (heavy reliance). These findings demonstrate that current peer review processes do not include effective citation verification and that the problem extends beyond NeurIPS to other major conferences, government reports, and professional consulting. We propose mandatory automated citation verification at submission as an implementable solution to prevent fabricated citations from becoming normalized in scientific literature.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper analyzes 100 AI-generated hallucinated citations found in 53 accepted NeurIPS 2025 papers, despite each paper having been reviewed by 3–5 experts. It proposes a five-category failure-mode taxonomy and reports that all 100 hallucinations exhibited compound failure modes, which helps explain why peer review missed them.

Key findings

  • Primary failure modes among the 100 citations were Total Fabrication (66%), Partial Attribute Corruption (27%), Identifier Hijacking (4%), Placeholder Hallucination (2%), and Semantic Hallucination (1%).
  • Every hallucination (100%) exhibited compound failure modes; secondary characteristics were dominated by Semantic Hallucination (63%) and Identifier Hijacking (29%).
  • The 100 hallucinations spanned 53 papers (≈1% of accepted papers), with a mean of 1.89 hallucinations per contaminated paper, median 2, and range 1–13; 49 of 53 papers (92%) contained 1–2 hallucinations, while 4 outlier papers contained 4–13.
  • Among the 66 Total Fabrications, 50 cases (76%) used Semantic Hallucination as a secondary mode to make fabricated titles appear domain-appropriate.

Methods

  • Data came from GPTZero’s automated Hallucination Check scan of 4,841 of the 5,290 accepted NeurIPS 2025 papers, followed by human expert verification at GPTZero; the author used the complete public dataset of 100 verified hallucinations across 53 papers.
  • The author manually classified each citation into a five-category taxonomy—Total Fabrication, Partial Attribute Corruption, Identifier Hijacking, Semantic Hallucination, and Placeholder Hallucination—assigning both a primary and secondary failure category.
  • Coding was performed over three days using a structured spreadsheet documenting citation text, primary/secondary codes, classification reasoning, and evidence from GPTZero notes.

Limitations

  • Stated: the analysis covers only citations flagged by GPTZero’s automated tool, so citations that appear correct but subtly misrepresent source content would not be detected; author intent cannot be determined; taxonomy boundaries—especially PAC vs. TF—can be ambiguous; and the data come from a single conference and discipline, so the 66% TF / 27% PAC distribution may not generalize.
  • Evident: classification was performed by a single manual coder without reported inter-rater reliability, and the study relies on GPTZero’s prior verification and public report rather than independent re-verification of all 100 citations.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2602.05930

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

7 of 35 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § 1 Introduction · p. 2

    Citations are particularly vulnerable to hallucination because they involve precise metadata that language models may blend, paraphrase, or fabricate while maintaining surface-level plausibility.

    “Citations are particularly vulnerable to hallucination because they involve precise metadata (author names, titles, venues, publication dates, DOIs) that language models may blend, paraphrase, or fabricate while maintaining surface-level plausibility [7].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [7] [7] Hanan Alkaissi and Samy I McFarlane. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus , 15(2), 2023. - source

      The 'PMID: PubMed ID' section reports that ChatGPT returned the same references with different years and PMID numbers belonging to other papers; the Figure 2 caption likewise says its PMID numbers corresponded to different papers. This supports plausible-looking errors in citation metadata, but the paper does not establish that citations are particularly vulnerable relative to other content or document the claimed blending or fabrication of author names, titles, venues, or DOIs. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Premiseown data
    § ABSTRACT · p. 1

    LLMs frequently hallucinate by generating citations to sources that do not exist.

    “yet they frequently hallucinate by generating citations to sources that do not exist.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [5] [5] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys , 55(12):138, 2023. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [6] [6] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems , 43(2):1-55, 2025. - source

      Section 2.3 (p. 6) describes LLMs fabricating unverifiable claims, including claims about things that do not exist. It does not say that they frequently generate citations to nonexistent sources, so the specific example and its frequency are unsupported by the passages found. (LLM, conf=0.80)

      Experimental — accuracy not yet measured
    • cites [7] [7] Hanan Alkaissi and Samy I McFarlane. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus , 15(2), 2023. - source

      The Figure 2 caption and the “PMID: PubMed ID” section report that ChatGPT supplied references whose PMID numbers belonged to different papers, including when asked for more recent references. That supports the point that it can generate unreliable citations, but the paper does not establish that the cited sources did not exist or that LLMs do this frequently. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Findingown data
    § ABSTRACT · p. 1

    These fabricated citations evaded detection despite review by 3-5 expert researchers per paper.

    “Despite review by 3-5 expert researchers per paper, these fabricated citations evaded detection, appearing in 53 published papers ( ≈ 1% of all accepted papers).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] [1] GPTZero. GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers. https://gptzero.me/ news/neurips/ , 2026. Accessed: 2026-02-05. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingown data
    § ABSTRACT · p. 1

    The distribution of secondary characteristics was dominated by Semantic Hallucination (63%) and Identifier Hijacking (29%).

    “The distribution of secondary characteristics was dominated by Semantic Hallucination (63%) and Identifier Hijacking (29%)”
    Quote in the paperSubstantial claim
  • Findingown data
    § ABSTRACT · p. 1

    The distribution of hallucinations across contaminated papers is bimodal: 92% of contaminated papers contain 1-2 hallucinations, while 8% contain 4-13 hallucinations.

    “The distribution exhibits a bimodal pattern: 92% of contaminated papers contain 1-2 hallucinations (minimal AI use) while 8% contain 4-13 hallucinations (heavy reliance).”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 24
  • May combine two claims. 47

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1.1 Current Understanding and Research Gaps · p. 2

    Studies of citation behavior have identified problems with citation accuracy and misconduct.

    “Studies of citation behavior have identified problems with citation accuracy [12] and misconduct [13].”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [12] [12] Anastasia Rivkin. Manuscript referencing errors and their impact on shaping current evidence. American Journal of Pharmaceutical Education , 84(7), 2020. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [13] [13] Ferric C Fang, R Grant Steen, and Arturo Casadevall. Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences , 109(42):17028-17033, 2012. - source

      The paper reports that misconduct accounted for most retractions among the PubMed-indexed articles it studied. Its Methods describe a study of retraction causes, not citation behavior; its separate discussion of citations to retracted articles does not identify misconduct through citation behavior. Thus it supports the misconduct concern attached to [13], but not the sentence’s characterization of this as a study of citation behavior. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1.1 Current Understanding and Research Gaps · p. 2

    Previous research has documented LLM hallucination in medical information, legal reasoning, and general factual queries.

    “Previous research has documented LLM hallucination in various contexts, including medical information [8], legal reasoning [9], and general factual queries [10].”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [8] [8] Peter Lee, Sebastien Bubeck, and Joseph Petro. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. New England Journal of Medicine , 388(13):1233-1239, 2023. - source
    • cites [9] [9] Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis , 16(1):64-93, 2024. - source
    • cites [10] [10] Vipula Rawte, Amit Sheth, and Amitava Das. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922 , 2023. - source

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 Introduction · p. 2

    LLMs are known to hallucinate, generating fluent, confident-sounding text that is factually incorrect.

    “LLMs are known to hallucinate 2 , 3 , generating fluent, confident-sounding text that is factually incorrect [5, 6].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [5] [5] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys , 55(12):138, 2023. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [6] [6] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems , 43(2):1-55, 2025. - source

      Section 7.2 (p. 34) says LLMs can confidently produce falsehoods, and §2.3 (p. 6) describes outputs that contradict real-world facts. These passages support the confident-sounding, factually incorrect part of the claim, but our copy does not establish that such text is fluent. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data
    § ABSTRACT · p. 1

    The 100 hallucinations appeared in 53 published papers, approximately 1% of all accepted papers.

    “Despite review by 3-5 expert researchers per paper, these fabricated citations evaded detection, appearing in 53 published papers ( ≈ 1% of all accepted papers).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] [1] GPTZero. GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers. https://gptzero.me/ news/neurips/ , 2026. Accessed: 2026-02-05. -

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 2.1 Data Source · p. 3

    GPTZero's analysis identified 100 hallucinated citations spanning 53 distinct papers, representing 1% of all accepted papers.

    “GPTZero's analysis identified 100 hallucinated citations spanning 53 distinct papers 5 (representing 1% of all accepted papers).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] [1] GPTZero. GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers. https://gptzero.me/ news/neurips/ , 2026. Accessed: 2026-02-05. -

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 1 Introduction · p. 1

    The fabricated citations are not typographical errors or formatting mistakes; they are citations to sources that do not exist, with invented authors and/or fabricated titles, and/or false publication details.

    “These are not typographical errors or formatting mistakes; they are citations to sources that do not exist, with invented authors and/or fabricated titles, and/or false publication details.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] [1] GPTZero. GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers. https://gptzero.me/ news/neurips/ , 2026. Accessed: 2026-02-05. -

      Not checked in the free run.

  • May combine two claims.

    Findingown data
    § 4.3 The Dominance of Total Fabrication: Implications for Detection · p. 9

    Two-thirds of hallucinations involve wholesale invention with zero overlap with reality, contradicting the hypothesis that LLMs retrieve and corrupt real sources.

    “Our data contradicts this: two-thirds of hallucinations involve wholesale invention with zero overlap with reality.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 4.1 The Systematic Failure of Elite Peer Review · p. 9

    Multiple expert reviewers examined each paper; two citations contained obvious errors ('Firstname Lastname,' 'URL or arXiv ID to be updated'); none were caught during review.

    “The observable facts are these: (1) Multiple expert reviewers examined each paper; (2) Two citations contained obvious errors ('Firstname Lastname,' 'URL or arXiv ID to be updated'); (3) None were caught during review.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Distribution of Hallucination Counts Across 53 Contaminated Papers (Median = 2, Mean = 1.89, Range: 1-13) · p. 8

    The dominant pattern involved Total Fabrications (66% primary) that also employed Semantic Hallucination characteristics (63% secondary across all citations).

    “The dominant pattern involved Total Fabrications (66% primary) that also employed Semantic Hallucination characteristics (63% secondary across all citations).”
    • Also stated in: § 4.2 Why Compound Failures Succeed: The Multi-Layered Deception Strategy · p. 9 · plain
      “with the most common pattern being Total Fabrication (66% primary) layered with Semantic Hallucination (63% secondary).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Distribution of Hallucination Counts Across 53 Contaminated Papers (Median = 2, Mean = 1.89, Range: 1-13) · p. 8

    Identifier Hijacking appeared as a secondary characteristic in 29% of cases, typically combined with Total Fabrication or Partial Attribute Corruption as the primary failure mode.

    “Identifier Hijacking appeared as a secondary characteristic in 29% of cases, typically combined with Total Fabrication or Partial Attribute Corruption as the primary failure mode.”
    • Also stated in: § 4.2 Why Compound Failures Succeed: The Multi-Layered Deception Strategy · p. 9 · boosted
      “The 29% secondary Identifier Hijacking rate across all citations shows that many fabrications include valid arXiv IDs or DOIs that link to real papers.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 3.7 Distribution Across Papers · p. 7

    The 100 hallucinations span 53 distinct NeurIPS papers, with a mean of 1.89 hallucinations per contaminated paper (range: 1-13, median: 2).

    “The 100 hallucinations span 53 distinct NeurIPS papers, with a mean of 1.89 hallucinations per contaminated paper (range: 1-13, median: 2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation
    § 4.1 The Systematic Failure of Elite Peer Review · p. 9

    Each paper accepted by NeurIPS 2025 underwent review by 3-5 expert researchers, yet 53 papers containing 100 fabricated citations were published.

    “Each paper accepted by NeurIPS 2025 underwent review by 3-5 expert researchers, yet 53 papers containing 100 fabricated citations were published.”
    • Also stated in: § 5 Conclusion · p. 13 · plain
      “This study documents 100 AI-generated hallucinated citations that appeared in papers accepted by NeurIPS 2025”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] [1] GPTZero. GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers. https://gptzero.me/ news/neurips/ , 2026. Accessed: 2026-02-05. -

      Not checked in the free run.

  • May combine two claims.

    Findingown data
    § 3.7 Distribution Across Papers · p. 7

    The paper with the highest number of hallucinations (13 citations) demonstrates that citation fabrication can be extensive rather than isolated, suggesting reliance on AI tools throughout the writing process rather than occasional consultation.

    “The paper with the highest number of hallucinations (13 citations): 'Efficient semantic uncertainty quantification in language models via diversity-steered sampling' demonstrates that citation fabrication can be extensive rather than isolated, suggesting reliance on AI tools throughout the writing process rather than occasional consultation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given
    § Distribution of 100 Hallucinated Citations by Failure Mode · p. 5

    A 2022 version of Beltran et al. (2024, NFL-BA: Improving endoscopic SLAM with near-field light bundle adjustment, arXiv:2412.13176v1) contained this exact fabricated citation in its references, though it was corrected in later versions.

    “A 2022 version of another paper (Beltran et al., 2024, NFL-BA: Improving endoscopic SLAM with near-field light bundle adjustment, arXiv:2412.13176v1) contained this exact fabricated citation in its references, though it was corrected in later versions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3.7 Distribution Across Papers · p. 7

    The distribution exhibits a clear bimodal pattern: 49 of 53 papers (92%) contain only 1-2 hallucinations, while 4 outlier papers contain 4-13 hallucinations each.

    “The distribution exhibits a clear bimodal pattern: 49 of 53 papers (92%) contain only 1-2 hallucinations, while 4 outlier papers contain 4-13 hallucinations each.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 3.8 Compound Failure Modes: A Critical Finding · p. 7

    Among the 100 fabricated citations, secondary failure characteristics were: Semantic Hallucination 63%, Identifier Hijacking 29%, Placeholder Hallucination 4%, Partial Attribute Corruption 3%, Total Fabrication 1%.

    “| Semantic Hallucination (SH) | 63 | 63.0% |”
    • Also stated in: § 3.8 Compound Failure Modes: A Critical Finding · p. 7 · plain
      “| Identifier Hijacking (IH) | 29 | 29.0% |”
    • Also stated in: § 3.8 Compound Failure Modes: A Critical Finding · p. 7 · plain
      “| Placeholder Hallucination (PH) | 4 | 4.0% |”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation
    § 2.3 Coding Procedure · p. 4

    The coding approach is consistent with established practices in research integrity content analysis where categories are based on observable features rather than subjective interpretation.

    “This approach is consistent with established practices in research integrity content analysis where categories are based on observable features rather than subjective interpretation [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] [14] Rosaline S Barbour. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog? BMJ , 322(7294):1115-1117, 2001.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction · p. 2

    Fabricated citations have appeared in U.S. government reports requiring corrections.

    “Fabricated citations have also appeared in U.S. government reports requiring corrections [3]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [3] [3] Lauren Weber and Caitlin Gilbert. White house MAHA report may have garbled science by using AI, experts say. https://www.washingtonpost.com/health/2025/05/29/maha-rfk-jr-ai-garble/ , May 2025. Accessed: 2026-02-05. -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation
    § 1.1 Current Understanding and Research Gaps · p. 2

    The phenomenon extends beyond isolated hallucinations to broader patterns of synthetic content degradation, including AI Slop.

    “The phenomenon extends beyond isolated hallucinations to encompass broader patterns of synthetic content degradation, including what has been termed AI Slop , low-quality, derivative outputs that contaminate digital ecosystems and degrade informational value [11].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] [11] Samar Ansari. AI slop and data pollution in the age of generative AI: Strategic risks, economic consequences, and governance pathways for business, management, and the creative industries. Available at SSRN, DOI: 10.2139/ssrn.5649410 , 2025. - source

  • May combine two claims.

    Interpretationreasoning
    § 1 Introduction · p. 2

    The common factor across these cases is the use of LLMs in writing workflows combined with inadequate verification of AI-generated content.

    “The common factor across these cases is the use of large language models (LLMs) in writing workflows combined with inadequate verification of AI-generated content.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1.1 Current Understanding and Research Gaps · p. 2

    The intersection of AI-generated fabricated citations in peer-reviewed literature has received limited systematic analysis.

    “However, the intersection of these issues: AIgenerated fabricated citations in peer-reviewed literature, has received limited systematic analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    NeurIPS 2025 had 5,290 accepted papers out of 21,575 submissions, a 24.52% acceptance rate.

    “was conducted with 5,290 accepted papers (reflecting a 24.52% acceptance rate out of 21,575 submissions 1 ).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] [1] GPTZero. GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers. https://gptzero.me/ news/neurips/ , 2026. Accessed: 2026-02-05. -

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § ABSTRACT · p. 1

    These compound structures exploit multiple verification heuristics simultaneously, explaining why peer review fails to detect them.

    “These compound structures exploit multiple verification heuristics simultaneously, explaining why peer review fails to detect them.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 1 Introduction · p. 2

    If researchers who specialize in LLMs cannot reliably detect AI-generated fabrications in their own review process, the problem is systematic, not isolated.

    “If researchers who specialize in LLMs cannot reliably detect AI-generated fabrications in their own review process, the problem is systematic, not isolated.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction · p. 2

    Fabricated citations have appeared in professional consulting outputs requiring $98,000 (AUD) refunds.

    “and professional consulting outputs requiring $98,000 (AUD) refunds [4].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] [4] GPTZero. Deloitte's citation situation & GPTZero's citation solution. https://gptzero.me/news/ deloitte-australia-citation-check/ , 2025. Accessed: 2026-01-26. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisecitation
    § 1 Introduction · p. 2

    Analysis of ICLR 2026 submissions found over 50 hallucinations in papers under review, some rated 8/10 by reviewers.

    “Analysis of ICLR 2026 submissions found over 50 hallucinations in papers under review, some rated 8/10 by reviewers [2].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2] [2] GPTZero. GPTZero finds over 50 new hallucinations in ICLR 2026 submissions. https://gptzero.me/ news/iclr-2026/ , 2025. Accessed: 2026-02-05. -

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data
    § 5 Conclusion · p. 13

    Because 100% of hallucinations in our dataset exhibited compound failure modes, simple link-checking is insufficient.

    “However, because 100% of hallucinations in our dataset exhibited compound failure modes, simple link-checking is insufficient.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.5 Beyond NeurIPS: A Systemic Problem · p. 11

    The problem is not isolated to low-quality venues or negligent authors; it appears at the most prestigious conferences in AI, produced by authors at elite institutions (NYU, Cambridge, MIT, Genentech, Google).

    “The problem is not isolated to low-quality venues or negligent authors. It appears at the most prestigious conferences in AI, produced by authors at elite institutions (NYU, Cambridge, MIT, Genentech, Google).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § Distribution of 100 Hallucinated Citations by Failure Mode · p. 5

    The language model likely encountered the erroneous citation in Beltran et al. (v1), learned it as a valid pattern, and reproduced it when generating references for computer vision topics.

    “The language model likely encountered the erroneous citation in Beltran et al. (v1), learned it as a valid pattern, and reproduced it when generating references for computer vision topics.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 4.7 Limitations · p. 12

    Our analysis analyzed only citations flagged by GPTZero's automated tool; citations that appear correct but subtly misrepresent source content would not be detected.

    “First, we analyzed only citations flagged by GPTZero's automated tool. Citations that appear correct but subtly misrepresent source content would not be detected.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § Compound Failure Structure: Primary vs Secondary Characteristics (All 100 Citations Exhibit Both Primary and Secondary Failure Modes) · p. 10

    Single-attribute checking fails, and comprehensive verification requires checking multiple attributes and confirming they all point to the same real source.

    “Compound failures (100% rate) mean single-attribute checking fails.”
    • Also stated in: § Compound Failure Structure: Primary vs Secondary Characteristics (All 100 Citations Exhibit Both Primary and Secondary Failure Modes) · p. 10 · plain
      “Comprehensive verification requires checking multiple attributes and confirming they all point to the same real source.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.4 Addressing the 'Harmless Error' Counterargument · p. 11

    The 100% compound failure rate makes the problem worse because these are multi-layered fabrications that actively mislead readers by appearing legitimate.

    “The 100% compound failure rate makes this problem worse. These are not simple mistakes, they are multi-layered fabrications that actively mislead readers by appearing legitimate through semantic plausibility, familiar author names, or working identifiers that point to different papers.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 4.5 Beyond NeurIPS: A Systemic Problem · p. 11

    GPTZero's analysis of ICLR 2026 submissions identified over 50 additional hallucinations in papers under review, with some papers receiving average ratings of 8/10 despite containing fabricated citations.

    “GPTZero's analysis of ICLR 2026 submissions identified over 50 additional hallucinations in papers under review, with some papers receiving average ratings of 8/10 despite containing fabricated citations [2].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [2] [2] GPTZero. GPTZero finds over 50 new hallucinations in ICLR 2026 submissions. https://gptzero.me/ news/iclr-2026/ , 2025. Accessed: 2026-02-05. -

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 4.2 Why Compound Failures Succeed: The Multi-Layered Deception Strategy · p. 9

    Partial Attribute Corruption (27% primary) exploits name recognition by attributing real scholars to papers they didn't write, or citing real papers with corrupted metadata.

    “Partial Attribute Corruption (27% primary) exploits name recognition by attributing real scholars to papers they didn't write, or citing real papers with corrupted metadata.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning
    § 4.7 Limitations · p. 12

    Our taxonomy classifies hallucinations by their most prominent failure characteristic as the primary code, with secondary characteristics captured separately; boundaries between categories (particularly PAC vs. TF) can be ambiguous when citations blend multiple real and fake elements.

    “Third, our taxonomy classifies hallucinations by their most prominent failure characteristic as the primary code, with secondary characteristics captured separately. The boundaries between categories (particularly PAC vs. TF) can be ambiguous when citations blend multiple real and fake elements.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.5 Beyond NeurIPS: A Systemic Problem · p. 11

    These cases span academic conferences, government agencies, and professional consulting firms, all contexts with ostensibly rigorous quality control.

    “These cases span academic conferences, government agencies, and professional consulting firms, all contexts with ostensibly rigorous quality control.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § 4.7 Limitations · p. 12

    We cannot determine author intent, whether hallucinations resulted from deliberate fraud, negligent use of AI tools, or honest mistakes.

    “Second, we cannot determine author intent , whether hallucinations resulted from deliberate fraud, negligent use of AI tools, or honest mistakes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning
    § 4.1 The Systematic Failure of Elite Peer Review · p. 9

    Under these conditions, detailed verification of every citation becomes practically impossible.

    “Under these conditions, detailed verification of every citation becomes practically impossible.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: NeurIPS 2025 received 21,575 submissions.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation
    § 4.1 The Systematic Failure of Elite Peer Review · p. 9

    This submission tsunami reflects broader trends in AI-driven content proliferation, where automation enables mass production of superficially competent but substantively hollow outputs across academic, corporate, and creative domains.

    “This submission tsunami reflects broader trends in AI-driven content proliferation, where automation enables mass production of superficially competent but substantively hollow outputs across academic, corporate, and creative domains [11].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] [11] Samar Ansari. AI slop and data pollution in the age of generative AI: Strategic risks, economic consequences, and governance pathways for business, management, and the creative industries. Available at SSRN, DOI: 10.2139/ssrn.5649410 , 2025. - source

  • May combine two claims.

    Interpretationown data
    § Distribution of 100 Hallucinated Citations by Failure Mode · p. 5

    The 66% prevalence of Total Fabrication indicates that AI language models default to wholesale invention when generating citations, creating complete fictional sources rather than attempting to corrupt or reference real scholarly work.

    “The 66% prevalence of Total Fabrication indicates that AI language models default to wholesale invention when generating citations, creating complete fictional sources rather than attempting to corrupt or reference real scholarly work.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 4.7 Limitations · p. 12

    We cannot claim that our 66% TF / 27% PAC distribution generalizes to all domains or all AI writing tools.

    “we cannot claim that our 66% TF / 27% PAC distribution generalizes to all domains or all AI writing tools.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning
    § 4.4 Addressing the 'Harmless Error' Counterargument · p. 11

    If citation linkages are fabricated, the entire knowledge graph becomes corrupted, and future researchers may waste time or propagate hallucinations.

    “If those linkages are fabricated, the entire knowledge graph becomes corrupted. Future researchers who trust these citations will waste time searching for non-existent sources or will propagate the hallucinations in their own work.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning
    § 4.4 Addressing the 'Harmless Error' Counterargument · p. 11

    A hallucinated citation is fundamentally fraudulent, whether or not the fraud was intentional, unlike a typographical error in a correctly cited source.

    “A hallucination fabricates 'Smith, J. et al. (2024). Paper that does not exist' → the reference is fundamentally fraudulent, whether or not the fraud was intentional.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 5 Conclusion · p. 13

    The common factor is the use of AI writing tools without adequate verification infrastructure to detect generated fabrications.

    “The common factor is the use of AI writing tools without adequate verification infrastructure to detect generated fabrications.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § Distribution of Hallucination Counts Across 53 Contaminated Papers (Median = 2, Mean = 1.89, Range: 1-13) · p. 8

    The 76% of Total Fabrications that layer semantic plausibility on top of wholesale invention demonstrates structured fabrication that anticipates superficial human verification rather than random noise.

    “The fact that 76% of Total Fabrications specifically layer semantic plausibility on top of wholesale invention demonstrates this is not random noise but structured fabrication that anticipates how humans verify citations superficially rather than comprehensively.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 4.8.1 On Contamination Inheritance · p. 12

    Our five-category taxonomy (TF, PAC, IH, PH, SH) was designed to classify hallucinations based on observable output characteristics rather than generative mechanisms.

    “Our five-category taxonomy (TF, PAC, IH, PH, SH) was designed to classify hallucinations based on observable output characteristics rather than generative mechanisms.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 3.7 Distribution Across Papers · p. 7

    This suggests two distinct author behaviors: cautious consultation of AI tools for 'polishing' or 'gap-filling' citations (the majority) versus systematic reliance on AI-generated citations throughout the writing process (a small minority).

    “This suggests two distinct author behaviors: cautious consultation of AI tools for 'polishing' or 'gap-filling' citations (the majority) versus systematic reliance on AI-generated citations throughout the writing process (a small minority).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 3.1 Prevalence by Failure Mode · p. 4

    Partial Attribute Corruption accounts for over one-quarter of cases (27%), suggesting a secondary strategy of blending real and fabricated elements.

    “Partial Attribute Corruption accounts for over one-quarter of cases (27%), suggesting a secondary strategy of blending real and fabricated elements.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.1 The Systematic Failure of Elite Peer Review · p. 9

    The failure is systematic, not isolated: hallucinated citations appeared across multiple institutions, research areas, and paper types, indicating that current peer review processes do not include effective citation verification.

    “The failure is systematic, not isolated: hallucinated citations appeared across multiple institutions, research areas, and paper types, indicating that current peer review processes do not include effective citation verification.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § 4.6 Solutions: Mandatory Automated Verification · p. 11

    The technical barriers to automated citation verification are minimal, as these tools are already deployed in research integrity investigations.

    “The technical barriers are minimal, these tools are already deployed in research integrity investigations.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Automated citation verification tools exist and are already used in research integrity investigations.

  • May combine two claims.

    Interpretationown data
    § 5 Conclusion · p. 13

    AI-generated hallucinations are structured fabrications that anticipate human verification behavior, not random errors.

    “AI-generated hallucinations are not random errors but structured fabrications that anticipate human verification behavior.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.3 The Dominance of Total Fabrication: Implications for Detection · p. 9

    LLMs generate citations through the same next-token prediction mechanism that produces all text, rather than through retrieval and modification of real bibliographic data.

    “This suggests that LLMs generate citations through the same next-token prediction mechanism that produces all text, rather than through retrieval and modification of real bibliographic data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data
    § 5 Conclusion · p. 13

    Verification tools must cross-reference authors, titles, venues, dates, and identifiers simultaneously to be effective.

    “Verification tools must cross-reference authors, titles, venues, dates, and identifiers simultaneously to be effective.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § Compound Failure Structure: Primary vs Secondary Characteristics (All 100 Citations Exhibit Both Primary and Secondary Failure Modes) · p. 10

    Automated verification tools that simply check 'does this exact citation exist?' will catch the majority of cases because 66% of hallucinations are Total Fabrications.

    “Because 66% of hallucinations are Total Fabrications, automated verification tools that simply check 'does this exact citation exist?' will catch the majority of cases.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationown data
    § 4.6 Solutions: Mandatory Automated Verification · p. 11

    The compound failure structure documented suggests that verification tools must check multiple attributes simultaneously and confirm they all reference the same source.

    “The compound failure structure we documented suggests that verification tools must check multiple attributes simultaneously and confirm they all reference the same source:”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.5 Beyond NeurIPS: A Systemic Problem · p. 11

    This dynamic exemplifies a recursive contamination cycle, wherein synthetic outputs saturate information ecosystems and undermine the verification infrastructure upon which knowledge systems depend.

    “This dynamic exemplifies what has been characterized as a recursive contamination cycle, wherein synthetic outputs saturate information ecosystems and undermine the verification infrastructure upon which knowledge systems depend [11].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] [11] Samar Ansari. AI slop and data pollution in the age of generative AI: Strategic risks, economic consequences, and governance pathways for business, management, and the creative industries. Available at SSRN, DOI: 10.2139/ssrn.5649410 , 2025. - source

  • May combine two claims.

    Interpretationreasoning
    § 4.5 Beyond NeurIPS: A Systemic Problem · p. 11

    The contamination is driven by systemic factors: tool accessibility, publication pressure, inadequate verification infrastructure, rather than individual misconduct.

    “This suggests the contamination is driven by systemic factors: tool accessibility, publication pressure, inadequate verification infrastructure, rather than individual misconduct.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § Compound Failure Structure: Primary vs Secondary Characteristics (All 100 Citations Exhibit Both Primary and Secondary Failure Modes) · p. 10

    The 27% PAC cases require verification tools to check not just existence but metadata consistency.

    “The 27% PAC cases require more sophisticated verification. Partial Attribute Corruption involves blending real and fake elements, requiring verification tools to check not just existence but metadata consistency.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationown data
    § 3.1 Prevalence by Failure Mode · p. 4

    The remaining categories-Identifier Hijacking (4%), Placeholder Hallucination (2%), and Semantic Hallucination (1%)-represent less common but potentially more sophisticated failure modes.

    “The remaining categories: Identifier Hijacking (4%), Placeholder Hallucination (2%), and Semantic Hallucination (1%), represent less common but potentially more sophisticated failure modes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data
    § 4.7 Limitations · p. 12

    The data comes from a single conference (NeurIPS 2025) in a single discipline (AI/ML).

    “Finally, the data comes from a single conference (NeurIPS 2025) in a single discipline (AI/ML).”
    Quote in the paperSubstantial claim
60 of 60

  • The paper relies on this without support.

    LLMs are increasingly used in academic writing workflows.

    “Large language models (LLMs) are increasingly used in academic writing workflows”

  • The paper relies on this without support.

    The 100 fabricated citations were generated by LLMs.

    “100 AIgenerated hallucinated citations”

  • The paper relies on this without support.

    The NeurIPS 2025 accepted papers are representative of elite peer review.

    “The NeurIPS 2025 contamination is especially concerning because it occurred at a premier AI conference where authors and reviewers are experts in the very systems that produce hallucinations.”

  • The paper relies on this without support.

    The five-category taxonomy captures all relevant failure modes of AI-generated citations.

    “we developed a five-category taxonomy based on the hallucination mechanism.”

  • The paper relies on this without support.

    NeurIPS 2025 received 21,575 submissions.

    “NeurIPS 2025 received 21,575 submissions, requiring thousands of volunteer reviewers.”

  • The paper relies on this without support.

    The fabricated citations in the dataset were generated by AI writing tools.

    “This study documents 100 AI-generated hallucinated citations that appeared in papers accepted by NeurIPS 2025”

  • The paper relies on this without support.

    Citations serve evidentiary support, reproducibility, and credit attribution functions.

    “Citations serve three critical functions: (1) Evidentiary support: they establish that claimed prior work actually exists and says what the authors claim; (2) Reproducibility: they enable readers to trace findings back to source data and methods; (3) Credit attribution: they acknowledge intellectual debt and situate new work within existing knowledge.”

  • The paper relies on this without support.

    The 53 papers containing 100 fabricated citations represent approximately 1% contamination rate at NeurIPS 2025.

    “This represents approximately 1% contamination rate at one of the world's most selective AI conferences.”

  • The paper relies on this without support.

    Automated citation verification tools exist and are already used in research integrity investigations.

    “Tools already exist for this purpose, including GPTZero's Hallucination Check, Crossref's citation checking APIs, and academic database verification protocols.”
    “Automated citation verification tools exist and are already used in research integrity investigations.”
9 of 9

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • A Total Fabrication example uses a valid identifier

    Minor

    The taxonomy defines Total Fabrication as requiring fabricated identifiers, but the cited compound example calls its metadata a Total Fabrication while stating that its arXiv ID is valid.

  • TF-plus-SH overlap is reported as both 63 and 50 cases

    Minor

    Figure 3 says the TF-primary, SH-secondary combination accounts for 63% of all 100 citations, or 63 cases. The results text explicitly reports 50 cases with that combination.

  • Reported median is incompatible with the paper counts

    Minor

    A median of 2 among 53 papers requires at least 27 papers to have at least two hallucinations. Given the reported groups, the minimum total would then be 49 + 27 + 3×4 + 13 = 101, not 100.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [1][1] GPTZero. GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers. https://gptzero.me/ news/neurips/ , 2026. Accessed: 2026-02-05. -
  2. [2][2] GPTZero. GPTZero finds over 50 new hallucinations in ICLR 2026 submissions. https://gptzero.me/ news/iclr-2026/ , 2025. Accessed: 2026-02-05. -

Cited by

Sorted by citations

Authors
Ansari, S.
Affiliations
University of Chester (GB)
Corresponding author
Ansari, Samar
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2602.05930
arXiv
2602.05930
OpenAlex
W7128093946
Semantic Scholar
0386e43a3899448ca1a427edaff586c8de82e70b
Cited by
0
References
9
Type
posted content
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
Keywords
large language models, peer review failure, failure mode classification, semantic hallucination, citation verification, scientific integrity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review