CoChat
Sign Up
PreprintCited by 1scientometrics and bibliometrics researchAcademic integrity and plagiarismAcademic Writing and Publishing

BibAgent: An Agentic Framework for Traceable Miscitation Detection in Scientific Literature

Open access
Li, Peiran, Lin, Fangzhou, Xing, Shuo, Zheng, Xiang, Hong, Xi, Yang, Siyuan, Sun, Jiashuo, Tu, Zhengzhong +1 more·2026·arXiv
DOI 10.48550/arxiv.2601.16993arXiv 2601.16993
Open this paper in CoChat — free accountOpen PDF

Abstract

Citations are the bedrock of scientific authority, yet their integrity is compromised by widespread miscitations: ranging from nuanced distortions to fabricated references. Systematic citation verification is currently unfeasible; manual review cannot scale to modern publishing volumes, while existing automated tools are restricted by abstract-only analysis or small-scale, domain-specific datasets in part due to the "paywall barrier" of full-text access. We introduce BibAgent, a scalable, end-to-end agentic framework for automated citation verification. BibAgent integrates retrieval, reasoning, and adaptive evidence aggregation, applying distinct strategies for accessible and paywalled sources. For paywalled references, it leverages a novel Evidence Committee mechanism that infers citation validity via downstream citation consensus. To support systematic evaluation, we contribute a 5-category Miscitation Taxonomy and MisciteBench, a massive cross-disciplinary benchmark comprising 6,350 miscitation samples spanning 254 fields. Our results demonstrate that BibAgent outperforms state-of-the-art Large Language Model (LLM) baselines in citation verification accuracy and interpretability, providing scalable, transparent detection of citation misalignments across the scientific literature.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2601.16993

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Real citations deployed to support claims the cited references do not actually make are more prevalent and harder to detect than hallucinated titles.

    “The more prevalent and harder-to-detect variant-real citations deployed to support claims the cited references do not actually make [35], [36], [37], [38]-remains an open challenge for which reliable detection methods remain under active development.”

    From LLM hallucinations in the wild: Large-scale evidence from non-existent citations (2026) · cites this paper as [36]

    The Introduction (p. 2) describes real references being misused to support claims and explains why checking such citations is difficult. It does not say that this problem is more prevalent or harder to detect than hallucinated titles. The Introduction (p. 2) also supports the sentence’s characterization of reliable detection as an open challenge. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Reliable detection methods for misrepresenting citations remain under active development.

    “remains an open challenge for which reliable detection methods remain under active development.”

    From LLM hallucinations in the wild: Large-scale evidence from non-existent citations (2026) · cites this paper as [36]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 21

  1. Athar, A. and Teufel, S. Detection of implicit citations for sentiment detection. In Van Den Bosch, A. and Shatkay, H. (eds.), Proceedings of the Workshop on Detecting Structure in Scholarly Discourse , pp. 18-26, Jeju Island, Korea, July 2012. Association for Computational Linguistics. URL https://aclanthology.org/ W12-4303/ .
  2. Clarivate. Journal citation reports, 2025. URL https:
  3. Dai, C., Chen, Q., Wan, T., Liu, F., Gong, Y., and Wang, Q. Literary runaway: Increasingly more references cited per academic research article from 1980 to 2019. Plos one , 16(8):e0255849, 2021.
  4. De Lacey, G., Record, C., and Wade, J. How accurate are quotations and references in medical journals? Br Med J (Clin Res Ed) , 291(6499):884-886, 1985.
  5. Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 , 2024. · arXiv
  6. Jergas, H. and Baethge, C. Quotation accuracy in medical journal articles-a systematic review and meta-analysis. PeerJ , 3:e1364, 2015.
  7. Kilicoglu, H., Peng, Z., Tafreshi, S., Tran, T., Rosemblat, G., and Schneider, J. Confirm or refute?: A comparative study on citation sentiment classification in clinical research publications. Journal of biomedical informatics , 91:103123, 2019.
  8. Li, X., Burns, G., and Peng, N. A paragraph-level multi-task learning model for scientific fact-verification, 2021. URL https://arxiv.org/abs/2012.14500 . · arXiv
  9. Liang, W., Zhang, Y., Wu, Z., Lepp, H., Ji, W., Zhao, X., Cao, H., Liu, S., He, S., Huang, Z., et al. Mapping the increasing use of llms in scientific papers. arXiv preprint arXiv:2404.01268 , 2024. · arXiv
  10. Pride, D. and Knoth, P. Incidental or influential?-challenges in automatically detecting citation importance using publication full texts. In International conference on theory and practice of digital Libraries , pp. 572-578. Springer, 2017.
  11. Pride, D. and Knoth, P. An authoritative approach to citation classification. In Proceedings of the ACM/IEEE joint conference on digital libraries in 2020 , pp. 337-340, 2020.
  12. Rao, V. S., Kumar, A., Lakkaraju, H., and Shah, N. B. Detecting llm-generated peer reviews. PLoS One , 20(9): e0331871, 2025.
  13. Rekdal, O. B. Academic urban legends. Social Studies of Science , 44(4):638-654, 2014.
  14. Teufel, S., Siddharthan, A., and Tidhar, D. Automatic classification of citation function. In Proceedings of the 2006 conference on empirical methods in natural language processing , pp. 103-110, 2006.
  15. Wadden, D. and Lo, K. Overview and insights from the sciver shared task on scientific claim verification. arXiv preprint arXiv:2107.08188 , 2021. · arXiv
  16. Wadden, D., Lin, S., Lo, K., Wang, L. L., van Zuylen, M., Cohan, A., and Hajishirzi, H. Fact or fiction: Verifying scientific claims. arXiv preprint arXiv:2004.14974 , 2020. · arXiv
  17. Wadden, D., Lo, K., Wang, L. L., Cohan, A., Beltagy, I., and Hajishirzi, H. MultiVerS: Improving scientific claim verification with weak supervision and fulldocument context. In Carpuat, M., de Marneffe, M.-C., and Meza Ruiz, I. V. (eds.), Findings of the Association for Computational Linguistics: NAACL 2022 , pp. 61-76, Seattle, United States, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022. findings-naacl.6. URL https://aclanthology. org/2022.findings-naacl.6/ . · DOI
  18. Walters, W. H. and Wilder, E. I. Fabrication and errors in the bibliographic citations generated by chatgpt. Scientific Reports , 13(1):14045, 2023.
  19. Waltman, L. A review of the literature on citation impact indicators. Journal of informetrics , 10(2):365-391, 2016.
  20. Wei, J., Huang, D., Lu, Y., Zhou, D., and Le, Q. V. Simple synthetic data reduces sycophancy in large language models, 2024. URL https://arxiv.org/abs/2308. 03958 .
  21. Zhang, T. M. and Abernethy, N. F. Detecting reference errors in scientific literature with large language models, 2024. URL https://arxiv.org/abs/2411. 06101 .

Cited by

Sorted by citations

Authors
Li, P., Lin, F., Xing, S., Zheng, X., Hong, X., Yang, S., Sun, J., Tu, Z., & Ni, C.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2601.16993
arXiv
2601.16993
OpenAlex
W7125769172
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by-nc-sa/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Safety Research › Academic integrity and plagiarism
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
Keywords
citation verification, agentic framework, large language models, scientific publications, evidence aggregation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review