The DOI matches its DataCite record; 71 of the 71 references it cites were found in a registry or an index; none of the 26 sources its claims cite is checked for support yet.
Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents
Open accessAbstract
Large language models and deep research agents supply citation URLs to support their claims, yet the reliability of these citations has not been systematically measured. We address six research questions about citation URL validity using 10 models and agents on DRBench (53,090 URLs) and 3 models on ExpertQA (168,021 URLs across 32 academic fields). We find that 3--13\% of citation URLs are hallucinated -- they have no record in the Wayback Machine and likely never existed -- while 5--18\% are non-resolving overall. Deep research agents generate substantially more citations per query than search-augmented LLMs but hallucinate URLs at higher rates. Domain effects are pronounced: non-resolving rates range from 5.4\% (Business) to 11.4\% (Theology), with per-model effects even larger. Decomposing failures reveals that some models fabricate every non-resolving URL, while others show substantial link-rot fractions indicating genuine retrieval. As a solution, we release urlhealth, an open-source tool for URL liveness checking and stale-vs-hallucinated classification using the Wayback Machine. In agentic self-correction experiments, models equipped with urlhealth reduce non-resolving citation URLs by $6\textrm{--}79\times$ to under 1\%, though effectiveness depends on the model's tool-use competence. The tool and all data are publicly available. Our characterization findings, failure taxonomy, and open-source tooling establish that citation URL validity is both measurable at scale and correctable in practice.
Overview
CoChat Summary
TL;DR
This paper measures citation URL validity for commercial LLMs and deep research agents, finding that 3–13% of citation URLs are hallucinated and 5–18% are non-resolving across DRBench and ExpertQA. It releases `urlhealth`, a Wayback Machine–based URL liveness/classification tool, and shows agentic self-correction can reduce non-resolving URLs by 6–79× to under 1%.
Key findings
- Across 10 models on DRBench (53,090 URLs) and 3 models on ExpertQA (168,021 URLs across 32 fields), hallucinated URL rates range from 3% to 13%, while non-resolving rates range from 5% to 18%.
- Deep research agents hallucinate more than search-augmented LLMs: pooled hallucination is 10.7% [10.2, 11.2] vs. 4.8% [4.3, 5.2], and non-resolving is 16.2% [15.7, 16.8] vs. 6.8% [6.2, 7.3].
- Domain effects are pronounced: non-resolving rates range from 5.4% [4.9, 5.9] (Business) to 11.4% [8.1, 14.6] (Theology), with per-model variation reaching 4.3× (Claude Sonnet 4.5: 4.0% Mathematics to 17.4% Healthcare/Medicine).
- In agentic self-correction on 435 ExpertQA questions, `urlhealth` reduces non-resolving citation URLs by 6–79× to under 1% (all p < 10⁻³⁵): GPT-5.1 16.0%→0.6%, Gemini 6.1%→0.1%, Claude 4.9%→0.8%.
Methods
- Uses two datasets: DRBench (100 multilingual research queries, pre-collected outputs from 23 models; 10 analyzed) and ExpertQA (2,177 expert-curated questions across 32 fields; 3 models evaluated).
- Extracts URLs from model text or API response structures, tests each with HTTP HEAD (falling back to GET), classifies non-resolving URLs via Wayback Machine snapshots as hallucinated (no snapshot) or stale (snapshot exists).
- Evaluates `urlhealth` in an agentic self-correction loop on 435 ExpertQA questions, where models can iteratively verify and correct citations; analysis uses bootstrap 95% CIs and two-proportion z-tests.
Limitations
- Stated: hallucination estimates rely on incomplete Wayback Machine coverage; URL liveness is point-in-time; bot-blocking/rate limiting can cause false positives; results cover three providers on two English-dominated benchmarks and may not generalize.
- Stated: DRBench uses pre-collected data without re-running models, so prompt variation and API version drift are uncontrolled; only three models were tested on ExpertQA due to API cost constraints.
- Stated/evident: fabricated snippets and invented bibliographic entries are documented qualitatively but not quantified; 403 responses and Reddit URLs are handled with conservative assumptions that make non-resolving rates lower bounds.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 3 Not valid · 14 Not checked · 140 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Detailed self-correction behavior.The three models exhibit distinct behavioral profiles.”
- Also stated in: § E Limitations · plain
“The three models exhibit distinct self-correction profiles.”
§ Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents > D Agentic self-correction details
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § E Limitations · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Claude Sonnet 4.5 (blue) concentrates at 3-5 rounds (83% of questions, median 4), reflecting the cost of verifying its large citation sets (18.4 URLs/question vs. 9.7-11.1 for the other models).”
§ Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents > E Limitations
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At the other extreme, fields like Architecture show tight subfield distributions, suggesting that non-resolving URL rates are relatively uniform within stable-content domains.”
§ Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents > Subfield-level analysis.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“429 responses are 99.1% not dead”
§ Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents > Sensitivity to classification decisions.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Automated URL checking is subject to false positives from bot-blocking and rate limiting.”
§ Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents > E Limitations
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
65 more claims
Create a free account to see all 157 claims and to verify the 5 references still open.
- It may state two separate claims as one. 65
- Probably not a substantial claim. 34
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents > 3.3 URL extraction and classification
“incomplete archive coverage can misclassify some hallucinated URLs as stale.”
§ Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents > E Limitations
“absent Wayback snapshots under-count stale URLs and over-count hallucinated ones”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“with a greater than 10× increase from 2024 (0.28% of papers) to 2025 (2.59%).”
§ Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents > Domain-specific citation fabrication.
Reported 10 · recomputed 9.25
Citations
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 4,688
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608
Chelli, M., Descamps, J., Lavoué, V., Trojani, C., Azar, M., Deckert, M., Raynier, J.-L., Clowez, G., Boileau, P., & Ruetsch-Chelli, C. (2024) · Journal of Medical Internet Research · doi:10.2196/53164 · Cited by 602
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024) · Journal of Legal Analysis · doi:10.1093/jla/laae003 · Cited by 559
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Manakul, P., Liusie, A., & Gales, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.557 · Cited by 557
Fabrication and errors in the bibliographic citations generated by ChatGPT
Walters, W. H., & Wilder, E. I. (2023) · Scientific Reports · doi:10.1038/s41598-023-41032-5 · Cited by 499
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.741 · Cited by 407
WebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272
Enabling Large Language Models to Generate Text with Citations
Gao, T., Yen, H., Yu, J., & Chen, D. (2023) · arXiv · doi:10.48550/arxiv.2305.14627 · Cited by 223
Enabling Large Language Models to Generate Text with Citations
Gao, T., Yen, H., Yu, J., & Chen, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.398 · Cited by 216
Aljamaan, F., Temsah, M.-H., Altamimi, I., Al-Eyadhy, A., Jamal, A., Alhasan, K., Mesallam, T. A., Farahat, M., & Malki, K. H. (2024) · JMIR Medical Informatics · doi:10.2196/54345 · Cited by 211
Evaluating Verifiability in Generative Search Engines
Liu, N. F., Zhang, T., & Liang, P. (2023) · arXiv · Cited by 143 · cited 1×
“Liu et al. (2023) audited four generative search engines (Bing Chat, NeevaAI, Perplexity, YouChat) and found that only 51.5% of generated sentences are fully supported by their citations, despite responses appearing highly informative (average perceived utility 4.50/5).”
InfluentialEvaluating Verifiability in Generative Search Engines
Liu, N., Zhang, T., & Liang, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.467 · Cited by 143
RARR: Researching and revising what language models say, using language models
Gao, L., Dai, Z., Pasupat, P., Chen, A., Chaganty, A. T., Fan, Y., Zhao, V., Lao, N., Lee, H., Juan, D.-C., et al. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.910 · Cited by 141
Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations
Zittrain, J. L., Albert, K., & Lessig, L. (2013) · Legal Information Management · doi:10.1017/s1472669614000255 · Cited by 135
How much of the web is archived?
Ainsworth, S. G., AlSum, A., SalahEldeen, H. M., Weigle, M. C., & Nelson, M. L. (2011) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/1998076.1998100 · Cited by 104
LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-Context QA
Zhang, J., Bai, Y., Lv, X., Gu, W., Liu, D., Zou, M., Cao, S., Hou, L., Dong, Y., Feng, L., et al. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.264 · Cited by 97
Mugaanyi, J., Cai, L., Cheng, S., Lu, C., & Huang, J. (2024) · Journal of Medical Internet Research · doi:10.2196/52935 · Cited by 74
An automated framework for assessing how well LLMs cite relevant medical references
Wu, K., Wu, E., Wei, K., Zhang, A., Casasola, A., Nguyen, T., Riantawan, S., Shi, P., Ho, D., & Zou, J. (2025) · Nature Communications · doi:10.1038/s41467-025-58551-6 · Cited by 72
Teaching language models to support answers with verified quotes
Menick, J., Trebacz, M., Mikulik, V., Aslanides, J., Song, F., Chadwick, M., Glaese, M., Young, S., Campbell-Gillingham, L., Irving, G., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.11147 · Cited by 53
Profiling web archive coverage for top-level domain and content language
AlSum, A., Weigle, M. C., Nelson, M. L., & Herbert Van De Sompel, A. (2014) · International Journal on Digital Libraries · doi:10.1007/s00799-014-0118-y · Cited by 48
Sakai, Y., Kamigaito, H., & Watanabe, T. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.2189 · Cited by 47
Effective Large Language Model Adaptation for Improved Grounding and Citation Generation
Ye, X., Sun, R., Arık, S. Ö., & Pfister, T. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.346 · Cited by 43
Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models
Shao, Y., Jiang, Y., Kanell, T., Xu, P., Khattab, O., & Lam, M. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.347 · Cited by 41
Linardon, J., Jarman, H. K., McClure, Z., Anderson, C., Liu, C., & Messer, M. (2025) · JMIR Mental Health · doi:10.2196/80371 · Cited by 40
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Manakul, P., Liusie, A., & Gales, M. J. F. (2023) · arXiv · doi:10.48550/arxiv.2303.08896 · Cited by 38
Do Language Models Know When They’re Hallucinating References?
Agrawal, A., Suzgun, M., Mackey, L., & Kalai, A. (2024) · Findings of the Association for Computational Linguistics: EACL 2024 · doi:10.18653/v1/2024.findings-eacl.62 · Cited by 37
Do Language Models Know When They’re Hallucinating References?
Agrawal, A., Suzgun, M., Mackey, L., & Kalai, A. T. (2023) · arXiv · doi:10.48550/arxiv.2305.18248 · Cited by 37
Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG
Singh, A., Ehtesham, A., Kumar, S., Khoei, T. T., & Vasilakos, A. V. (2025) · arXiv · doi:10.48550/arxiv.2501.09136 · Cited by 36
Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias
Algaba, A., Mazijn, C., Holst, V., Tori, F., Wenmackers, S., & Ginis, V. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.381 · Cited by 35
CiteME: Can Language Models Accurately Cite Scientific Claims?
Press, O., Hochlehnert, A., Prabhu, A., Udandarao, V., Press, O., & Bethge, M. (2024) · neural information processing systems · doi:10.52202/079017-0252 · Cited by 33
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
Bohnet, B., Tran, V. Q., Verga, P., Aharoni, R., Andor, D., Soares, L. B., Ciaramita, M., Eisenstein, J., Ganchev, K., Herzig, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.08037 · Cited by 30
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29
Ansari, S. (2026) · arXiv · doi:10.48550/arxiv.2602.05930 · Cited by 28
ExpertQA: Expert-Curated Questions and Attributed Answers
Malaviya, C., Lee, S., Chen, S., Sieber, E., Yatskar, M., & Roth, D. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.167 · Cited by 28
How deep do large language models internalize scientific literature and citation practices?
Algaba, A., Holst, V., Tori, F., Mobini, M., Verbeken, B., Wenmackers, S., & Ginis, V. (2026) · Quantitative Science Studies · doi:10.1162/qss.a.502 · Cited by 23
Correctness is not Faithfulness in Retrieval Augmented Generation Attributions
Wallat, J., Heuss, M., Rijke, M. D., & Anand, A. (2025) · Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) · doi:10.1145/3731120.3744592 · Cited by 21
CheckIfExist: Detecting Citation Hallucinations in the Era of AI-Generated ContentPreprint
Abbonato, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.15871 · Cited by 18
How Well Are Arabic Websites Archived?
Alkwai, L. M., Nelson, M. L., & Weigle, M. C. (2015) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/2756406.2756912 · Cited by 15
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · arXiv · doi:10.48550/arxiv.2305.14251 · Cited by 14
LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA
Zhang, J., Bai, Y., Lv, X., Gu, W., Liu, D., Zou, M., Cao, S., Hou, L., Dong, Y., Feng, L., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.02897 · Cited by 14
How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices?
Algaba, A., Holst, V., Tori, F., Mobini, M., Verbeken, B. F., Wenmackers, S., & Ginis, V. (2025) · Quantitative Science Studies · doi:10.48550/arxiv.2504.02767 · Cited by 14
HALoGEN: Fantastic LLM Hallucinations and Where to Find Them
Ravichander, A., Ghela, S., Wadden, D., & Choi, Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.71 · Cited by 13
HALoGEN: Fantastic LLM Hallucinations and Where to Find Them
Ravichander, A., Ghela, S., Wadden, D., & Choi, Y. (2025) · arXiv · doi:10.48550/arxiv.2501.08292 · Cited by 13
(2022) · doi:10.48550/arxiv.2210.08726 · Cited by 12
SemanticCite: Citation Verification with AI-Powered Full-Text Analysis and Evidence-Based ReasoningPreprint
Haan, S. W. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.16198 · Cited by 11
Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models
Shao, Y., Jiang, Y., Kanell, T. A., Xu, P., Khattab, O., & Lam, M. S. (2024) · arXiv · doi:10.48550/arxiv.2402.14207 · Cited by 10
Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations
Zittrain, J., & Albert, K. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2329161 · Cited by 7
Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM AttributionPreprint
Saxena, Y., Bommireddy, R., Padia, A., & Gaur, M. (2025) · arXiv (Cornell University) · doi:10.13016/m2l19h-mtre · Cited by 6
CiteME: Can Language Models Accurately Cite Scientific Claims?
Press, O., Hochlehnert, A., Prabhu, A., Udandarao, V., Press, O., & Bethge, M. (2024) · arXiv · doi:10.48550/arxiv.2407.12861 · Cited by 4
Correctness is not Faithfulness in RAG AttributionsPreprint
Wallat, J., Heuss, M., de Rijke, M., & Anand, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.18004 · Cited by 4
Learning to Generate Answers with Citations via Factual Consistency Models
Aly, R., Tang, Z., Tan, S., & Karypis, G. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.641 · Cited by 3
GhostCite: A Large-Scale Analysis of Citation Validity in the Age of Large Language Models
Xu, Z., Qiu, Y., Sun, L., Miao, F., Wu, F., Li, X., Wang, X., Lu, H., Zhang, Z., Hu, Y., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06718 · Cited by 3
DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
Du, M., Xu, B., Zhu, C., Wang, X., & Mao, Z. (2025) · arXiv · doi:10.48550/arxiv.2506.11763 · Cited by 2
ExpertQA: Expert-Curated Questions and Attributed Answers
Malaviya, C., Lee, S., Chen, S., Sieber, E., Yatskar, M., & Roth, D. (2023) · arXiv · doi:10.48550/arxiv.2309.07852 · Cited by 2
Sakai, Y., Kamigaito, H., & Watanabe, T. (2026) · arXiv · doi:10.48550/arxiv.2601.18724 · Cited by 2
CiteGuard: Faithful Citation Attribution for LLMs via Retrieval-Augmented Validation
Choi, Y. M., Guo, X., Fung, Y. R., & Wang, Q. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.282 · Cited by 1
Shi, K., Sun, W., Zhang, Z., Sun, L., Chawla, N. V., & Ye, Y. (2026) · arXiv · doi:10.48550/arxiv.2602.23452 · cited 3×
“FacTool (Chern et al., 2023) queries Google Scholar to verify citation existence; CheckIfExist (Abbonato, 2026) cascades queries across CrossRef, Semantic Scholar, and OpenAlex; CiteAudit (Yuan et al., 2026) proposes a multi-agent verification pipeline; and GhostCite (Xu et al., 2026) provides large-scale evidence of…”
DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity
Zhong, J., Zhang, H., Southern, C., Yang, J., Wang, T., Jung, K., Zhang, S., Yarats, D., Ho, J., & Ma, J. (2026) · arXiv · doi:10.48550/arxiv.2602.11685
CiteEval: Principle-Driven Citation Evaluation for Source Attribution
Xu, Y., Qi, P., Chen, J., Liu, K., Han, R., Liu, L., Min, B., Castelli, V., Gupta, A., & Wang, Z. (2025) · arXiv · doi:10.48550/arxiv.2506.01829
BibAgent: An Agentic Framework for Traceable Miscitation Detection in Scientific Literature
Li, P., Lin, F., Xing, S., Zheng, X., Hong, X., Yang, S., Sun, J., Tu, Z., & Ni, C. (2026) · arXiv · doi:10.48550/arxiv.2601.16993
CiteGuard: Faithful Citation Attribution for LLMs via Retrieval-Augmented Validation
Choi, Y. M., Guo, X., Fung, Y. R., & Wang, Q. (2025) · arXiv · doi:10.48550/arxiv.2510.17853
Generation-Time vs. Post-hoc Citation: A Holistic Evaluation of LLM Attribution
Saxena, Y., Bommireddy, R., Padia, A., & Gaur, M. (2025) · arXiv · doi:10.48550/arxiv.2509.21557
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Dahl, M., Magesh, V., Suzgun, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.01301
How Much of the Web Is Archived?
Ainsworth, S. G., AlSum, A., SalahEldeen, H., Weigle, M. C., & Nelson, M. L. (2012) · arXiv · doi:10.48550/arxiv.1212.6177
Large Language Models Reflect Human Citation Patterns with a Heightened Citation Bias
Algaba, A., Mazijn, C., Holst, V., Tori, F., Wenmackers, S., & Ginis, V. (2024) · arXiv · doi:10.48550/arxiv.2405.15739
Profiling Web Archive Coverage for Top-Level Domain and Content Language
AlSum, A., Weigle, M. C., Nelson, M. L., & de Sompel, H. V. (2013) · arXiv · doi:10.48550/arxiv.1309.4008
Research works printed but not linked: 2
- Gokul Srinath Seetha Ram. Citation drift: Measuring reference stability in multi-turn LLM conversations. In Proceedings of the Third Workshop for Artificial Intelligence for Scientific Publications, pp. 186-191, Mumbai, India and virtual, December 2025. Association for Computational Linguistics. URL https://aclanthology.org/2025.wasp-main.20/.
- Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter. Measuring Attribu- tion in Natural Language Generation Models. Computational Linguistics, 49(4):777-840, December 2023. doi: 10.1162/coli a 00486. URL https://direct.mit.edu/coli/article/ 49/4/777/116438. · doi:10.1162/colia00486
Cited by
HALLMARK: How to Diagnose LLM Citation Verifiers and When to Deploy ThemPreprint
Kim, K., Brendel, W., & Reizinger, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.18360 · Cited by 5
Per-Entity Bias Mapping for AI Visibility: Why Brand Mentions Require Entity-Specific CalibrationPreprint
Varga, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.21595 · Cited by 4
Verified Misguidance: Measuring Structural Citation Failures in Search-Augmented LLMsPreprint
Seo, Y., Jeong, W., Kim, E., Jang, H., & Lee, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.28565 · Cited by 2
Mohr, S. (2026) · doi:10.5194/egusphere-2026-5952 · Cited by 2
CentaurTA: A Self-Improving Human-Agents Collaboration Framework for Thematic Analysis
Wang, L., Huang, M., & Dragut, E. C. (2026) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2026.findings-acl.778
Metadata
- Authors
- Rao, D., Wong, E., & Callison-Burch, C.
- Affiliations
- University of Pennsylvania (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2604.03173
- OpenAlex
- W7150875458
- Semantic Scholar
- 14dc8f8dc652f5927d4e3e666715239ffb74672b
- Cited by
- 16
- References
- 48
- Citations by year
- 2026: 16
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- public-domain
- creativecommons.org/publicdomain/zero/1.0/legalcode
- Locations
- arXiv (Cornell University) (submittedVersion, public-domain)
- arXiv (Cornell University) (public-domain)
- Funders
- U.S. Department of Defense
- Defense Advanced Research Projects Agency
- Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- citation hallucination, large language models, deep research agents, Wayback Machine, link rot, tool use, citation reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-13, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-13 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
