The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
CoverageBench: Evaluating Information Coverage across Tasks and Domains
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Cumulated gain-based evaluation of IR techniques
Järvelin, K., & Kekäläinen, J. (2002) · ACM Transactions on Information Systems · doi:10.1145/582415.582418 · Cited by 4,841
The probabilistic relevance framework: BM25 and beyond
Robertson, S., & Zaragoza, H. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000019 · Cited by 3,374
The use of MMR, diversity-based reranking for reordering documents and producing summaries
Carbonell, J., & Goldstein, J. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.291025 · Cited by 2,329
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Khattab, O., & Zaharia, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401075 · Cited by 1,301
Novelty and diversity in information retrieval evaluation
Clarke, C. L. A., Kolla, M., Cormack, G. V., Vechtomova, O., Ashkan, A., Büttcher, S., & MacKinnon, I. (2008) · Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1390334.1390446 · Cited by 962
Retrieval-Augmented Generation for Large Language Models: A Survey
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2312.10997 · Cited by 876
Rank-biased precision for measurement of retrieval effectiveness
Moffat, A., & Zobel, J. (2008) · ACM Transactions on Information Systems · doi:10.1145/1416950.1416952 · Cited by 620
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505 · cited 3×
“%) were recovered in the expanded judgment pool.”
InfluentialDouze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.-E., Lomeli, M., Hosseini, L., & Jégou, H. (2026) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3618474 · Cited by 478
Multi-News: A Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model
Fabbri, A., Li, I., She, T., Li, S., & Radev, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1102 · Cited by 449
Beyond independent relevance: methods and evaluation metrics for subtopic retrieval
Zhai, C. X., Cohen, W. W., & Lafferty, J. (2003) · Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval · doi:10.1145/860435.860440 · Cited by 366
Lin, J., Ma, X., Lin, S.-C., Yang, J.-H., Pradeep, R., & Nogueira, R. (2021) · Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3404835.3463238 · Cited by 360
The power of noise: Redefining retrieval for rag systems
Cuconasu, F., Trappolini, G., Siciliano, F., Filice, S., Campagnano, C., Maarek, Y., Tonellotto, N., & Silvestri, F. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657834 · Cited by 236
Evaluating diversified search results using per-intent graded relevance
Sakai, T., & Song, R. (2011) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2009916.2010055 · Cited by 163
Intent-based diversification of web search results: metrics and algorithms
Chapelle, O., Ji, S., Liao, C. Y., Velipasaoglu, E., Lai, L., & Wu, S.-L. (2011) · Information Retrieval · doi:10.1007/s10791-011-9167-7 · Cited by 125
A comparative analysis of cascade measures for novelty and diversity
Clarke, C. L. A., Craswell, N., Soboroff, I. M., & Ashkan, A. (2011) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/1935826.1935847 · Cited by 122
Self-citationWill Pyramids Built of Nuggets Topple Over?
Lin, J., & Demner-Fushman, D. (2006) · Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics - · doi:10.3115/1220835.1220884 · Cited by 77
Ragnarök: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track
Pradeep, R., Thakur, N., Sharifymoghaddam, S., Zhang, E., Nguyen, R., Campos, D., Craswell, N., & Lin, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_9 · Cited by 46
A distribution of impact factors of journals in the area of software: An empirical study
Sahoo, B. B., & Rao, I. K. R. (2006) · Information Processing & Management · doi:10.1016/j.ipm.2006.03.013 · Cited by 40
Summary of the NTCIR-10 INTENT-2 task
Sakai, T., Dou, Z., Yamamoto, T., Liu, Y., Zhang, M., Kato, M. P., Song, R., & Iwata, M. (2013) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2484028.2484104 · Cited by 27
Lin, J., & Zhang, P. (2007) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1277741.1277799 · Cited by 17
Thakur, N., Pradeep, R., Upadhyay, S., Campos, D., Craswell, N., Soboroff, I. M., Dang, H. T., & Lin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730165 · Cited by 12
Self-citationInitial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Framework
Pradeep, R., Thakur, N., Upadhyay, S., Campos, D., Craswell, N., & Lin, J. (2024) · arXiv · doi:10.48550/arxiv.2411.09607 · Cited by 6
Controlled Retrieval-augmented Context Evaluation for Long-form RAG
Ju, J.-H., Verberne, S., de Rijke, M., & Yates, A. (2025) · doi:10.18653/v1/2025.findings-emnlp.1151 · Cited by 2
Self-citation
Research works printed but not linked: 22
- 10.6028/NIST.SP.500-278.web-overview · DOI
- 10.6028/NIST.SP.500-294.web-overview · DOI
- 10.6028/NIST.SP.500-296.web-overview · DOI
- Jeffrey Dalton, Chenyan Xiong, and Jamie Callan. 2021. CAsT 2020: The Conversational Assistance Track Overview. In The Twenty-Ninth Text REtrieval Conference Proceedings (TREC 2020) (NIST Special Publication, Vol. 1266). National Institute of Standards and Technology (NIST). https://trec.nist.gov/pubs/trec29/papers/ OVERVIEW.C.pdf
- Michael D. Ekstrand, Graham McDonald, Amifa Raj, and Isaac Johnson. 2023. Overview of the TREC 2022 Fair Ranking Track. arXiv:2302.05558 [cs.IR] https://arxiv.org/abs/2302.05558 · arXiv
- Thibault Formal Carlos Lassance Benjamin Piwowarski and Stéphane Clinchant. 2021. SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval. arXiv:2109.10086 [cs.IR] https://arxiv.org/abs/2109.10086 · arXiv
- Dawn Lawrie, Sean MacAvaney, James Mayfield, Paul McNamee, Douglas W. Oard, Luca Soldaini, and Eugene Yang. 2025. Overviewof the TREC 2024 NeuCLIR Track. arXiv:2509.14355 [cs.IR] https://arxiv.org/abs/2509.14355 · arXiv
- Dawn Lawrie, Sean MacAvaney, James Mayfield, Luca Soldaini, Eugene Yang, and Andrew Yates. 2026. Overview of the TREC 2025 RAGTIME Track. arXiv:2602.10024 [cs.IR] https://arxiv.org/abs/2602.10024 · arXiv
- 10.5555/3495724.3496517 · doi:10.5555/3495724.3496517
- Jimmy Lin and Dina Demner-Fushman. 2005. Evaluating Summaries and Answers: Two Sides of the Same Coin?. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Jade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare Voss (Eds.). Association for Computational Linguistics, Ann Arbor, Michigan, 41-48. https://aclanthology.org/W05-0906/
- Yiqun Liu, Ruihua Song, Min Zhang, Zhicheng Dou, Takehiro Yamamoto, Makoto P. Kato, Hiroaki Ohshima, and Ke Zhou. 2014. Overview of the NTCIR-11 IMine Task. In Proceedings of the 11th NTCIR Conference on Evaluation of Information Access Technologies, NTCIR-11, National Center of Sciences, Tokyo, Japan, December 9-12, 2014, Noriko Kando, Hideo Joho, and Kazuaki Kishida (Eds.). National Institute of Informatics (NII). http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings11/pdf/NTCIR/ OVERVIEW/01-NTCIR11-OV-IMINE-LiuY.pdf
- Ani Nenkova and Rebecca Passonneau. 2004. Evaluating Content Selection in Summarization: The Pyramid Method. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004. Association for Computational Linguistics, Boston, Massachusetts, USA, 145-152. https://aclanthology.org/N04-1019/
- Paul Over. 1997. TREC-6 Interactive Report. In Proceedings of The Sixth Text REtrieval Conference, TREC 1997, Gaithersburg, Maryland, USA, November 19-21, 1997 (NIST Special Publication, Vol. 500-240), Ellen M. Voorhees and Donna K. Harman (Eds.). National Institute of Standards and Technology (NIST), 73-81. http://trec.nist.gov/pubs/trec6/papers/t6irep.ps
- 10.1016/j.ipm · DOI
- PaulOver and James Yen. 2004. An Introduction to DUC-2004: Intrinsic Evaluation of Generic News Text Summarization Systems. In Proceedings of the HLT/NAACL 2004 Document Understanding Workshop (DUC-2004). http://www-nlpir.nist.gov/ projects/duc/pubs/2004slides/duc2004intro.pdf
- Ruihua Song, Min Zhang, Tetsuya Sakai, Makoto P. Kato, Yiqun Liu, Miho Sugimoto, Qinglei Wang, and Naoki Orii. 2011. Overview of the NTCIR-9 INTENT Task. In Proceedings of the 9th NTCIR Workshop Meeting on Evaluation of Information Access Technologies: Information Retrieval, Question Answering and Cross-Lingual Information Access, NTCIR-9, National Center of Sciences, Tokyo, Japan, December 6-9, 2011, Noriko Kando, Daisuke Ishikawa, and Miho Sugimoto (Eds.). National Institute of Informatics (NII). http://research.nii.ac.jp/ntcir/ workshop/OnlineProceedings9/NTCIR/01-NTCIR9-OV-INTENT-SongR.pdf
- Ellen M. Voorhees and Dawn M. Tice. 2000. The TREC-8 Question Answering Track. In Proceedings of the Second International Conference on Language Resources and Evaluation (LREC'00), M. Gavrilidou, G. Carayannis, S. Markantonatou, S. Piperidis, and G. Stainhauer (Eds.). European Language Resources Association (ELRA), Athens, Greece. https://aclanthology.org/L00-1018/
- William Walden, Marc Mason, Orion Weller, Laura Dietz, John Conroy, Neil Molino, Hannah Recknor, Bryan Li, Gabrielle Kaili-May Liu, Yu Hou, Dawn Lawrie, James Mayfield, and Eugene Yang. 2025. Auto-ARGUE: LLM-Based Report Generation Evaluation. arXiv:2509.26184 [cs.IR] https://arxiv.org/abs/2509.26184 · arXiv
- Orion Weller, Kathryn Ricci, Eugene Yang, Andrew Yates, Dawn Lawrie, and Benjamin Van Durme. 2025. Rank1: Test-Time Compute for Reranking in Information Retrieval. In Second Conference on Language Modeling. https://openreview.net/forum?id=Pg0PAvbhGv
- Takehiro Yamamoto, Yiqun Liu, Min Zhang, Zhicheng Dou, Ke Zhou, Ilya Markov, Makoto P. Kato, Hiroaki Ohshima, and Sumio Fujita. 2016. Overview of the NTCIR-12 IMine-2 Task. In Proceedings of the 12th NTCIR Conference on Evaluation of Information Access Technologies, National Center of Sciences, Tokyo, Japan, June 7-10, 2016, Noriko Kando, Tetsuya Sakai, and Mark Sanderson (Eds.). National Institute of Informatics (NII). http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings12/pdf/ntcir/OVERVIEW/01-NTCIR12-OV-IMINE-YamamotoT.pdf
- Eugene Yang, Andrew Yates, Kathryn Ricci, Orion Weller, Vivek Chari, Benjamin Van Durme, and Dawn Lawrie. 2025. Rank-K: Test-Time Reasoning for Listwise Reranking. arXiv:2505.14432 [cs.IR] https://arxiv.org/abs/2505.14432 · arXiv
- Yanzhao Zhang Mingxin Li Dingkun Long Xin Zhang Huan Lin Baosong Yang Pengjun Xie An Yang Dayiheng Liu Junyang Lin Fei Huang and Jingren Zhou. 2025. Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models. arXiv:2506.05176 [cs.CL] https://arxiv.org/abs/2506.05176 · arXiv
Metadata
- Authors
- Samuel, S., Yates, A., Lawrie, D., Soboroff, I., Adriaanse, T., Van Durme, B., & Yang, E.
- Venue
- Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval
- Publisher
- ACM
- Published
- 2026-07-19
- Pages
- 3388-3395
- Cited by
- 0
- References
- 25
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
