The DOI matches its DataCite record; 9 of the 9 references it cites were found in a registry or an index.
Overview of the TREC 2020 deep learning track
Open accessAbstract
This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime. We again have a document retrieval task and a passage retrieval task, each with hundreds of thousands of human-labeled training queries. We evaluate using single-shot TREC-style evaluation, to give us a picture of which ranking methods work best when large data is available, with much more comprehensive relevance labeling on the small number of test queries. This year we have further evidence that rankers with BERT-style pretraining outperform other rankers in the large data regime.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Our experiments with the TREC Deep Learning (DL) Tracks from 2019-2023 [4-8] verified the claims by Thomas et al. [21] about LLM assessment being an effective alternative to humanbased assessment, using OpenAI's latest model GPT-4o.”
From UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessor (2024) · cites this paper as [4]
The cited source says what the claim attributes to it.
In the citing paper
“Our experiments with the TREC Deep Learning (DL) Tracks from 2019-2023 [4-8] verified the claims by Thomas et al. [21] about LLM assessment being an effective alternative to humanbased assessment, using OpenAI's latest model GPT-4o.”
From UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessor (2024) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
The Arcade Learning Environment: An Evaluation Platform for General Agents
Bellemare, M. G., Naddaf, Y., Veness, J., & Bowling, M. (2013) · Journal of Artificial Intelligence Research · doi:10.1613/jair.3912 · Cited by 2,161
Learning to Match using Local and Distributed Representations of Text for Web Search
Mitra, B., Díaz, F., & Craswell, N. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3038912.3052579 · Cited by 465
Self-citationUMass at TREC 2004: Novelty and HARD
AbdulJaleel, N., Allan, J., Croft, W. B., Díaz, F., Larkey, L. S., Li, X., Smucker, M. D., & Wade, C. (2004) · doi:10.21236/ada460118 · Cited by 274
From Neural Re-Ranking to Neural Ranking
Zamani, H., Dehghani, M., Croft, W. B., Miller, E. G., & Kamps, J. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3271800 · Cited by 212
High accuracy retrieval with multiple nested ranker
Matveeva, I. V., Burges, C. J. C., Burkard, T., Laucius, A., & Wong, L. (2006) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1148170.1148246 · Cited by 133
Optimizing Query Evaluations Using Reinforcement Learning for Web Search
Rosset, C., Jose, D., Ghosh, G., Mitra, B., & Tiwary, S. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210127 · Cited by 31
Self-citation
Research works printed but not linked: 12
- Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 , 2016. · arXiv
- Leonid Boytsov, David Novak, Yury Malkov, and Eric Nyberg. Off the beaten path: Let's replace term-based retrieval with k-nn search. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management , pages 1099-1108. ACM, 2016.
- Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, and Bodo Billerbeck. Orcas: 18 million clicked query-document pairs for analyzing search. arXiv preprint arXiv:2006.05324 , 2020. · arXiv
- Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 , 2016. · arXiv
- Bhaskar Mitra and Nick Craswell. An updated duet model for passage re-ranking. arXiv preprint arXiv:1903.07666 , 2019. · arXiv
- Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, and Emine Yilmaz. Incorporating query term independence assumption for efficient retrieval and ranking using deep neural networks. arXiv preprint arXiv:1907.03693 , 2019. · arXiv
- Rodrigo Nogueira, Wei Yang, Jimmy Lin, and Kyunghyun Cho. Document expansion by query prediction. arXiv preprint arXiv:1904.08375 , 2019. · arXiv
- Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval , 3(4):333-389, 2009.
- Trevor Strohman, Donald Metzler, Howard Turtle, and W Bruce Croft. Indri: A language model-based search engine for complex queries. In Proceedings of the International Conference on Intelligent Analysis , volume 2, pages 2-6. Citeseer, 2005.
- Lidan Wang, Jimmy Lin, and Donald Metzler. A cascade ranking model for efficient ranked retrieval. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval , pages 105-114. ACM, 2011.
- Wei Yang, Kuang Lu, Peilin Yang, and Jimmy Lin. Critically examining the 'neural hype': Weak baselines and the additivity of effectiveness gains from neural ranking models. In Proc. SIGIR , pages 1129-1132. ACM, 2019a.
- Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le. Xlnet: Generalized autoregressive pretraining for language understanding. arXiv preprint arXiv:1906.08237 , 2019b. · arXiv
Cited by
Sorted by citations
Precise Zero-Shot Dense Retrieval without Relevance Labels
Gao, L., Ma, X., Lin, J., & Callan, J. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.99 · Cited by 362
Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling
Hofstätter, S., Lin, S.-C., Yang, J.-H., Lin, J., & Hanbury, A. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462891 · Cited by 374
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents
Sun, W., Yan, L., Ma, X., Wang, S., Ren, P., Chen, Z., Yin, D., & Ren, Z. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.923 · Cited by 270
Query2doc: Query Expansion with Large Language Models
Wang, L., Yang, N., & Wei, F. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.585 · Cited by 214
COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List
Gao, L., Dai, Z., & Callan, J. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.241 · Cited by 265
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
Information Retrieval: Recent Advances and Beyond
Hambarde, K., & Proença, H. P. M. C. (2023) · IEEE Access · doi:10.1109/access.2023.3295776 · Cited by 212
Generation-Augmented Retrieval for Open-Domain Question Answering
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., & Chen, W. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.316 · Cited by 146
Semantic Models for the First-Stage Retrieval: A Comprehensive Review
Guo, J., Cai, Y., Fan, Y., Sun, F., Zhang, R., & Cheng, X. (2022) · ACM Transactions on Information Systems · doi:10.1145/3486250 · Cited by 153
Wang, F., Zhang, Z., Zhang, X., Wu, Z., Mo, T., Lu, Q., Wang, W., Li, R., Xu, J., Tang, X., et al. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3768165 · Cited by 106
Yang, L., Zhang, M., Li, C., Bendersky, M., & Najork, M. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3411908 · Cited by 97
da Cunha, W. L. M., Mangaravite, V., Gomes, C., Canuto, S., Resende, E., Nascimento, C., Viegas, F. A. R., França, C., Martins, W. S., Almeida, J. M., et al. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2020.102481 · Cited by 92
TILDE: Term Independent Likelihood moDEl for Passage Re-ranking
Zhuang, S., & Zuccon, G. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462922 · Cited by 92
PARADE: Passage Representation Aggregation forDocument Reranking
Li, C., Yates, A., MacAvaney, S., He, B., & Sun, Y. (2023) · ACM Transactions on Information Systems · doi:10.1145/3600088 · Cited by 184
BERT-based Dense Retrievers Require Interpolation with BM25 for Effective Passage Retrieval
Wang, S., Zhuang, S., & Zuccon, G. (2021) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3471158.3472233 · Cited by 84
Sun, S., & Duh, K. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.340 · Cited by 64
The Information Retrieval Experiment Platform
Fröbe, M., Reimer, J. H., MacAvaney, S., Deckers, N., Reich, S., Bevendorff, J., Stein, B., Hagen, M., & Potthast, M. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591888 · Cited by 62
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Craswell, N., Mitra, B., Yılmaz, E., Campos, D., & Lin, J. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3462804 · Cited by 66
Self-citationLess is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder
Lu, S., He, D., Xiong, C., Ke, G., Malik, W., Dou, Z., Bennett, P., Liu, T.-Y., & Overwijk, A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.220 · Cited by 49
How to Train Your Dragon: Diverse Augmentation Towards Generalizable Dense Retrieval
Lin, S.-C., Asai, A., Li, M., Oğuz, B., Lin, J., Mehdad, Y., Yih, W.-T., & Chen, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.423 · Cited by 147
Generative Relevance Feedback with Large Language Models
Mackie, I., Chatterjee, S., & Dalton, J. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591992 · Cited by 65
SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval
Wang, L., Yang, N., Huang, X., Jiao, B., Yang, L., Jiang, D., Majumder, R., & Wei, F. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.125 · Cited by 111
Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Prediction
Ma, X., Guo, J., Zhang, R., Fan, Y., & Cheng, X. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531772 · Cited by 35
Clickbait Spoiling via Question Answering and Passage Retrieval
Hagen, M., Fröbe, M., Jurk, A., & Potthast, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.484 · Cited by 35
Lexically-Accelerated Dense Retrieval
Kulkarni, H., MacAvaney, S., Goharian, N., & Frieder, O. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591715 · Cited by 31
Guo, Y., Ma, Z., Mao, J., Qian, H., Zhang, X., Jiang, H., Cao, Z., & Dou, Z. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532086 · Cited by 30
ConTextual Masked Auto-Encoder for Dense Passage Retrieval
Wu, D. D., Ma, G., Lin, M. S., Lin, Z., Wang, Z., & Hu, S. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i4.25598 · Cited by 46
A Statutory Article Retrieval Dataset in French
Louis, A., & Spanakis, G. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.468 · Cited by 52
A 'Pointwise-Query, Listwise-Document' based Query Performance Prediction Approach
Datta, S., MacAvaney, S., Ganguly, D., & Greene, D. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531821 · Cited by 33
How Does Generative Retrieval Scale to Millions of Passages?
Pradeep, R., Hui, K., Gupta, J. P., Lelkes, A. D., Zhuang, H., Lin, J., Metzler, D., & Tran, V. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.83 · Cited by 70
Zhuang, S., & Zuccon, G. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531951 · Cited by 33
Soft prompt tuning for augmenting dense retrieval with large language models
Peng, Z., Wu, X., Wang, Q., & Fang, Y. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112758 · Cited by 43
Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval
Tang, H., Sun, X., Jin, B., Wang, J., Zhang, F., & Wu, W. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.392 · Cited by 26
Challenges in Generalization in Open Domain Question Answering
Liu, L., Lewis, P., Riedel, S., & Stenetorp, P. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.155 · Cited by 34
Exploring the Landscape of Recommender Systems Evaluation: Practices and Perspectives
Bauer, C., Zangerle, E., & Said, A. (2023) · ACM Transactions on Recommender Systems · doi:10.1145/3629170 · Cited by 25
Long Document Ranking with Query-Directed Sparse Transformer
Jiang, J., Xiong, C., Lee, C.-J., & Wang, W. W. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.412 · Cited by 26
Otero, D., Parapar, J., & Ferro, N. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3614916 · Cited by 22
Dealing with textual noise for robust and effective BERT re-ranking
Chen, X., He, B., Hui, K., Sun, L., & Sun, Y. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103135 · Cited by 24
Multi-stage Training with Improved Negative Contrast for Neural Passage Retrieval
Lu, J. R., Ábrego, G. H., Ma, J., Ni, J., & Yang, Y. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.492 · Cited by 23
Towards a Better Understanding of Human Reading Comprehension with Brain Signals
Ye, Z., Xie, X., Liu, Y., Wang, Z., Chen, X., Zhang, M., & Ma, S. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511966 · Cited by 18
To Interpolate or not to Interpolate
Li, H., Wang, S., Zhuang, S., Mourad, A., Ma, X., Lin, J., & Zuccon, G. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531884 · Cited by 18
Trial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision
Wang, Z., & Sun, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.476 · Cited by 29
A Static Pruning Study on Sparse Neural Retrievers
Lassance, C., Lupart, S., Déjean, H., Clinchant, S., & Tonellotto, N. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591941 · Cited by 17
Fine-Grained Distillation for Long Document Retrieval
Zhou, Y., Shen, T., Geng, X., Tao, C., Shen, J., Long, G., Xu, C., & Jiang, D. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29947 · Cited by 26
UMBRELA: UMbrela is the (Open-Source Reproduction of the) Bing RELevance Assessor
Upadhyay, S., Pradeep, R., Thakur, N., Craswell, N., & Lin, J. (2024) · arXiv · doi:10.48550/arxiv.2406.06519 · Cited by 16
“To demonstrate LLM capabilities for generating relevance assessment, we took human relevance judgments (also known as qrels) from the Deep Learning Tracks in TREC 2019–2023 [4–8] and re-assessed them with UMBRELA.”
MethodSelf-citationOptimizing Guided Traversal for Fast Learned Sparse Retrieval
Qiao, Y., Yang, Y., Lin, H., & Yang, T. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583497 · Cited by 16
The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
Li, M., Popa, D. N., Chagnon, J., Cinar, Y. G., & Gaussier, É. (2022) · ACM Transactions on Information Systems · doi:10.1145/3568394 · Cited by 20
Choi, E., Lee, S., Choi, M., Ko, H., Song, Y.-I., & Lee, J. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557456 · Cited by 20
Towards Robust Ranker for Text Retrieval
Zhou, Y., Shen, T., Geng, X., Tao, C., Xu, C., Long, G., Jiao, B., & Jiang, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.332 · Cited by 20
SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes
Li, M., Lin, S.-C., Ma, X., & Lin, J. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591977 · Cited by 15
Understanding Differential Search Index for Text Retrieval
Chen, X., Liu, Y., He, B., Sun, L., & Sun, Y. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.681 · Cited by 19
QUEST: A Retrieval Dataset of Entity-Seeking Queries with Implicit Set Operations
Malaviya, C., Shaw, P., Chang, M.-W., Lee, K., & Toutanova, K. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.784 · Cited by 36
CogTaskonomy: Cognitively Inspired Task Taxonomy Is Beneficial to Transfer Learning in NLP
Luo, Y., Xu, M., & Xiong, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.64 · Cited by 14
Representation Sparsification with Hybrid Thresholding for Fast SPLADE-based Document Retrieval
Qiao, Y., Yang, Y., He, S., & Yang, T. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3592051 · Cited by 12
Soft Prompt Decoding for Multilingual Dense Retrieval
Huang, Z., Zeng, H., Zamani, H., & Allan, J. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591769 · Cited by 13
Xiong, Y., Tu, X., & Zhao, W. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104232 · Cited by 13
On the Interpolation of Contextualized Term-based Ranking with BM25 for Query-by-Example Retrieval
Abolghasemi, A., Askari, A., & Verberne, S. (2022) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3539813.3545133 · Cited by 11
Zerveas, G., Rekabsaz, N., Cohen, D., & Eickhoff, C. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.727 · Cited by 16
Li, M., Lin, S.-C., Oğuz, B., Ghoshal, A., Lin, J., Mehdad, Y., Yih, W.-T., & Chen, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.663 · Cited by 31
Implicit Feedback for Dense Passage Retrieval
Zhuang, S., Li, H., & Zuccon, G. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531994 · Cited by 11
Query2Prod2Vec: Grounded Word Embeddings for eCommerce
Bianchi, F., Tagliabue, J., & Yu, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-industry.20 · Cited by 10
Contrastive Document Representation Learning with Graph Attention Networks
Xu, P., Chen, X., Ma, X., Huang, Z., & Xiang, B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.327 · Cited by 13
Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencoder
Liu, Z., Zhang, H., Xiong, C., Liu, Z., Gu, Y., & Li, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.384 · Cited by 19
Multi-stage Large Language Model Pipelines Can Outperform GPT-4o in Relevance Assessment
Schnabel, J. A., Trippas, J. R., Scholer, F., & Hettiachchi, D. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3715488 · Cited by 9
Human Preferences as Dueling Bandits
Yan, X., Luo, C., Clarke, C. L. A., Craswell, N., Voorhees, E. M., & Castells, P. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531991 · Cited by 9
Self-citationFew-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision
Sun, S. Y., Qian, Y., Liu, Z., Xiong, C., Zhang, K., Bao, J., Liu, Z., & Bennett, P. N. (2021) · doi:10.18653/v1/2021.acl-long.390 · Cited by 14
Building efficient and effective OpenQA systems for low-resource languages
Budur, E., Özçelik, R., Soylu, D., Khattab, O., Güngör, T., & Potts, C. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112243 · Cited by 12
LexBoost: Improving Lexical Document Retrieval with Nearest Neighbors
Kulkarni, H., Goharian, N., Frieder, O., & MacAvaney, S. (2024) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/3685650.3685658 · Cited by 8
RetroMAE-2: Duplex Masked Auto-Encoder For Pre-Training Retrieval-Oriented Language Models
Liu, Z., Xiao, S., Shao, Y., & Cao, Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.148 · Cited by 18
Expand, Highlight, Generate: RL-driven Document Generation for Passage Reranking
Askari, A., Aliannejadi, M., Meng, C., Kanoulas, E., & Verberne, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.623 · Cited by 12
Intent-Oriented Dynamic Interest Modeling for Personalized Web Search
Bai, Y., Zhou, Y., Dou, Z., & Wen, J.-R. (2024) · ACM Transactions on Information Systems · doi:10.1145/3639817 · Cited by 9
Balanced Topic Aware Sampling for Effective Dense Retriever: A Reproducibility Study
Wang, S., & Zuccon, G. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591915 · Cited by 7
Revisiting Document Expansion and Filtering for Effective First-Stage Retrieval
Mansour, W. A., Zhuang, S., Zuccon, G., & Mackenzie, J. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657850 · Cited by 5
Collective Relevance Labeling for Passage Retrieval
Kim, J., Kim, M., & Hwang, S.-W. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.305 · Cited by 5
ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference
Hui, K., Zhuang, H., Chen, T., Qin, Z., Lü, J., Bahri, D., Ma, J., Gupta, J. P., dos Santos, C. N., Tay, Y., et al. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.295 · Cited by 6
Query sampler: generating query sets for analyzing search engines using keyword research tools
Schultheiß, S., Lewandowski, D., von Mach, S., & Yagci, N. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1421 · Cited by 5
Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-Hoc Retrieval
Su, W., Ai, Q., Li, X., Chen, J., Liu, Y., Wu, X., & Hou, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29869 · Cited by 17
CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion
He, X., Gong, Y., Jin, A.-L., Zhang, H., Dong, A., Jiao, J., Yiu, S., & Duan, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.651 · Cited by 7
Pan, M., Zhou, S., Chen, J., Huang, E. A., & Huang, J. X. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2024.104058 · Cited by 5
Decoding a Neural Retriever’s Latent Space for Query Suggestion
Adolphs, L., Huebscher, M. C., Buck, C., Girgin, S., Bachem, O., Ciaramita, M., & Hofmann, T. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.601 · Cited by 9
Zhan, J., Mao, J., Liu, Y., Guo, J., Zhang, M., & Ma, S. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/754 · Cited by 4
Movin, M., & Hauff, C. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730119 · Cited by 3
A Time-Optimized Content Creation Workflow for Remote Teaching
Hofstätter, S., Althammer, S., Sertkan, M., & Hanbury, A. (2022) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3478431.3499421 · Cited by 4
SqCLIRIL: Spoken query cross-lingual information retrieval in Indian languages
Dave, B., & Majumder, P. (2025) · Pattern Recognition Letters · doi:10.1016/j.patrec.2025.08.022 · Cited by 5
Cross-Model Comparative Loss for Enhancing Neuronal Utility in Language Understanding
Zhu, Y., Pang, L., Wu, K., Lan, Y., Shen, H., & Cheng, X. (2024) · ACM Transactions on Information Systems · doi:10.1145/3652599 · Cited by 3
Compact Token Representations with Contextual Quantization for Efficient Document Re-ranking
Yang, Y., Qiao, Y., & Yang, T. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.51 · Cited by 7
Neural methods for effective, efficient, and exposure-aware information retrieval
Mitra, B. (2021) · ACM SIGIR Forum · doi:10.1145/3476415.3476434 · Cited by 3
Self-citationSocialformer: Social Network Inspired Long Document Modeling for Document Ranking
Zhou, Y., Dou, Z., Yuan, H., & Ma, Z.-Y. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511962 · Cited by 2
Wolaytta-English Cross-lingual Information Retrieval using Neural Machine Translation
Bedecho, A. T., & Woldeyohannis, M. M. (2022) · International Conference on Information and Communication Technology for Development for Africa (ICT4DA) · doi:10.1109/ict4da56482.2022.9971267 · Cited by 2
Ranger: A Toolkit for Effect-Size Based Multi-Task Evaluation
Sertkan, M., Althammer, S., & Hofstätter, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.56 · Cited by 5
Hybrid and Collaborative Passage Reranking
Zhang, Z., Zhou, W., Shi, J., & Li, H. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.880 · Cited by 3
Query-as-context Pre-training for Dense Passage Retrieval
Xing, W., Ma, G., Qian, W., Lin, Z., & Hu, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.118 · Cited by 7
Survey of Query correction for Thai business-oriented information retrieval
Kittiworapanya, P., Saelek, N., Lertpiya, A., & Chalothorn, T. (2020) · International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP) · doi:10.1109/isai-nlp51646.2020.9376809 · Cited by 2
Discriminative Language Model via Self-Teaching for Dense Retrieval
Chen, L., Zhang, R., Guo, J., Fan, Y., & Cheng, X. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557582 · Cited by 1
Pseudo-Relevance for Enhancing Document Representation
Kim, J., Hwang, S.-W., Song, S. H., Ko, H., & Song, Y.-I. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.800 · Cited by 1
Genetic Generative Information Retrieval
Kulkarni, H., Young, Z., Goharian, N., Frieder, O., & MacAvaney, S. (2023) · ACM Symposium on Document Engineering (DocEng) · doi:10.1145/3573128.3609340 · Cited by 1
Die Erkennung unangemessener Inhalte im Internet: KI-Verfahren, Evaluierung und Herausforderungen
Mandl, T. (2020) · Bibliotheksdienst · doi:10.1515/bd-2020-0035
Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning TrackPreprint
Mitra, B., Hofstätter, S., Zamani, H., & Craswell, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.07368 · Cited by 2
Self-citationPseudo Label based Contrastive Sampling for Long Text Retrieval
Zhu, L., Shi, S., & Huang, H. (2021) · International Conference on Asian Language Processing · doi:10.1109/ialp54817.2021.9675219
Improving the Performance of Pre-trained Systems in Sentence Retrieval
Rughbeer, Y., Pillay, A. W., & Jembere, E. (2021) · International Conference on Information Fusion (FUSION) · doi:10.23919/fusion49465.2021.9627046
Metadata
- Authors
- Craswell, N., Mitra, B., Yilmaz, E., & Campos, D.
- Affiliations
- Microsoft (United States) (US)
- University of Illinois Urbana-Champaign (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2102.07662
- OpenAlex
- W3175111331
- Cited by
- 107
- References
- 9
- Citations by year
- 2025: 6 · 2024: 10 · 2023: 38 · 2022: 31 · 2021: 16
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- ad hoc retrieval, document retrieval, passage retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
