Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents
Open accessAbstract
Large Language Models (LLMs) have demonstrated remarkable zero-shot generalization across various language-related tasks, including search engines.However, existing work utilizes the generative ability of LLMs for Information Retrieval (IR) rather than direct passage ranking.The discrepancy between the pretraining objectives of LLMs and the ranking objective poses another challenge.In this paper, we first investigate generative LLMs such as ChatGPT and GPT-4 for relevance ranking in IR.Surprisingly, our experiments reveal that properly instructed LLMs can deliver competitive, even superior results to state-of-the-art supervised methods on popular IR benchmarks.Furthermore, to address concerns about data contamination of LLMs, we collect a new test set called NovelEval, based on the latest knowledge and aiming to verify the model's ability to rank unknown knowledge.Finally, to improve efficiency in real-world applications, we delve into the potential for distilling the ranking capabilities of ChatGPT into small specialized models using a permutation distillation scheme.Our evaluation results turn out that a distilled 440M model outperforms a 3B supervised model on the BEIR benchmark.The code to reproduce our results is available at www.github.com/sunnweiwei/RankGPT.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
RBO is a widely used metric for evaluating the similarity between two ranked lists and is particularly relevant for information retrieval.
“Rank-biased overlap (RBO)foot_6 is a widely used metric for evaluating the similarity between two ranked lists and is particularly relevant for tasks in information retrieval (Oh et al., 2022;Sun et al., 2023).”
From Benchmarking Cognitive Biases in Large Language Models as Evaluators (2024) · cites this paper as Sun et al., 2023
The RBO discussion (page unavailable) says the metric calculates similarity between two rankings and uses it to assess consistency in a passage re-ranking task. This supports its relevance to information retrieval, but the paper does not establish that RBO is widely used. (LLM, conf=0.82)
Experimental — accuracy not yet measuredSun et al. (2023) show that LLMs can match or exceed strong supervised re-rankers on standard IR benchmarks, validating the 'generative engine as LLM re-ranker' abstraction.
“A representative empirical anchor is Sun et al. (2023), who show that LLMs can match or exceed strong supervised re-rankers on standard IR benchmarks, validating the 'generative engine as LLM re-ranker' abstraction we build on.”
From E-GEO: A Testbed for Generative Engine Optimization in E-Commerce (2025) · cites this paper as Sun et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575 · see citing works
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836 · see citing works
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · see citing works
MS MARCO: A Human Generated MAchine Reading COmprehension DatasetPreprint
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T. G., et al. (2016) · arXiv (Cornell University) · doi:10.4230/lipics.giscience.2025.5 · Cited by 1,190 · see citing works
Nogueira, R., & Cho, K. (2019) · arXiv · doi:10.48550/arxiv.1901.04085 · Cited by 615 · see citing works
Document Ranking with a Pretrained Sequence-to-Sequence Model
Nogueira, R. F., Jiang, Z., Pradeep, R., & Lin, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.63 · Cited by 521 · see citing works
Holistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480 · see citing works
Precise Zero-Shot Dense Retrieval without Relevance Labels
Gao, L., Ma, X., Lin, J., & Callan, J. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.99 · Cited by 362 · see citing works
Self-citationAutoregressive Entity RetrievalPreprint
Cao, N. D., Izacard, G., Riedel, S., & Petroni, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00904 · Cited by 307 · see citing works
SGPT: GPT Sentence Embeddings for Semantic SearchPreprint
Muennighoff, N. (2022) · arXiv.org · doi:10.48550/arxiv.2202.08904 · Cited by 285 · see citing works
WebGPT: Browser-assisted question-answering with human feedback
Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.09332 · Cited by 272 · see citing works
Large Language Models for Information Retrieval: A Survey
Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., & Wen, J.-R. (2026) · ACM Transactions on Information Systems · doi:10.1145/3748304 · Cited by 225 · see citing works
Self-citationQuery2doc: Query Expansion with Large Language Models
Wang, L., Yang, N., & Wei, F. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.585 · Cited by 214 · see citing works
Pretrained Transformers for Text Ranking: BERT and Beyond
Yates, A., Nogueira, R. F., & Lin, J. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441667 · Cited by 183 · see citing works
The LambdaLoss Framework for Ranking Metric Optimization
Wang, X., Li, C., Golbandi, N., Bendersky, M., & Najork, M. (2018) · Proceedings of the 27th ACM International Conference on Information and Knowledge Management · doi:10.1145/3269206.3271784 · Cited by 136 · see citing works
Transformer Memory as a Differentiable Search Index
Tay, Y., Tran, V., Dehghani, M., Ni, J., Bahri, D., Mehta, H., Qin, Z., Hui, K., Zhao, Z., Gupta, J., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1587 · Cited by 128 · see citing works
Teaching Small Language Models to Reason
Magister, L. C., Mallinson, J., Adamek, J., Malmi, E., & Severyn, A. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2023.acl-short.151 · Cited by 127 · see citing works
mMARCO: A Multilingual Version of the MS MARCO Passage Ranking DatasetPreprint
Bonifacio, L., Jeronymo, V., Abonizio, H. Q., Campiotti, I., Fadaee, M., Lotufo, R., & Nogueira, R. F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.13897 · Cited by 126 · see citing works
Overview of the TREC 2020 Deep Learning Track
Craswell, N., Mitra, B., Yilmaz, E., & Campos, D. (2021) · arXiv · doi:10.48550/arxiv.2102.07662 · Cited by 107 · see citing works
An Analysis of the Softmax Cross Entropy Loss for Learning-to-Rank with Binary Relevance
Bruch, S., Wang, X., Bendersky, M., & Najork, M. (2019) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3341981.3344221 · Cited by 96 · see citing works
Generate rather than Retrieve: Large Language Models are Strong Context Generators
Yu, W., Iter, D., Wang, S., Xu, Y., Ju, M., Sanyal, S., Zhu, C., Zeng, M., & Jiang, M. (2022) · arXiv · doi:10.48550/arxiv.2209.10063 · Cited by 95 · see citing works
Self-citationImproving Passage Retrieval with Zero-Shot Question Generation
Sachan, D., Lewis, M., Joshi, M., Aghajanyan, A., Yih, W.-T., Pineau, J., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.249 · Cited by 90 · see citing works
Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval
Zhang, X., Ma, X., Shi, P., & Lin, J. (2021) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2021.mrl-1.12 · Cited by 65 · see citing works
Self-citationREPLUG: Retrieval-augmented black-box language models
Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., & Yih, W.-T. (2023) · arXiv · doi:10.48550/arxiv.2301.12652 · Cited by 65 · see citing works
Shallow pooling for sparse labels
Arabzadeh, N., Vtyurina, A., Yan, X., & Clarke, C. L. A. (2022) · Information Retrieval · doi:10.1007/s10791-022-09411-0 · Cited by 54 · see citing works
Specializing Smaller Language Models towards Multi-Step ReasoningPreprint
Fu, Y., Peng, H. M., Ou, L., Sabharwal, A., & Khot, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2301.12726 · Cited by 52 · see citing works
Promptagator: Few-shot Dense Retrieval From 8 Examples
Dai, Z., Zhao, V. Y., Ma, J., Luan, Y., Ni, J., Lu, J., Bakalov, A., Guu, K., Hall, K. B., & Chang, M.-W. (2022) · arXiv · doi:10.48550/arxiv.2209.11755 · Cited by 43 · see citing works
Questions Are All You Need to Train a Dense Passage Retriever
Sachan, D. S., Lewis, M., Yogatama, D., Zettlemoyer, L., Pineau, J., & Zaheer, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00564 · Cited by 35 · see citing works
Learning to Tokenize for Generative Retrieval
Sun, W., Yin, D., Chen, Z., Wang, S., Zhu, H., Ren, P., Chen, Z., Yin, D., de Rijke, M., & Ren, Z. (2023) · neural information processing systems · doi:10.52202/075280-2010 · Cited by 34 · see citing works
Self-citationZero-Shot Listwise Document Reranking with a Large Language ModelPreprint
Ma, X., Zhang, X., Pradeep, R., & Lin, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.02156 · Cited by 23 · see citing works
Self-citationInPars: Data Augmentation for Information Retrieval using Large Language ModelsPreprint
Bonifacio, L., Abonizio, H., Fadaee, M., & Nogueira, R. F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.05144 · Cited by 21 · see citing works
Pre-training Methods in Information Retrieval
Fan, Y., Xie, X., Cai, Y., Chen, J., Ma, X., Li, X., Zhang, R., & Guo, J. (2022) · doi:10.1561/9781638280637 · Cited by 18 · see citing works
Self-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Flagged first, then by citations
3 of the 39 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- TMLR.
- Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane A. Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Few-shot learning with retrieval augmented language models. Journal of Machine Learning Re- search, 24(251):1-43.
- Nandan Thakur, Nils Reimers, Andreas Ruckl'e, Ab- hishek Srivastava, and Iryna Gurevych. 2021. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. In NeurIPS 2021.
- Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023b. Self-instruct: Aligning language model with self generated instructions. In ACL 2023.
- Lixin Zou, Shengqiang Zhang, Hengyi Cai, Dehong Ma, Suqi Cheng, Daiting Shi, Shuaiqiang Wang, Zhicong Cheng, and Dawei Yin. 2021. Pre-trained language model based ranking in baidu search. In KDD 2021. Method DL19 DL20 Covid NFCorpus Touche DBPedia SciFact Signal News Robust04 BEIR (Avg) BM25 50.58 47.96 59.47 30.75 44.22 31.80 67.89 33.05 39.52 40.70 43.42 Supervised train on MS MRACO monoBERT (340M) 70.50 67.28 70.01 36.88 31.75 41.87 71.36 31.44 44.62 49.35 47.16 monoT5 (220M) 71.48 66.99 78.34 37.38 30.82 42.42 73.40 31.67 46.83 51.72 49.07 monoT5 (3B) 71.83 68.89 80.71 38.97 32.41 44.45 76.57 32.55 48.49 56.71 51.36 Cohere Rerank-v2 73.22 67.08 81.81 36.36 32.51 42.51 74.44 29.60 47.59 50.78 49.45 Unsupervised instructional permutation generation ChatGPT 65.80 62.91 76.67 35.62 36.18 44.47 70.43 32.12 48.85 50.62 49.37 GPT-4 75.59 70.56 85.51 38.47 38.57 47.12 74.95 34.40 52.89 57.55 53.68 Specialized Models train on MARCO labels or ChatGPT predicted permutations MARCO Pointwise BCE 65.57 56.72 70.82 33.10 17.08 32.28 55.37 19.30 41.52 46.00 39.43 MARCO Listwise CE 65.99 57.97 66.31 32.61 20.15 30.79 37.57 18.09 38.11 39.93 35.45 MARCO RankNet 66.34 58.51 70.29 34.23 20.27 29.62 49.01 23.22 39.82 43.87 38.79 MARCO LambdaLoss 64.82 56.16 72.86 34.20 19.51 32.55 51.88 26.22 42.47 45.28 40.62 ChatGPT Listwise CE 65.39 58.80 76.29 35.73 38.19 40.24 64.49 31.37 47.61 48.00 47.74 ChatGPT RankNet 65.75 59.34 81.26 36.57 39.03 42.10 68.77 31.55 52.54 52.44 50.53 ChatGPT LambdaLoss 67.17 60.56 80.63 36.74 36.73 43.75 68.21 32.58 49.00 50.51 49.77 deberta-v3-xsmall (70M) 64.75 55.07 78.21 35.95 35.42 41.37 67.86 30.04 47.68 49.91 48.31 deberta-v3-small (142M) 67.85 58.84 78.88 36.55 36.16 40.99 66.66 30.29 49.17 49.73 48.55 deberta-v3-base (184M) 70.28 62.52 80.81 36.15 37.25 44.06 71.70 32.45 50.84 51.33 50.57 deberta-v3-large (435M) 70.66 67.15 84.64 38.48 39.27 47.36 74.18 32.53 51.19 56.55 53.03 deberta-v3-large 5K 70.93 64.32 84.43 38.66 40.72 46.28 73.88 31.93 52.24 55.89 53.00 deberta-v3-large 3K 70.79 63.91 84.21 38.73 39.83 45.74 74.41 31.92 52.29 57.42 53.07 deberta-v3-large 1K 69.90 64.81 83.38 38.94 36.65 44.46 71.96 30.19 50.73 53.74 51.26 deberta-v3-large 500 69.71 62.00 83.54 37.23 33.68 44.56 70.48 28.70 45.64 42.67 48.31 deberta-v3-large label 10K 66.61 57.26 74.36 33.94 18.09 34.95 35.35 21.38 39.00 44.94 37.75 deberta-v3-large label 5K 68.98 61.38 80.73 35.68 20.48 37.34 54.63 24.25 36.94 51.13 42.64 deberta-v3-large label 3K 67.41 60.42 79.82 35.49 24.54 37.39 47.31 23.29 39.87 50.65 42.29 deberta-v3-large label 1K 65.55 60.93 77.70 33.29 23.36 36.38 31.10 21.71 34.28 38.31 37.01 deberta-v3-large label 500 60.59 54.45 76.20 32.93 19.66 31.54 45.66 13.99 33.48 44.49 37.24 deberta-v3-large monoT5-3B 73.05 68.82 84.78 38.55 34.43 43.61 75.45 30.75 49.85 56.80 51.78 deberta-v3-large chatgpt+label 72.42 67.30 85.96 38.75 35.06 45.43 71.81 28.52 45.91 55.57 50.88 deberta-v3-base label 10k 65.66 59.84 71.63 34.65 16.53 32.59 34.65 22.64 37.60 44.02 36.79 deberta-v3-small label 10k 63.63 52.83 68.17 30.48 18.12 31.72 33.62 18.02 34.57 36.09 33.85 deberta-v3-xsmall label 10k 60.89 51.15 63.58 28.67 14.87 27.12 20.60 18.97 32.61 32.67 29.89 llama-7b 71.33 66.06 78.23 37.60 34.87 45.46 76.13 34.17 51.79 55.22 51.68 vicuna-7b 71.80 66.89 78.32 36.87 31.81 45.40 74.23 34.28 51.13 52.91 50.62 llama-7b 10k label 65.22 56.85 75.36 36.24 20.88 37.34 69.04 25.22 41.21 49.21 44.31 llama-7b 5k label 69.24 58.97 80.49 37.55 28.23 39.66 71.79 26.04 44.09 53.83 47.71
Cited by
Sorted by citations
A survey on large language models for recommendation
Wu, L., Zheng, Z., Qiu, Z., Wang, H., Gu, H., Shen, T., Qin, C., Zhu, C., Zhu, H., Liu, Q., et al. (2024) · World Wide Web · doi:10.1007/s11280-024-01291-2 · Cited by 515 · see citing works
How Can Recommender Systems Benefit from Large Language Models: A Survey
Lin, J., Dai, X., Xi, Y., Liu, W., Chen, B., Zhang, H., Liu, Y., Wu, C., Li, X., Zhu, C., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3678004 · Cited by 242 · see citing works
Large Language Models for Information Retrieval: A Survey
Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., & Wen, J.-R. (2026) · ACM Transactions on Information Systems · doi:10.1145/3748304 · Cited by 225 · see citing works
Self-citationDense Text Retrieval Based on Pretrained Language Models: A Survey
Zhao, W. X., Liu, J., Ren, R., & Wen, J.-R. (2024) · ACM Transactions on Information Systems · doi:10.1145/3637870 · Cited by 211 · see citing works
Uncovering ChatGPT’s Capabilities in Recommender Systems
Dai, S., Shao, N., Zhao, H., Yu, W., Si, Z., Xu, C., Sun, Z., Zhang, X., & Xu, J. (2023) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/3604915.3610646 · Cited by 239 · see citing works
Gallent Torres, C., Zapata González, A., & Ortego Hernando, J. L. (2023) · RELIEVE - Revista Electrónica de Investigación y Evaluación Educativa · doi:10.30827/relieve.v29i2.29134 · Cited by 170 · see citing works
Data-efficient Fine-tuning for LLM-based Recommendation
Lin, X., Wang, W., Li, Y., Yang, S., Feng, F., Wei, Y., & Chua, T. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657807 · Cited by 181 · see citing works
Data-efficient Fine-tuning for LLM-based Recommendation
Lin, X., Wang, W., Li, Y., Yang, S., Feng, F., Wei, Y., & Chua, T.-S. (2024) · arXiv · doi:10.48550/arxiv.2401.17197 · Cited by 165 · see citing works
Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges
Wang, J., Huang, J. X., Tu, X., Wang, J., Huang, A. J., Laskar, M. T. R., & Bhuiyan, A. (2024) · ACM Computing Surveys · doi:10.1145/3648471 · Cited by 142 · see citing works
KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion
Wei, Y., Huang, Q., Zhang, Y., & Kwok, J. T.-Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.580 · Cited by 96 · see citing works
Benchmarking Cognitive Biases in Large Language Models as Evaluators
Koo, R., Lee, M., Raheja, V., Park, J. I., Kim, Z. M., & Kang, D. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.29 · Cited by 70 · see citing works
When Search Engine Services Meet Large Language Models: Visions and Challenges
Xiong, H., Bian, J., Li, Y., Li, X., Du, M., Wang, S., Yin, D., & Helal, S. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3451185 · Cited by 82 · see citing works
Self-citationAccuracy and Political Bias of News Source Credibility Ratings by Large Language Models
Yang, K.-C., & Menczer, F. (2025) · Proceedings of the 17th ACM Web Science Conference 2025 · doi:10.1145/3717867.3717903 · Cited by 81 · see citing works
Evaluating Generative Ad Hoc Information Retrieval
Gienapp, L., Scells, H., Deckers, N., Bevendorff, J., Wang, S., Kiesel, J., Syed, S., Fröbe, M., Zuccon, G., Stein, B., et al. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657849 · Cited by 51 · see citing works
Self-citationEnhancing Recommendation Diversity by Re-ranking with Large Language Models
Carraro, D., & Bridge, D. (2024) · ACM Transactions on Recommender Systems · doi:10.1145/3700604 · Cited by 46 · see citing works
Abdollahi, M., Yeganli, S. F., Baharloo, M. M., & Baniasadi, A. (2024) · Electronics · doi:10.3390/electronics14010120 · Cited by 47 · see citing works
Patel, L., Jha, S., Pan, M., Gupta, H., Asawa, P., Guestrin, C., & Zaharia, M. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3749646.3749685 · Cited by 23 · see citing works
Know where to go: Make LLM a relevant, responsible, and trustworthy searchers
Shi, X. D., Liu, J., Liu, Y., Cheng, Q., & Lu, W. P. (2024) · Decision Support Systems · doi:10.1016/j.dss.2024.114354 · Cited by 31 · see citing works
Found in the middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Hsieh, C.-Y., Chuang, Y.-S., Li, C.-L., Wang, Z., Le, L., Kumar, A., Glass, J., Ratner, A., Lee, C.-Y., Krishna, R., et al. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.890 · Cited by 31 · see citing works
Multimodal Misinformation Detection using Large Vision-Language Models
Tahmasebi, S., Müller-Budack, E., & Ewerth, R. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679826 · Cited by 29 · see citing works
Ranked List Truncation for Large Language Model-based Re-Ranking
Meng, C., Arabzadeh, N., Askari, A., Aliannejadi, M., & de Rijke, M. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657864 · Cited by 28 · see citing works
RecRanker: Instruction Tuning Large Language Model as Ranker for Top-k Recommendation
Luo, S., He, B., Zhao, H., Shao, W., Qi, Y., Huang, Y., Zhou, A., Yao, Y., Li, Z., Xiao, Y., et al. (2024) · ACM Transactions on Information Systems · doi:10.1145/3705728 · Cited by 43 · see citing works
LLMCDSR: Enhancing Cross-Domain Sequential Recommendation with Large Language Models
Xin, H., Sun, Y., Wang, C., & Xiong, H. (2025) · ACM Transactions on Information Systems · doi:10.1145/3715099 · Cited by 27 · see citing works
Vizniuk, A., Diachenko, G., Laktionov, І., Siwocha, A., Xiao, M., & Smoląg, J. (2025) · Journal of Artificial Intelligence and Soft Computing Research · doi:10.2478/jaiscr-2025-0007 · Cited by 32 · see citing works
Can Large Language Models Understand Real-World Complex Instructions?
He, Q., Zeng, J., Huang, W., Chen, L., Xiao, J., He, Q., Zhou, X., Liang, J., & Xiao, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29777 · Cited by 24 · see citing works
Query Performance Prediction Using Relevance Judgments Generated by Large Language Models
Meng, C., Arabzadeh, N., Askari, A., Aliannejadi, M., & de Rijke, M. (2025) · ACM Transactions on Information Systems · doi:10.1145/3736402 · Cited by 49 · see citing works
Balog, K., Metzler, D., & Qin, Z. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730348 · Cited by 26 · see citing works
Federated Recommendation via Hybrid Retrieval Augmented Generation
Zeng, H., Yue, Z., Jiang, Q., & Wang, D. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825302 · Cited by 22 · see citing works
Leveraging LLMs for Unsupervised Dense Retriever Ranking
Khramtsova, E., Zhuang, S., Baktashmotlagh, M., & Zuccon, G. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657798 · Cited by 23 · see citing works
A survey on large language models with multilingualism: recent advances and new frontiers
Huang, K., Mo, F., Zhang, X., Li, H., Li, Y., Zhang, Y., yi, W., Mao, Y., Liu, J., Xu, Y., et al. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11534-5 · Cited by 73 · see citing works
Embedding Search for Quranic Texts based on Large Language Models
Alqarni, M. (2024) · The International Arab Journal of Information Technology · doi:10.34028/iajit/21/2/7 · Cited by 17 · see citing works
Ruksakulpiwat, S., Phianhasin, L., Benjasirisan, C., Ding, K., Ajibade, A., Kumar, A., & Stewart, C. (2024) · JMIR mHealth and uHealth · doi:10.2196/51526 · Cited by 17 · see citing works
Multi-granular Adversarial Attacks against Black-box Neural Ranking Models
Liu, Y.-A., Zhang, R., Guo, J., de Rijke, M., Fan, Y., & Cheng, X. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657704 · Cited by 19 · see citing works
Sabbagh, V. B., Lima, C. B. D. C., & Xexeo, G. B. (2024) · SPE Journal · doi:10.2118/223612-pa · Cited by 16 · see citing works
An Investigation of Prompt Variations for Zero-Shot LLM-Based Rankers
Sun, S., Zhuang, S., Wang, S., & Zuccon, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_12 · Cited by 18 · see citing works
Self-citationLLM-enhanced multimodal fusion for cross-domain sequential recommendation
Wu, W., Chen, Z. H., Zhang, W., Song, S., Qiu, X., Huang, X., Ma, F., & Xiao, J. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132228 · Cited by 16 · see citing works
Self-citationLeveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Models
Liu, Q., Wang, B., Wang, N., & Mao, J. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714554 · Cited by 21 · see citing works
PaRaDe: Passage Ranking using Demonstrations with LLMs
Drozdov, A., Zhuang, H., Dai, Z., Qin, Z., Rahimi, R., Wang, X., Alon, D., Iyyer, M., McCallum, A., Metzler, D., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.950 · Cited by 13 · see citing works
Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers?
Li, M., Zhuang, H., Hui, K., Qin, Z., Lin, J., Jagerman, R., Wang, X., & Bendersky, M. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657979 · Cited by 17 · see citing works
Song, Z., Hwang, G.-Y., Zhang, X., Huang, S., & Park, B.-K. (2025) · Scientific Reports · doi:10.1038/s41598-025-85715-7 · Cited by 16 · see citing works
CTRL: Connect Collaborative and Language Model for CTR Prediction
Li, X., Chen, B., Hou, L., & Tang, R. (2025) · ACM Transactions on Recommender Systems · doi:10.1145/3713080 · Cited by 13 · see citing works
LitSearch: A Retrieval Benchmark for Scientific Literature Search
Ajith, A., Xia, M., Chevalier, A., Goyal, T., Chen, D., & Gao, T. (2024) · arXiv · doi:10.48550/arxiv.2407.18940 · Cited by 13
"In-Context Learning" or: How I learned to stop worrying and love "Applied Information Retrieval"
Parry, A., Ganguly, D., & Chandra, M. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657842 · Cited by 14 · see citing works
Large Language Model Powered Agents for Information Retrieval
Zhang, A., Deng, Y., Lin, Y., Chen, X., Wen, J.-R., & Chua, T. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3661375 · Cited by 13 · see citing works
Yi, F., Singh, A., & Tao, Z. (2024) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000101 · Cited by 12 · see citing works
Ragnarök: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track
Pradeep, R., Thakur, N., Sharifymoghaddam, S., Zhang, E., Nguyen, R., Campos, D., Craswell, N., & Lin, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_9 · Cited by 24 · see citing works
Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-ranking
Schlatt, F., Fröbe, M., Scells, H., Zhuang, S., Koopman, B., Zuccon, G., Stein, B., Potthast, M., & Hagen, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_31 · Cited by 17 · see citing works
Beyond Utility: Evaluating LLM as Recommender
Jiang, C., Wang, J., Ma, W., Clarke, C. L. A., Wang, S., Wu, C., & Zhang, M. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714759 · Cited by 12 · see citing works
Self-citationGuiding Retrieval Using LLM-Based Listwise Rankers
Rathee, M., MacAvaney, S., & Anand, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_15 · Cited by 16 · see citing works
APEER : Automatic Prompt Engineering Enhances Large Language Model Reranking
Jin, C., Peng, H.-W., Zhao, S.-Y., Wang, Z.-T., Xu, W., Han, L.-G., Zhao, J.-H., Zhong, K., Rajasekaran, S., & Metaxas, D. N. (2024) · The Web Conference · doi:10.1145/3701716.3717574 · Cited by 64 · see citing works
The Convergence of AI/ML and Cybersecurity: Advancing Digital Forensic Techniques
Iyengar, S. S., Nabavirazavi, S., Hariprasad, Y., Prasad, H. B., & Mohan, C. K. (2025) · Signals and communication technology · doi:10.1007/978-3-031-89327-8_4 · Cited by 12 · see citing works
Xiong, Y., Tu, X., & Zhao, W. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104232 · Cited by 12 · see citing works
Sifting through the Chaff: On Utilizing Execution Feedback for Ranking the Generated Code Candidates
Sun, Z., Wan, Y., Li, J., Zhang, H., Jin, Z., Li, G., & Lyu, C. (2024) · Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3691620.3695000 · Cited by 10 · see citing works
Is ChatGPT a Good Multi-Party Conversation Solver?
Tan, C.-H., Gu, J.-C., & Ling, Z.-H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.326 · Cited by 11 · see citing works
TWOLAR: A TWO-Step LLM-Augmented Distillation Method for Passage Reranking
Baldelli, D., Jiang, J., Aizawa, A., & Torroni, P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56027-9_29 · Cited by 10 · see citing works
Enhancing Biomedical Question Answering with Large Language Models
Yang, H., Li, S., & Gonçalves, T. (2024) · Information · doi:10.3390/info15080494 · Cited by 10 · see citing works
Adapting Standard Retrieval Benchmarks to Evaluate Generated Answers
Arabzadeh, N., Bigdeli, A., & Clarke, C. L. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56060-6_26 · Cited by 15 · see citing works
RankLLM: A Python Package for Reranking with LLMs
Sharifymoghaddam, S., Pradeep, R., Slavescu, A., Nguyen, R., Xu, A., Chen, Z., Zhang, Y., Chen, Y., Xian, J., & Lin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730331 · Cited by 11 · see citing works
Self-citationImproving RAG Quality for Large Language Models with Topic-Enhanced Reranking
Ampazis, N. (2024) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-63215-0_6 · Cited by 10 · see citing works
Enhancing ID-based Recommendation with Large Language Models
Chen, L., Gao, C., Du, X., Luo, H., Jin, D., Li, Y., & Wang, M. (2024) · ACM Transactions on Information Systems · doi:10.1145/3704263 · Cited by 9 · see citing works
Self-Calibrated Listwise Reranking with Large Language Models
Ren, R., Wang, Y., Zhou, K., Zhao, W. X., Wang, W., Liu, J., Wen, J.-R., & Chua, T. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714658 · Cited by 12 · see citing works
Criteria-Based LLM Relevance Judgments
Farzi, N., & Dietz, L. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744591 · Cited by 9 · see citing works
Large Language Models for Information Retrieval: Challenges and Chances
Breuer, T., Frihat, S., Fuhr, N., Lewandowski, D., Schaer, P., & Schenkel, R. (2025) · Datenbank-Spektrum · doi:10.1007/s13222-025-00503-x · Cited by 13 · see citing works
Ranking Manipulation for Conversational Search Engines
Pfrommer, S., Bai, Y., Gautam, T., & Sojoudi, S. (2024) · arXiv · doi:10.48550/arxiv.2406.03589 · Cited by 8 · see citing works
Neural Passage Quality Estimation for Static Pruning
Chang, X., Mishra, D., Macdonald, C., & MacAvaney, S. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657765 · Cited by 9 · see citing works
Lv, Z., Zhan, T., Wang, W., Lin, X., Zhang, S., Zhang, W., Li, J., Kuang, K., & Wu, F. Q. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709335 · Cited by 12 · see citing works
Data Augmentation for Sample Efficient and Robust Document Ranking
Anand, A., Leonhardt, J., Singh, J., Rudra, K., & Anand, A. (2023) · ACM Trans. Inf. Syst. · doi:10.1145/3634911 · Cited by 15 · see citing works
Schlatt, F., Fröbe, M., Scells, H., Zhuang, S., Koopman, B., Zuccon, G., Stein, B., Potthast, M., & Hagen, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_1 · Cited by 11 · see citing works
Data Imputation with Limited Data Redundancy Using Data Lakes
Yang, C., Luo, Y., Cui, C., Fan, J., Chai, C., & Tang, N. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3748191.3748200 · Cited by 6 · see citing works
Robust Neural Information Retrieval: An Adversarial and Out-of-Distribution Perspective
Liu, Y.-A., Zhang, R., Guo, J., de Rijke, M., Fan, Y., & Cheng, X. (2025) · ACM Transactions on Information Systems · doi:10.1145/3768153 · Cited by 14 · see citing works
Towards Adaptive Personalized Conversational Information Retrieval
Mo, F., Hui, Y., Tian, Y., Tan, Z., Meng, C., Su, Z., Huang, K., & Nie, J. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761255 · Cited by 7 · see citing works
Accelerating earth science discovery via multi-agent LLM systems
Pantiukhin, D., Shapkin, B., Kuznetsov, I., Jost, A. A., & Koldunov, N. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1674927 · Cited by 17 · see citing works
Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Review
Vats, A., Raja, R., Jain, V., & Chadha, A. R. (2025) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata66926.2025.11401642 · Cited by 7 · see citing works
Zhang, Y., Huang, B., Yuan, W., Jiang, Z., Peng, L., Chen, S., & Tan‐Soo, J. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104090 · Cited by 7 · see citing works
LLM4SZZ: Enhancing SZZ Algorithm with Context-Enhanced Assessment on Large Language Models
Tang, L., Liu, J., Liu, Z., Yang, X., & Bao, L. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728885 · Cited by 6 · see citing works
Do Large Language Models Favor Recent Content? A Study on Recency Bias in LLM-Based Reranking
Fang, H., Tao, S., Chen, N., Chang, K.-X., & Sakai, T. (2025) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3767695.3769493 · Cited by 6 · see citing works
Liu, Y.-A., Zhang, R., Guo, J., & de Rijke, M. (2025) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3701551.3703476 · Cited by 6 · see citing works
Rank-Without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models
Zhang, C., Hofstätter, S., Lewis, P., Tang, R., & Lin, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_15 · Cited by 6 · see citing works
LLM-enhanced representation learning for graph collaborative filtering recommendation models
Mou, D. V., Wei, Z., Ni, L., Song, N., Sun, Y., Chu, W., & Jin, B. (2025) · Journal of Intelligent Information Systems · doi:10.1007/s10844-025-00933-9 · Cited by 6 · see citing works
Leveraging RAG With Transformer for Context-Based Personalized Recommendations
Alamri, F. S., Rehman, A. U., Alghofaily, B., Ahmed, A. R., & Saleem, K. (2025) · IEEE Access · doi:10.1109/access.2025.3574073 · Cited by 8 · see citing works
Unify Graph Learning with Text: Unleashing LLM Potentials for Session Search
Wu, S., Tu, Q., Liu, H., Xu, J., Liu, Z., Zhang, G., Wang, R., Chen, X., & Yan, R. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645574 · Cited by 6 · see citing works
Decoy Effect in Search Interaction: Understanding User Behavior and Measuring System Vulnerability
Chen, N., Liu, J., Fang, H., Luo, Y., Sakai, T., & Wu, X.-M. (2024) · ACM Transactions on Information Systems · doi:10.1145/3708884 · Cited by 7 · see citing works
Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modality
Ma, X., Gao, L., Zhuang, S., Zhan, J. S., Callan, J., & Lin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730135 · Cited by 5 · see citing works
Self-citationReproducing NevIR: Negation in Neural Information Retrieval
van den Elsen, C., Barkhof, F., Nijdam, T., Lupart, S., & Aliannejadi, M. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730294 · Cited by 7 · see citing works
LLMTrajQuery: an LLM-based generative approach to semantic trajectory queries
Yu, S., Liu, J., Gao, C., Niu, W., Guo, X., Liu, H., & Xu, M. (2025) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2025.2569745 · Cited by 5 · see citing works
Chopra, A., & Azam, R. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-62495-7_12 · Cited by 5 · see citing works
Enhancing Dataset Search with Compact Data Snippets
Chen, Q., Chen, J., Zhou, X., & Cheng, G. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657837 · Cited by 5 · see citing works
Ruan, Q., Shi, D., & Bauernhansl, T. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103538 · Cited by 5 · see citing works
LLM-BL: Large Language Models are Zero-Shot Rankers for Bug Localization
Li, Z., Jiang, Z., Huang, Q., & Gu, Q. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00064 · Cited by 4 · see citing works
Applying Large Language Model For Relevance Search In Tencent
Ye, D., Liu, J., Hu, J., Fan, J., Tian, B., Liang, H., & Ma, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737193 · Cited by 4 · see citing works
Abdallah, A., Abdalla, M. I., Piryani, B., Mozafari, J., Ali, M., & Jatowt, A. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761484 · Cited by 5 · see citing works
How Do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective
Liu, Q., Duan, H., Mao, J., & Wen, J.-R. (2025) · ACM Transactions on Information Systems · doi:10.1145/3774942 · Cited by 6 · see citing works
Liu, Y.-A., Zhang, R., Guo, J., & de Rijke, M. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3661380 · Cited by 4 · see citing works
Neural Lexical Search with Learned Sparse Retrieval
Yates, A., Lassance, C., MacAvaney, S., Nguyen, T. D.-M., & Lei, Y. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698441 · Cited by 4 · see citing works
Rocchietti, G., Rulli, C., Nardini, F. M., Muntean, C. I., Perego, R., & Frieder, O. (2025) · IEEE Access · doi:10.1109/access.2025.3529741 · Cited by 4 · see citing works
Agent4Ranking: Semantic Robust Ranking via Personalized Query Rewriting Using Multi-Agent LLMs
Li, X., Su, L., Jia, P., Cheng, S., Wang, J., Yin, D., & Zhao, X. (2025) · ACM Transactions on Information Systems · doi:10.1145/3749099 · Cited by 4 · see citing works
Self-citationContextual Interaction for Argument Post Quality Assessment
Wang, Y., Chen, X., He, B., & Sun, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.645 · Cited by 4 · see citing works
Rewriting Conversational Utterances with Instructed Large Language Models
Galimzhanova, E., Muntean, C. I., Nardini, F. M., Perego, R., & Rocchietti, G. (2023) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat59888.2023.00014 · Cited by 5 · see citing works
Gupta, D., Bartels, D., & Demner‐Fushman, D. (2025) · Scientific Data · doi:10.1038/s41597-025-05233-z · Cited by 4 · see citing works
- Authors
- Sun, W., Yan, L., Ma, X., Wang, S., Ren, P., Chen, Z., Yin, D., & Ren, Z.
- Affiliations
- Shandong University (CN)
- Baidu (China) (CN)
- Leiden University (NL)
- Venue
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Published
- 2023
- Pages
- 14918-14937
- OpenAlex
- W4389523765
- Cited by
- 270
- References
- 39
- Citations by year
- 2026: 94 · 2025: 104 · 2024: 56 · 2023: 16
- FWCI
- 69.39
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Leiden Repository (Leiden University) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Shandong University
- National Natural Science Foundation of China (61972234, 62102234, 62272274, 62202271, 62072279)
- China Scholarship Council
- Natural Science Foundation of Shandong Province (ZR2021QF129)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- large language models, ChatGPT, GPT-4, re-ranking, information retrieval, passage ranking, relevance ranking, contaminated data, knowledge distillation, BEIR benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
