The DOI matches its Crossref record; 36 of the 39 references it cites were found in a registry or an index.
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents
Open accessAbstract
Large Language Models (LLMs) have demonstrated remarkable zero-shot generalization across various language-related tasks, including search engines.However, existing work utilizes the generative ability of LLMs for Information Retrieval (IR) rather than direct passage ranking.The discrepancy between the pretraining objectives of LLMs and the ranking objective poses another challenge.In this paper, we first investigate generative LLMs such as ChatGPT and GPT-4 for relevance ranking in IR.Surprisingly, our experiments reveal that properly instructed LLMs can deliver competitive, even superior results to state-of-the-art supervised methods on popular IR benchmarks.Furthermore, to address concerns about data contamination of LLMs, we collect a new test set called NovelEval, based on the latest knowledge and aiming to verify the model's ability to rank unknown knowledge.Finally, to improve efficiency in real-world applications, we delve into the potential for distilling the ranking capabilities of ChatGPT into small specialized models using a permutation distillation scheme.Our evaluation results turn out that a distilled 440M model outperforms a 3B supervised model on the BEIR benchmark.The code to reproduce our results is available at www.github.com/sunnweiwei/RankGPT.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Rank-biased overlap (RBO)foot_6 is a widely used metric for evaluating the similarity between two ranked lists and is particularly relevant for tasks in information retrieval (Oh et al., 2022;Sun et al., 2023).”
From Benchmarking Cognitive Biases in Large Language Models as Evaluators (2024) · cites this paper as Sun et al., 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A representative empirical anchor is Sun et al. (2023), who show that LLMs can match or exceed strong supervised re-rankers on standard IR benchmarks, validating the 'generative engine as LLM re-ranker' abstraction we build on.”
From E-GEO: A Testbed for Generative Engine Optimization in E-Commerce (2025) · cites this paper as Sun et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Dense passage retrieval for open-domain question answering
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W.-T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.550 · Cited by 3,575
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Bajaj, P., Campos, D., Craswell, N., Deng, L., Gao, J., Liu, X., Majumder, R., McNamara, A., Mitra, B., Nguyen, T., et al. (2016) · arXiv · doi:10.48550/arxiv.1611.09268 · Cited by 2,473
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Document Ranking with a Pretrained Sequence-to-Sequence Model
Nogueira, R. F., Jiang, Z., Pradeep, R., & Lin, J. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.63 · Cited by 686
Nogueira, R., & Cho, K. (2019) · arXiv · doi:10.48550/arxiv.1901.04085 · Cited by 615
Holistic Evaluation of Language Models
Bommasani, R., Liang, P., & Lee, T. (2023) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.15007 · Cited by 480
Research works printed but not linked: 5
- TMLR.
- Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane A. Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Few-shot learning with retrieval augmented language models. Journal of Machine Learning Re- search, 24(251):1-43.
- Nandan Thakur, Nils Reimers, Andreas Ruckl'e, Ab- hishek Srivastava, and Iryna Gurevych. 2021. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. In NeurIPS 2021.
- Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023b. Self-instruct: Aligning language model with self generated instructions. In ACL 2023.
- Lixin Zou, Shengqiang Zhang, Hengyi Cai, Dehong Ma, Suqi Cheng, Daiting Shi, Shuaiqiang Wang, Zhicong Cheng, and Dawei Yin. 2021. Pre-trained language model based ranking in baidu search. In KDD 2021. Method DL19 DL20 Covid NFCorpus Touche DBPedia SciFact Signal News Robust04 BEIR (Avg) BM25 50.58 47.96 59.47 30.75 44.22 31.80 67.89 33.05 39.52 40.70 43.42 Supervised train on MS MRACO monoBERT (340M) 70.50 67.28 70.01 36.88 31.75 41.87 71.36 31.44 44.62 49.35 47.16 monoT5 (220M) 71.48 66.99 78.34 37.38 30.82 42.42 73.40 31.67 46.83 51.72 49.07 monoT5 (3B) 71.83 68.89 80.71 38.97 32.41 44.45 76.57 32.55 48.49 56.71 51.36 Cohere Rerank-v2 73.22 67.08 81.81 36.36 32.51 42.51 74.44 29.60 47.59 50.78 49.45 Unsupervised instructional permutation generation ChatGPT 65.80 62.91 76.67 35.62 36.18 44.47 70.43 32.12 48.85 50.62 49.37 GPT-4 75.59 70.56 85.51 38.47 38.57 47.12 74.95 34.40 52.89 57.55 53.68 Specialized Models train on MARCO labels or ChatGPT predicted permutations MARCO Pointwise BCE 65.57 56.72 70.82 33.10 17.08 32.28 55.37 19.30 41.52 46.00 39.43 MARCO Listwise CE 65.99 57.97 66.31 32.61 20.15 30.79 37.57 18.09 38.11 39.93 35.45 MARCO RankNet 66.34 58.51 70.29 34.23 20.27 29.62 49.01 23.22 39.82 43.87 38.79 MARCO LambdaLoss 64.82 56.16 72.86 34.20 19.51 32.55 51.88 26.22 42.47 45.28 40.62 ChatGPT Listwise CE 65.39 58.80 76.29 35.73 38.19 40.24 64.49 31.37 47.61 48.00 47.74 ChatGPT RankNet 65.75 59.34 81.26 36.57 39.03 42.10 68.77 31.55 52.54 52.44 50.53 ChatGPT LambdaLoss 67.17 60.56 80.63 36.74 36.73 43.75 68.21 32.58 49.00 50.51 49.77 deberta-v3-xsmall (70M) 64.75 55.07 78.21 35.95 35.42 41.37 67.86 30.04 47.68 49.91 48.31 deberta-v3-small (142M) 67.85 58.84 78.88 36.55 36.16 40.99 66.66 30.29 49.17 49.73 48.55 deberta-v3-base (184M) 70.28 62.52 80.81 36.15 37.25 44.06 71.70 32.45 50.84 51.33 50.57 deberta-v3-large (435M) 70.66 67.15 84.64 38.48 39.27 47.36 74.18 32.53 51.19 56.55 53.03 deberta-v3-large 5K 70.93 64.32 84.43 38.66 40.72 46.28 73.88 31.93 52.24 55.89 53.00 deberta-v3-large 3K 70.79 63.91 84.21 38.73 39.83 45.74 74.41 31.92 52.29 57.42 53.07 deberta-v3-large 1K 69.90 64.81 83.38 38.94 36.65 44.46 71.96 30.19 50.73 53.74 51.26 deberta-v3-large 500 69.71 62.00 83.54 37.23 33.68 44.56 70.48 28.70 45.64 42.67 48.31 deberta-v3-large label 10K 66.61 57.26 74.36 33.94 18.09 34.95 35.35 21.38 39.00 44.94 37.75 deberta-v3-large label 5K 68.98 61.38 80.73 35.68 20.48 37.34 54.63 24.25 36.94 51.13 42.64 deberta-v3-large label 3K 67.41 60.42 79.82 35.49 24.54 37.39 47.31 23.29 39.87 50.65 42.29 deberta-v3-large label 1K 65.55 60.93 77.70 33.29 23.36 36.38 31.10 21.71 34.28 38.31 37.01 deberta-v3-large label 500 60.59 54.45 76.20 32.93 19.66 31.54 45.66 13.99 33.48 44.49 37.24 deberta-v3-large monoT5-3B 73.05 68.82 84.78 38.55 34.43 43.61 75.45 30.75 49.85 56.80 51.78 deberta-v3-large chatgpt+label 72.42 67.30 85.96 38.75 35.06 45.43 71.81 28.52 45.91 55.57 50.88 deberta-v3-base label 10k 65.66 59.84 71.63 34.65 16.53 32.59 34.65 22.64 37.60 44.02 36.79 deberta-v3-small label 10k 63.63 52.83 68.17 30.48 18.12 31.72 33.62 18.02 34.57 36.09 33.85 deberta-v3-xsmall label 10k 60.89 51.15 63.58 28.67 14.87 27.12 20.60 18.97 32.61 32.67 29.89 llama-7b 71.33 66.06 78.23 37.60 34.87 45.46 76.13 34.17 51.79 55.22 51.68 vicuna-7b 71.80 66.89 78.32 36.87 31.81 45.40 74.23 34.28 51.13 52.91 50.62 llama-7b 10k label 65.22 56.85 75.36 36.24 20.88 37.34 69.04 25.22 41.21 49.21 44.31 llama-7b 5k label 69.24 58.97 80.49 37.55 28.23 39.66 71.79 26.04 44.09 53.83 47.71
Cited by
Sorted by citations
Items 101–200 of 282
Gupta, D., Bartels, D., & Demner‐Fushman, D. (2025) · Scientific Data · doi:10.1038/s41597-025-05233-z · Cited by 9
PR-KGC: Text-enhanced Knowledge Graph Completion with Pair-wise Re-ranking
Gao, M., Zhang, F., Chen, W., Wang, T., Zheng, J., & Yao, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889671 · Cited by 3
Movin, M., & Hauff, C. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730119 · Cited by 3
Variations in Relevance Judgments and the Shelf Life of Test Collections
Parry, A., Fröbe, M., Scells, H., Schlatt, F., Faggioli, G., Zerhoudi, S., MacAvaney, S., & Yang, E. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730308 · Cited by 4
Thinking Machines: Mathematical Reasoning in the Age of LLMs
Asperti, A., Naibo, A., & Coen, C. S. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10010038 · Cited by 10
Leveraging Decoder Architectures for Learned Sparse Retrieval
Qiao, J., Nguyen, T. D.-M., Kanoulas, E., & Yates, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02899-0_2 · Cited by 3
Evaluating the Universality of “Do Anything Now” Jailbreak Prompts on Large Language Models
Nabavirazavi, S., Zad, S., & Iyengar, S. S. (2025) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc62904.2025.10903781 · Cited by 3
Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning
Zhuang, S., Ma, X., Yao, Z., Wang, S., Koopman, B., Lin, J., & Zuccon, G. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809961 · Cited by 98
Self-citationWang, S., Dai, S., & Jiang, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70352-2_20 · Cited by 3
Self-citationAnswering Ambiguous Questions via Iterative Prompting
Sun, W., Cai, H., Chen, H., Ren, P., Chen, Z., de Rijke, M., & Ren, Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.424 · Cited by 7
Self-citationCho, S., Seo, J., Jeong, S., & Park, J. C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.207 · Cited by 7
TalkSee: Interactive Video Retrieval Engine Using Large Language Model
Gu, G., Wu, Z., He, J., Song, L., Wang, Z., & Liang, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-53302-0_36 · Cited by 3
Can Large Language Models Assess Serendipity in Recommender Systems?
Tokutake, Y., Editorial Office, & Okamoto, K. (2024) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2024.p1263 · Cited by 3
An in-depth analysis of passage-level label transfer for contextual document ranking
Rudra, K., Fernando, Z. T., & Anand, A. (2023) · Information Retrieval · doi:10.1007/s10791-023-09430-5 · Cited by 4
BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents
Chen, Z., Ma, X., Zhuang, S., Nie, P., Zou, K., Sharifymoghaddam, S., Liu, A., Green, J., Patel, K., Meng, R., et al. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.1023 · Cited by 2
Self-citationLee, J., Lee, C., & Rew, J. (2024) · Journal of Broadcast Engineering · doi:10.5909/jbe.2024.29.5.581 · Cited by 2
Large Language Models as Narrative-Driven Recommenders
Eberhard, L., Ruprechter, T., & Helić, D. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714668 · Cited by 4
Guo, S., Liu, H., Chen, X., Xie, Y., Zhang, L., Han, T., Chen, H., Chang, Y., & Wang, Y.-X. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737254 · Cited by 3
Shan, R., Zhu, J., Lin, J., Zhu, C., Chen, B., Tang, R., Yu, Y., & Zhang, W. (2025) · ACM Transactions on Recommender Systems · doi:10.1145/3766553 · Cited by 2
OpenDecoder: Open Large Language Model Decoding to Incorporate Document Quality in RAG
Mo, F., Su, Z., Hui, Y., Zhang, J., Sun, J. A., Liu, Z., Zhang, C., Sakai, T., & Nie, J. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792524 · Cited by 19
Li, J., Hu, J., & Zhang, G. (2024) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2024.050005 · Cited by 3
Specialized model initialization and architecture optimization for few-shot code search
Zhang, F., Wu, Q., Peng, M., & Shen, Y. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107571 · Cited by 2
DDASR: Deep Diverse API Sequence Recommendation
Nan, S., Wang, J., Zhang, N., Li, D., & Li, B. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712188 · Cited by 2
Assessing Generalization Capability of Text Ranking Models in Polish
Dadas, S., & Grębowiec, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-84353-2_4 · Cited by 4
Exploring the Effectiveness of Multi-stage Fine-Tuning for Cross-Encoder Re-rankers
Pezzuti, F., MacAvaney, S., & Tonellotto, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_45 · Cited by 2
Zero-Shot Response-Aware Query Expansion Method for Conversational Retrieval
Wang, J., Chen, X., He, P., Zhang, F., Wang, L., & Sheng, J. (2024) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat62293.2024.00101 · Cited by 2
RankFlow: A Multi-Role Collaborative Reranking Workflow Utilizing Large Language Models
Jin, C., Peng, H., Zhang, A., Chen, N., Zhao, J., Xie, X., Li, K., Feng, S., Zhong, K.-W., Ding, C., et al. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3717575 · Cited by 4
Deng, X., Wang, X., & Stevenson, M. (2025) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3731120.3744614 · Cited by 5
Susanti, Y., & Färber, M. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737076 · Cited by 4
E-GEO: A Testbed for Generative Engine Optimization in E-Commerce
Bagga, P. S., Farias, V. F., Korkotashvili, T., Peng, T., & Wu, Y. (2025) · arXiv · doi:10.48550/arxiv.2511.20867 · Cited by 1
Cao, R., Zhou, X., & Zhou, G. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9431-7_13 · Cited by 1
Improving Generative Information Retrieval Systems Based on User Feedback
Ai, Q., Dou, Z., & Zhang, M. (2024) · The information retrieval series · doi:10.1007/978-3-031-73147-1_5 · Cited by 1
ContextualCoder: Adaptive In-Context Prompting for Programmatic Visual Question Answering
Shen, R., Inoue, N., Guan, D., Cai, R., Kot, A. C., & Shinoda, K. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3543043 · Cited by 1
Hosseini, K., Kober, T., Krapac, J., Vollgraf, R., Cheng, W., & Ramallo, A. P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_10 · Cited by 7
Ranking method for an academic paper search with an emphasis on comprehensiveness
Fukuda, S., & Tomiura, Y. (2025) · Discover Computing · doi:10.1007/s10791-025-09527-z · Cited by 1
Accelerating Listwise Reranking: Reproducing and Enhancing FIRST
Chen, Z. Y., Pradeep, R., & Lin, J. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730287 · Cited by 4
Self-citationLLM as User Simulator: Towards Training News Recommender without Real User Interactions
Park, C. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730224 · Cited by 1
A Reproducibility Study of LLM Setwise Reranker with Heapsort
Lawrie, D., Kayi, E. S., Mayfield, J., Yang, E., Yates, A., & Oard, D. W. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730338 · Cited by 1
Verdú, G., Calvera-Rayo, A., & Bolufer, A. B. F. (2025) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfaf098 · Cited by 1
Can LLMs Really Help Query Understanding In Web Search? A Practical Perspective
Ye, D., Qin, Y., Tian, B., Fan, J., Liu, J., Liang, H., & Ma, J. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3760842 · Cited by 3
+VeriRel: Verification Feedback to Enhance Document Retrieval for Scientific Fact Checking
Deng, X., Wang, X., & Stevenson, M. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3760822 · Cited by 2
Li, Z., Zhang, H., & Zhang, J. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228011 · Cited by 1
Liu, Z., Huang, P., Xu, Z., Li, X., Liu, S., Peng, C., Xin, H., Yan, Y., Wang, S., Han, X., et al. (2026) · AI Open · doi:10.1016/j.aiopen.2026.02.002 · Cited by 1
Self-citationLegal Case Entailment via Optimal Transport–Enhanced Retrieval and Large Language Model Ranking
Tran, T. N., Nguyen, P. M., & Nguyen, M. L. (2026) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3789254 · Cited by 1
Khalid, S. M., Wölker, T., Molano, L.-A. G., Graf, S., & Keller, A. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag089 · Cited by 1
From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews
Nogueira, B., Geyer, W., Anderson, A. W., Li, T. J.-J., Kim, D., Moniz, N., & Chawla, N. V. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794723 · Cited by 1
Neural Lexical Search with Learned Sparse Retrieval
Yates, A., Lassance, C., Rulli, C., Yang, E., MacAvaney, S., Singh, S. A. K., Nguyen, T. D.-M., & Lei, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_5 · Cited by 1
XProvence: Zero-Cost Multilingual Context Pruning for Retrieval-Augmented Generation
Mohamed, Y. H., Elhoseiny, M., Formal, T., & Chirkova, N. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21300-6_38 · Cited by 2
Hierarchical context enhancement for long-tail entity retrieval augmented generation
Peng, Y., & Pan, K. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1861536 · Cited by 1
Tian, Y., Li, W., Chen, X., Brook, M., Brubaker, M., Liljedahl, A., & Baral, C. R. (2026) · ACM Transactions on Spatial Algorithms and Systems · doi:10.1145/3828544 · Cited by 1
Salami, M. O., Fang, L., & Torvik, V. I. (2026) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3805696.3846065 · Cited by 1
Evaluating the Accuracy of ChatGPT in the Japanese Board-Certified Physiatrist Examination
Kato, Y., Ushida, K., & Momosaki, R. (2024) · Cureus · doi:10.7759/cureus.76214 · Cited by 1
Augmentation with Neighboring Information for Conversational Recommendation
Liu, Y., Pei, J., Zhang, W., Li, M., Che, W., & de Rijke, M. (2025) · ACM Transactions on Information Systems · doi:10.1145/3712588 · Cited by 1
Efficient Integration of ASR with Large Language Models to Enhance Video Search at Scale
Zhang, Q., Xiao, F., Li, T., Lin, L., Fang, H., Sun, H., Liu, R., Zhu, X., & Wang, J. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3715220 · Cited by 1
Wang, J., Zhang, F., Chen, X., He, P., Huang, E. A., & Huang, J. X. (2025) · ACM Transactions on Information Systems · doi:10.1145/3742423 · Cited by 1
In-Context Learning in LLMs to Improve Retrieval Models
Sinhababu, N. (2024) · Annual Meeting of the Forum for Information Retrieval Evaluation (FIRE) · doi:10.1145/3734947.3734957 · Cited by 1
DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task
Liu, W., Zhu, Y., Dou, Z., & Zhou, Y. (2026) · ACM Transactions on Information Systems · doi:10.1145/3788867 · Cited by 20
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. M. (2023) · Preprints.org · doi:10.20944/preprints202306.0363.v1
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. (2024) · Preprints.org · doi:10.20944/preprints202306.0363.v5 · Cited by 1
Ong, K. R., & Wong, W. P. (2025) · Research Square · doi:10.21203/rs.3.rs-7506627/v1 · Cited by 2
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Hsieh, C.-Y., Chuang, Y.-S., Li, C.-L., Wang, Z., Le, L. T., Kumar, A., Glass, J., Ratner, A., Lee, C.-Y., Krishna, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.16008
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. M. (2023) · Preprints.org · doi:10.20944/preprints202306.0363.v2
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. M. (2023) · Preprints.org · doi:10.20944/preprints202306.0363.v3
Withdrawn: Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. M. (2023) · Preprints.org · doi:10.20944/preprints202306.0363.v4
Alqarni, M. (2024) · The International Arab Journal of Information Technology · doi:10.34028/21/2/7
Artificial Intelligence: A Kalam and Sufi PerspectivePreprint
Khalili, A. (2024) · Preprints.org · doi:10.20944/preprints202306.0363.v6
Li, M., Li, X., Hu, M., & Yuan, D. (2024) · Entropy · doi:10.3390/e26040321
Xu, W. X., & Jin, X. (2024) · Frontiers in Computing and Intelligent Systems · doi:10.54097/wy0g8m89 · Cited by 1
DUNKS: Chunking and Summarizing Large and Heterogeneous Data for Dataset Search
Chen, Q., Zhou, X., Zhang, Z., & Cheng, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77850-6_5
Zero-Shot Audio Topic Reranking Using Large Language Models
Qian, M., Ma, R., Liusie, A., Loweimi, E., Knill, K., & Gales, M. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832363 · Cited by 3
Keyword-Enhanced Semantic Retrieval and Multi-Dimensional Relevance Ranking in RAG
Zhao, K., & Wang, G. (2024) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc62609.2024.10941711
Can Large Language Models Effectively Rerank News Articles for Background Linking?
Essam, M., & Elsayed, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88711-6_21
Evaluating Auto-complete Ranking for Diversity and Relevance
Singh, S., Farfade, S., & Comar, P. M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_21
LiT and Lean: Distilling Listwise Rerankers Into Encoder-Decoder Models
Tamber, M. S., Pradeep, R., & Lin, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_13
Agent-Based ChatGPT for Multi-Party Conversation Task Processing
万, 静. (2025) · Software Engineering and Applications · doi:10.12677/sea.2025.142022
ChartCitor: Answer Citations for ChartQA via Multi-Agent LLM Retrieval
Goswami, K., Mathur, P., Rossi, R. A., & Dernoncourt, F. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3701716.3716886 · Cited by 1
Aligning Web Query Generation with Ranking Objectives via Direct Preference Optimization
Coelho, J., Martins, B., Magalhães, J., & Xiong, C. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730162
Gosling Grows Up: Retrieval with Learned Dense and Sparse Representations Using Anserini
Lin, J., Chen, A. H., Lassance, C., Ma, X., Pradeep, R., Teofili, T., Xian, J., Yang, J.-H., Zhong, B., & Zhong, V. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730281 · Cited by 1
Self-citationNeural Lexical Search with Learned Sparse Retrieval
Yates, A., Lassance, C., Rulli, C., Yang, E., MacAvaney, S., Singh, S., Nguyen, T. D.-M., & Lei, Y. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3731693
Optimizing Compound Retrieval Systems
Oosterhuis, H., Jagerman, R., Qin, Z., & Wang, X. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730051
Precise Zero-Shot Pointwise Ranking with LLMs through Post-Aggregated Global Context Information
Long, K., Li, S., Xu, C., Tang, J., & Wang, T. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730061
IRA: Adaptive Interest-aware Representation and Alignment for Personalized Multi-interest Retrieval
Lee, Y., Jeong, H., Lim, C., Choi, J. D., Lim, H., Kim, H., Kwon, J., & Kim, S. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3731943 · Cited by 1
Supporting Retriever’s Training by Joint Likelihood-Based Soft-Label Generation
Peng, T., Chen, R., Xie, X. C., Lin, D. X., & Cui, Z. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9994-0_23
Alsaleh, N., Alnanih, R., & Alowidi, N. (2025) · Computers · doi:10.3390/computers14090363 · Cited by 1
An LLM-Based Active Assistant and Smart Manual for CT Imaging Workflows
Mamaghani, Z. A., Vorberg, L., Maier, A., Katzmann, A., & Taubmann, O. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-07502-4_6
Automated Advisor Matching for Student Theses Using Transformer-Based Topic Modeling
Izquierdo-Campoverde, I., Ayala-Tipan, C., Recalde, L., & Navarrete, R. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08366-1_11
RJAG: Retrieval Judgment Augmented Generation
Wang, K., Hu, Z., Ren, M., & Tao, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-2725-0_15
Anchor-based Pairwise Comparison via Large Language Model for Recommendation Reranking
Luo, Q., Chen, E., Shi, Z., & Wang, B. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3760859 · Cited by 1
DIVAgent: A Diversified Search Agent that Mimics the Human Search Process
Deng, Z., Qiao, J., Dou, Z., Wen, J.-R., & de Rijke, M. (2025) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3746252.3761059
Cognitive Support in Aviation Operations Utilizing Multimodal Large Language Models
Huang, R., Feng, S., Li, F., Chen, C., & Ong, Y.-S. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11229131
Eliminating Retrieval Knowledge Conflicts: Cross-Validation Re-ranking with Large Language Models
Wu, Q., Lin, H., Zheng, H., Xie, R., Yang, S., Sun, X., Kang, Z., & Kim, H. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228012
Modelo Conceitual de Serviço de Recomendações para TV 3.0
Poggy, A., Borges, P. V., Moraes, D. D. S., & Colcher, S. (2025) · Simpósio Brasileiro de Sistemas Multimídia e Web (WebMedia) · doi:10.5753/webmedia_estendido.2025.16811
Domain Adaptive Document Reranking for Retrieval Augmented Generation
Rachidy, Y., Hmamouche, Y., Sehbaoui, F., & Seghrouchni, A. E. F. (2025) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai66417.2025.00200
A Survey of Long-Document Retrieval in the PLM and LLM Era
Li, M., Zhang, Y., Lv, T., Zhao, S., Luo, M., Nie, E., & Zhou, G. (2026) · ACM Transactions on Information Systems · doi:10.1145/3849382 · Cited by 6
Ideuchi, M., Oida, Y., Higashiyama, S., Tanaka, H., Utiyama, M., & Watanabe, T. (2025) · Journal of Natural Language Processing · doi:10.5715/jnlp.32.918
InterQuest: A Mixed-Initiative Framework for Dynamic User Interest Modeling in Conversational Search
Mei, Y., Wang, Y., Wang, S., Wan, Q., Li, Z., Yu, C., Shi, W., & Shi, Y. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747753 · Cited by 1
Self-citationApplying Large Language Models to Search
Benedetti, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01563-1_4
Mohammadi, F., Maghool, S., & Ceravolo, P. (2025) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa66935.2025.11315321
Progressive Adaptation of Large Language Models for Multilingual Text Ranking
Zhang, L., Zhang, Y., Long, D., Xie, P., Zhang, M., Li, J., & Zhang, M. (2026) · ACM Transactions on Information Systems · doi:10.1145/3788859
Metadata
- Authors
- Sun, W., Yan, L., Ma, X., Wang, S., Ren, P., Chen, Z., Yin, D., & Ren, Z.
- Affiliations
- Shandong University (CN)
- Baidu (China) (CN)
- Leiden University (NL)
- Venue
- Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
- Published
- 2023
- Pages
- 14918-14937
- OpenAlex
- W4389523765
- Cited by
- 270
- References
- 39
- Citations by year
- 2026: 94 · 2025: 104 · 2024: 56 · 2023: 16
- FWCI
- 69.39
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Leiden Repository (Leiden University) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Shandong University
- National Natural Science Foundation of China (61972234, 62102234, 62272274, 62202271, 62072279)
- China Scholarship Council
- Natural Science Foundation of Shandong Province (ZR2021QF129)
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- large language models, ChatGPT, GPT-4, re-ranking, information retrieval, passage ranking, relevance ranking, contaminated data, knowledge distillation, BEIR benchmark
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
