MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval
Open accessAbstract
Motivation: Information retrieval (IR) is essential in biomedical knowledge acquisition and clinical decision support. While recent progress has shown that language model encoders perform better semantic retrieval, training such models requires abundant query–article annotations that are difficult to obtain in biomedicine. As a result, most biomedical IR systems only conduct lexical matching. In response, we introduce MedCPT, a first-of-its-kind Contrastively Pre-trained Transformer model for zero-shot semantic IR in biomedicine. Results: To train MedCPT, we collected an unprecedented scale of 255 million user click logs from PubMed. With such data, we use contrastive learning to train a pair of closely integrated retriever and re-ranker. Experimental results show that MedCPT sets new state-of-the-art performance on six biomedical IR tasks, outperforming various baselines including much larger models, such as GPT-3-sized cpt-text-XL. In addition, MedCPT also generates better biomedical article and sentence representations for semantic evaluations. As such, MedCPT can be readily applied to various real-world biomedical IR tasks. Availability and implementation: The MedCPT code and model are available at https://github.com/ncbi/MedCPT.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
MedCPT is a biomedical retriever designed to capture semantic relevance beyond exact keyword overlap.
“Dense retrieval was implemented using MedCPT [25], a biomedical retriever designed to capture semantic relevance beyond exact keyword overlap.”
From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [25]
MedCPT employs a bi-encoder for efficient candidate selection and a cross-encoder reranker for refined semantic alignment.
“MedCPT employs a bi-encoder for efficient candidate selection and a cross-encoder reranker for refined semantic alignment.”
From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [25]
Most existing Modular RAG systems in healthcare rely solely on either sparse (BM25) or dense (MedCPT, embedding-based) retrieval methods.
“Most existing Modular RAG systems in healthcare, such as Self-BioRAG [16] and Clinfo.ai[24], rely solely on either sparse (BM25) or dense (MedCPT [25], embedding-based) retrieval methods.”
From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [25]
MedCPT was specifically pretrained on biomedical literature using user click information.
“MedCPT was specifically pretrained on biomedical literature using user click information2.”
From Leveraging long context in retrieval augmented language models for medical question answering (2025) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 19 · Cited by 175
Research works printed but not linked: 15
- Brown T, Mann B, Ryder N et al. Language models are few-shot learn- ers. Adv Neural Inf Process Syst 2020;33:1877-901.
- Chen Q, Peng Y, Lu Z. BioSentVec: creating sentence embeddings for biomedical texts. In: 2019 IEEE International Conference on Healthcare Informatics (ICHI). Xian, China: IEEE, 2019, 1-5.
- Cohan A, Feldman S, Beltagy I et al. SPECTER: document-level repre- sentation learning using citation-informed transformers. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online, 2020, 2270-82.
- Dai Z, Callan J. Context-aware term weighting for first stage passage re- trieval. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Xian, China, 2020, 1533-6.
- Gao L, Dai Z, Callan J. Rethink training of BERT rerankers in multi- stage retrieval pipeline. In: Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28-April 1, 2021, Proceedings, Part II 43. Lucca, Italy: Springer, 2021, 280-6.
- Hofsta ¨tter S, Lin S, Yang J et al. Efficiently teaching an effective dense retriever with balanced topic aware sampling. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. Online, 2021, 113-22.
- Karpukhin V, O guz B, Min S et al. Dense passage retrieval for open- domain question answering. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online, 2020, 6769-81.
- Khattab O, Zaharia M. ColBERT: efficient and effective passage search via contextualized late interaction over BERT. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2020, 39-48.
- Ma J, Korotkov I, Hall K et al. Hybrid first-stage retrieval models for biomedical literature. In: Conference and Labs of the Evaluation Forum, Thessaloniki, Greece, 2020.
- Nogueira R, Lin J, Epistemic A. From doc2query to docTTTTTquery. 2019, Online preprint.
- Simpson M, Voorhees E, Hersh W. Overview of the TREC 2014 clinical decision support track. In: Text REtrieval Conference. Gaithersburgh, USA, 2014.
- Roberts K, Demner-Fushman D, Voorhees E et al. Overview of the TREC 2020 precision medicine track. In: The Text REtrieval Conference, Vol. 26. Gaithersburgh, USA, 2017.
- Shin A, Jin Q, Lu Z. Multi-stage literature retrieval system trained by PubMed search logs for biomedical question answering. In: Conference and Labs of the Evaluation Forum (CLEF), Thessaloniki, Greece, 2023.
- Vaswani A, Shazeer N, Parmar N et al. Attention is all you need. In: Advances in Neural Information Processing Systems, Vol. 30. Long Beach, USA, 2017.
- Voorhees E, Alam T, Bedrick S et al. TREC-COVID: constructing a pan- demic information retrieval test collection. In: ACM SIGIR Forum. New York, NY, USA: ACM, 2021, 1-12.
- Authors
- Jin, Q., Kim, W., Chen, Q., Comeau, D. C., Yeganova, L., Wilbur, W. J., & Lu, Z.
- Affiliations
- National Institutes of Health (US)
- Corresponding author
- Lu, Zhiyong
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-11-01
- Volume
- 39
- Issue
- 11
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- PMC
- PMC10627406
- OpenAlex
- W4388409299
- Cited by
- 175
- References
- 19
- Citations by year
- 2026: 85 · 2025: 59 · 2024: 28 · 2023: 3
- FWCI
- 41.77
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- public-domain
- Locations
- Bioinformatics (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- Funders
- NIH Intramural Research Program, National Library of Medicine
- Topics
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- biomedical information retrieval, contrastive learning, clinical decision support
- MeSH terms
- Language, Natural Language Processing, Review Literature as Topic, Semantics, Information Storage and Retrieval, PubMed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
