Cited by 176Information Retrieval and Search BehaviorRecommender Systems and Techniquesscientometrics and bibliometrics research

Best Match: New relevance search for PubMed

Open access
Fiorini, Nicolas, Canese, Kathi, Starchenko, Grisha, Kireev, Evgeny, Kim, Won, Miller, Vadim, Osipov, Maxim, Kholodov, Michael +4 more·2018·PLOS Biology

Abstract

PubMed is a free search engine for biomedical literature accessed by millions of users from around the world each day. With the rapid growth of biomedical literature-about two articles are added every minute on average-finding and retrieving the most relevant papers for a given query is increasingly challenging. We present Best Match, a new relevance search algorithm for PubMed that leverages the intelligence of our users and cutting-edge machine-learning technology as an alternative to the traditional date sort order. The Best Match algorithm is trained with past user searches with dozens of relevance-ranking signals (factors), the most important being the past usage of an article, publication date, relevance score, and type of article. This new algorithm demonstrates state-of-the-art retrieval performance in benchmarking experiments as well as an improved user experience in real-world testing (over 20% increase in user click-through rate). Since its deployment in June 2017, we have observed a significant increase (60%) in PubMed searches with relevance sort order: it now assists millions of PubMed searches each week. In this work, we hope to increase the awareness and transparency of this new relevance sort option for PubMed users, enabling them to retrieve information more effectively.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pbio.2005343

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The explosion of scientific literature, coupled with the complexity and specificity of many research questions, further adds to the challenges of systematic reviews.

    “The explosion of scientific literature, coupled with the complexity and specificity of many research questions, further adds to these challenges (Fiorini et al., 2018).”

    From Can large language models replace humans in systematic reviews? Evaluating <scp>GPT</scp> ‐4's efficacy in screening and extracting data from peer‐reviewed and grey literature in multiple languages (2024) · cites this paper as Fiorini et al., 2018

    The Introduction describes PubMed’s large biomedical literature collection and says query ambiguity and complexity can put irrelevant results first. That supports a general challenge in finding relevant research, but the paper does not connect it specifically to the conduct of systematic reviews or discuss the specificity of their research questions. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 42 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. [27]Li H. and Leckenby J. D. Internet advertising formats and effectiveness. Center for Interactive Advertising, pages 1– 31, 2004.

Cited by

Sorted by citations

Authors
Fiorini, N., Canese, K., Starchenko, G., Kireev, E., Kim, W., Miller, V., Osipov, M., Kholodov, M., Ismagilov, R., Mohan, S., Ostell, J., & Lu, Z.
Affiliations
National Institutes of Health (US)
Corresponding author
Lu, Zhiyong
Venue
PLOS Biology
Published
2018-08-28
Volume
16
Issue
8
Pages
e2005343
ISSN
  • 1544-9173 (ISSN-L)
  • 1545-7885
OpenAlex
W2889414415
Cited by
175
References
42
Citations by year
2026: 11 · 2025: 16 · 2024: 42 · 2023: 14 · 2022: 19 · 2021: 19 · 2020: 32 · 2019: 18 · 2018: 3
FWCI
27.77
Type
journal article
Language
en
Open access
Locations
  • PLoS Biology (publishedVersion, public-domain)
  • Europe PMC (PubMed Central) (publishedVersion, public-domain)
  • PubMed Central (publishedVersion, public-domain)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Figshare (submittedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
Keywords
PubMed, relevance ranking, biomedical literature retrieval, search result sorting, user search behavior, click-through rate
MeSH terms
Machine Learning, Algorithms, Humans, Publications, MEDLINE, Information Storage and Retrieval, PubMed, Data Mining, Search Engine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review