The DOI matches its Crossref record; 75 of the 80 references it cites were found in a registry or an index.
MeSH Now: automatic MeSH indexing at PubMed scale via learning to rank
Open accessAbstract
Background: MeSH indexing is the task of assigning relevant MeSH terms based on a manual reading of scholarly publications by human indexers. The task is highly important for improving literature retrieval and many other scientific investigations in biomedical research. Unfortunately, given its manual nature, the process of MeSH indexing is both time-consuming (new articles are not immediately indexed until 2 or 3 months later) and costly (approximately ten dollars per article). In response, automatic indexing by computers has been previously proposed and attempted but remains challenging. In order to advance the state of the art in automatic MeSH indexing, a community-wide shared task called BioASQ was recently organized. Methods: We propose MeSH Now, an integrated approach that first uses multiple strategies to generate a combined list of candidate MeSH terms for a target article. Through a novel learning-to-rank framework, MeSH Now then ranks the list of candidate terms based on their relevance to the target article. Finally, MeSH Now selects the highest-ranked MeSH terms via a post-processing module. Results: We assessed MeSH Now on two separate benchmarking datasets using traditional precision, recall and F 1 -score metrics. In both evaluations, MeSH Now consistently achieved over 0.60 in F-score, ranging from 0.610 to 0.612. Furthermore, additional experiments show that MeSH Now can be optimized by parallel computing in order to process MEDLINE documents on a large scale. Conclusions: We conclude that MeSH Now is a robust approach with state-of-the-art performance for automatic MeSH indexing and that MeSH Now is capable of processing PubMed scale documents within a reasonable time frame. Availability: http://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/MeSHNow/ .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, it may not provide keywords according to Medical Subject Headings, which is a criterion of the majority of the journal for better indexing of their articles.[24]”
From ChatGPT in academic writing: Maximizing its benefits and minimizing the risks (2023) · cites this paper as [24]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 2×
“LambdaMART [67] is a combination of MART and LambdaRank, where the MART algorithm can be viewed as generalizations of logistic regression [63] and LambdaRank is a method for learning arbitrary information retrieval measures [68].”
MethodRobust estimation of a location parameter
Huber, P. J. (1964) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177703732 · Cited by 7,228
The mathematics of statistical machine translation: parameter estimation
Brown, P. F., Pietra, V. J. D., Pietra, S. A. D., & Mercer, R. L. (1993) · Cited by 3,973
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., & Hullender, G. (2005) · Proceedings of the 22nd international conference on Machine learning - ICML '05 · doi:10.1145/1102351.1102363 · Cited by 2,836 · cited 1×
“In this work we evaluated several other learning-to-rank algorithms [43] on the BioASQ test dataset, including MART [63], RankNet [64], Coord-inate Ascent [65], AdaRank [66], and LambdaMART, which are available in RankLib v2.”
MethodLearning to rank: from pairwise approach to listwise approach
Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., & Li, H. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273513 · Cited by 1,960 · cited 1×
“In our previous work, we trained the ranking function with ListNet [62], which sorts the results based on a list of scores.”
MethodLearning to rank for information retrieval
Liu, T.-Y. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835676 · Cited by 1,856
Learning to Rank for Information Retrieval
Liu, T.-Y. (2009) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000016 · Cited by 1,568
AdaRank: a boosting algorithm for information retrieval
Xu, J., & Li, H. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277809 · Cited by 775 · cited 1×
“In this work we evaluated several other learning-to-rank algorithms [43] on the BioASQ test dataset, including MART [63], RankNet [64], Coord-inate Ascent [65], AdaRank [66], and LambdaMART, which are available in RankLib v2.”
Method(2006) · doi:10.7551/mitpress/7503.003.0029 · Cited by 753
Information Retrieval as Statistical Translation
Berger, A., & Lafferty, J. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130371 · Cited by 621
Research works printed but not linked: 5
- Mao Y, Wei C-H, Lu Z. NCBI at the 2014 BioASQ challenge task: large-scale biomedical semantic indexing and question answering. In: Proceedings of Question Answering Lab at CLEF. 2014.
- Liu K, Wu J, Peng S, Zhai C, Zhu S. The Fudan-UIUC participation in the BioASQ Challenge Task 2a: The Antinomyra system. Risk. 2014;129816:100.
- Mork JG, Demner-Fushman D, Schmidt SC, Aronson AR. Recent enhancements to the NLM medical text indexer. In: Working Notes for CLEF 2014 Conference, Sheffield, UK. 2014. p. 1328–36.
- Quoc C, Le V. Learning to rank with nonsmooth cost functions. In: NIPS’07, vol. 19. 2007. p. 193.
- Robertson SE, Walker S, Jones S, Hancock-Beaulieu MM, Gatford M. Okapi at TREC-3. Gaithersburg: NIST Special Publication; 1995. pp. 109–126 · doi:10.6028/nist.sp.500-225.adhoc-city
Cited by
Best Match: New relevance search for PubMed
Fiorini, N., Canese, K., Starchenko, G., Kireev, E., Kim, W., Miller, V., Osipov, M., Kholodov, M., Ismagilov, R., Mohan, S., et al. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2005343 · Cited by 175
Self-citationDeep neural network for hierarchical extreme multi-label text classification
Gargiulo, F., Silvestri, S., Ciampi, M., & Pietro, G. D. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.03.041 · Cited by 172
ML-Net: multi-label classification of biomedical texts with deep neural networks
Du, J., Chen, Q., Peng, Y., Xiang, Y., Tao, C., & Lu, Z. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz085 · Cited by 152
“We used the pre-trained PubMed word2vec (dimension: 200) [31] to map the tokens in the documents to high dimensional vectors.”
BackgroundMethodInfluentialSelf-citationRecent advances in biomedical literature mining
Zhao, S., Su, C., Lu, Z., & Wang, F. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa057 · Cited by 134
“Zhang et al. [82] provided a literature review on multi-label classification,and many models therein have been applied in this context, such as k-nearest neighbor (KNN) [83], Naive Bayes [84], SVM [85], learning to rank approaches [79, 86–88], etc.”
BackgroundMethodSelf-citationA Review of Cancer Genetics and Genomics Studies in Africa
Rotimi, S. O., Rotimi, O. A., & Salhia, B. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2020.606400 · Cited by 97
“Since PubMed Medical Subject Heading (MeSH) terms involve synonym control, it yields more precise and inclusive search results (16).”
Background
Metadata
- Authors
- Mao, Y., & Lu, Z.
- Affiliations
- Nanjing University of Chinese Medicine (CN)
- National Center for Biotechnology Information (US)
- Corresponding author
- Lu, Zhiyong
- Venue
- Journal of Biomedical Semantics
- Published
- 2017-04-17
- Volume
- 8
- Issue
- 1
- Pages
- 15-15
- ISSN
- 2041-1480 (ISSN-L)
- PMC
- PMC5392968
- OpenAlex
- W2606900069
- Cited by
- 87
- References
- 80
- Citations by year
- 2026: 8 · 2025: 9 · 2024: 5 · 2023: 10 · 2022: 12 · 2021: 10 · 2020: 12 · 2019: 12 · 2018: 7 · 2017: 1
- FWCI
- 3.57
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Journal of Biomedical Semantics (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (81674099, 81603498)
- Government of Jiangsu Province
- Six Talent Peaks Project in Jiangsu Province
- Priority Academic Program Development of Jiangsu Higher Education Institutions
- Qinglan Project of Jiangsu Province of China
- National Institutes of Health
- U.S. National Library of Medicine
- Topics
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Keywords
- MeSH indexing, automatic indexing, learning to rank, term ranking, post-processing, MEDLINE, PubMed, biomedical literature retrieval, parallel computing
- MeSH terms
- Abstracting and Indexing, Machine Learning, Automation, Benchmarking, PubMed, Medical Subject Headings
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: MeSH Now: automatic MeSH indexing at PubMed scale via learning to rank. https://app.cochat.ai/papers/10.1186/s13326-017-0123-3
- The paper
- Mao, Y., & Lu, Z. (2017). MeSH Now: automatic MeSH indexing at PubMed scale via learning to rank. Journal of Biomedical Semantics. https://doi.org/10.1186/s13326-017-0123-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

