Machine Learning Versus Standard Techniques for Updating Searches for Systematic Reviews: A Diagnostic Accuracy Study
The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
Letters1 August 2017Machine Learning Versus Standard Techniques for Updating Searches for Systematic Reviews: A Diagnostic Accuracy StudyPaul G. Shekelle, MD, PhD, Kanaka Shetty, MD, Sydne Newberry, PhD, Margaret Maglione, MPP, and Aneesa Motala, BAPaul G. Shekelle, MD, PhDFrom RAND Corporation, Santa Monica, California., Kanaka Shetty, MDFrom RAND Corporation, Santa Monica, California., Sydne Newberry, PhDFrom RAND Corporation, Santa Monica, California., Margaret Maglione, MPPFrom RAND Corporation, Santa Monica, California., and Aneesa Motala, BAFrom RAND Corporation, Santa Monica, California.Author, Article, and Disclosure Informationhttps://doi.org/10.7326/L17-0124 SectionsAboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Background: Systematic reviews are a cornerstone of evidence-based care and a necessary foundation for care recommendations to be labeled clinical practice guidelines. However, they become outdated relatively quickly and require substantial resources to maintain relevance. One particularly time-consuming task is updating the search to identify relevant articles published since the last search.We previously tested machine-learning approaches for making screening for updating more efficient by using 2 clinical topics as examples: medications to treat low bone density and off-label use of atypical antipsychotics. We tested 2 machine-learning algorithms: a generalized linear model with convex penalties (glmnet, R Foundation for Statistical ...References1. Dalal SR, Shekelle PG, Hempel S, Newberry SJ, Motala A, Shetty KD. A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Med Decis Making. 2013;33:343-55. [PMID: 22961102] doi:10.1177/0272989X12457243 CrossrefMedlineGoogle Scholar2. Joachims T. Text categorization with support vector machines: learning with many relevant features [Abstract]. In: Nedellec C, Rouveirol C, eds. Proceedings of the 10th European Conference on Machine Learning, Chemnitz, Germany, 21–23 April 1998. London: Springer-Verlag; 1998:137-42. Google Scholar3. Combs BP, Rappaport M, Caverly TJ, Matlock DD. "Due" for a scan: examining the utility of monitoring densitometry. JAMA Intern Med. 2013;173:2007-9. [PMID: 23877530] doi:10.1001/jamainternmed.2013.8998 CrossrefMedlineGoogle Scholar4. Cheng L, Xiong Y, Qin CZ, Zhang W, Chen XP, Li J, et al. HLA-B*58:01 is strongly associated with allopurinol-induced severe cutaneous adverse reactions in Han Chinese patients: a multicentre retrospective case-control clinical study [Letter]. Br J Dermatol. 2015;173:555-8. [PMID: 26104483] doi:10.1111/bjd.13688 CrossrefMedlineGoogle Scholar5. O'Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4:5. [PMID: 25588314] doi:10.1186/2046-4053-4-5 CrossrefMedlineGoogle Scholar Author, Article, and Disclosure InformationAffiliations: From RAND Corporation, Santa Monica, California.Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=L17-0124.This article was published at Annals.org on 13 June 2017. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoComputer-Aided Systematic Review Screening Comes of Age Brian J. Hemens and Alfonso Iorio Metrics Cited byApplying machine classifiers to update searches: Analysis from two case studiesArtificial Intelligence in HematologyThe automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrationsA rule‐based approach for automatically extracting data from systematic reviews and their updates to model the risk of conclusion changeSafe Healthcare Facilities: A Systematic Review on the Costs of Establishing and Maintaining Environmental Health in Facilities in Low- and Middle-Income CountriesComparison of a traditional systematic review approach with review-of-reviews and semi-automation as strategies to update the evidenceDecoding semi-automated title-abstract screening: findings from a convenience sample of reviewsFuture of evidence ecosystem series: 2. current opportunities and need for better tools and methodsIs it time for computable evidence synthesis?InformationResearchNovel text analytics approach to identify relevant literature for human health risk assessments: A pilot study with health effects of in utero exposuresPotential Technologies Review: A hybrid information retrieval framework to accelerate demand‐pull innovation in biomedical engineeringThe risk of conclusion change in systematic review updates can be estimated by learning from a database of published examplesAutomatic screening using word embeddings achieved high sensitivity and workload reduction for updating living network meta-analysesTrial2rev: Combini
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
O’Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., & Ananiadou, S. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-5 · Cited by 681
Cited by
Predicting the time needed for environmental systematic reviews and systematic maps
Haddaway, N. R., & Westgate, M. J. (2019) · Conservation Biology · doi:10.1111/cobi.13231 · Cited by 120
Topic search filters: a systematic scoping review
Damarell, R. A., May, N., Hammond, S., Sladek, R. M., & Tieman, J. J. (2019) · Health Information & Libraries Journal · doi:10.1111/hir.12244 · Cited by 48
Decoding semi-automated title-abstract screening: findings from a convenience sample of reviews
Gates, A., Gates, M., DaRosa, D., Elliott, S. A., Pillay, J., Rahman, S., Vandermeer, B., & Hartling, L. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01528-x · Cited by 31
Computer-Aided Systematic Review Screening Comes of Age
Hemens, B. J., & Iorio, A. (2017) · Annals of Internal Medicine · doi:10.7326/m17-1295 · Cited by 5
Metadata
- Authors
- Shekelle, P. G., Shetty, K., Newberry, S., Maglione, M., & Motala, A.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2017-08-01
- Volume
- 167
- Issue
- 3
- Pages
- 213-215
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2625299527
- Cited by
- 38
- References
- 1
- FWCI
- 1.80
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Machine Learning Versus Standard Techniques for Updating Searches for Systematic Reviews: A Diagnostic Accuracy Study. https://app.cochat.ai/papers/10.7326/l17-0124
- The paper
- Shekelle, P. G., Shetty, K., Newberry, S., Maglione, M., & Motala, A. (2017). Machine Learning Versus Standard Techniques for Updating Searches for Systematic Reviews: A Diagnostic Accuracy Study. Annals of Internal Medicine. https://doi.org/10.7326/l17-0124
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.