CoChat

Machine Learning Versus Standard Techniques for Updating Searches for Systematic Reviews: A Diagnostic Accuracy Study

Shekelle, Paul G., Shetty, Kanaka, Newberry, Sydne, Maglione, Margaret, Motala, Aneesa · 2017 · Annals of Internal Medicine
Cited by 38

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Abstract

Letters1 August 2017Machine Learning Versus Standard Techniques for Updating Searches for Systematic Reviews: A Diagnostic Accuracy StudyPaul G. Shekelle, MD, PhD, Kanaka Shetty, MD, Sydne Newberry, PhD, Margaret Maglione, MPP, and Aneesa Motala, BAPaul G. Shekelle, MD, PhDFrom RAND Corporation, Santa Monica, California., Kanaka Shetty, MDFrom RAND Corporation, Santa Monica, California., Sydne Newberry, PhDFrom RAND Corporation, Santa Monica, California., Margaret Maglione, MPPFrom RAND Corporation, Santa Monica, California., and Aneesa Motala, BAFrom RAND Corporation, Santa Monica, California.Author, Article, and Disclosure Informationhttps://doi.org/10.7326/L17-0124 SectionsAboutFull TextPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissions ShareFacebookTwitterLinkedInRedditEmail Background: Systematic reviews are a cornerstone of evidence-based care and a necessary foundation for care recommendations to be labeled clinical practice guidelines. However, they become outdated relatively quickly and require substantial resources to maintain relevance. One particularly time-consuming task is updating the search to identify relevant articles published since the last search.We previously tested machine-learning approaches for making screening for updating more efficient by using 2 clinical topics as examples: medications to treat low bone density and off-label use of atypical antipsychotics. We tested 2 machine-learning algorithms: a generalized linear model with convex penalties (glmnet, R Foundation for Statistical ...References1. Dalal SR, Shekelle PG, Hempel S, Newberry SJ, Motala A, Shetty KD. A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Med Decis Making. 2013;33:343-55. [PMID: 22961102] doi:10.1177/0272989X12457243 CrossrefMedlineGoogle Scholar2. Joachims T. Text categorization with support vector machines: learning with many relevant features [Abstract]. In: Nedellec C, Rouveirol C, eds. Proceedings of the 10th European Conference on Machine Learning, Chemnitz, Germany, 21–23 April 1998. London: Springer-Verlag; 1998:137-42. Google Scholar3. Combs BP, Rappaport M, Caverly TJ, Matlock DD. "Due" for a scan: examining the utility of monitoring densitometry. JAMA Intern Med. 2013;173:2007-9. [PMID: 23877530] doi:10.1001/jamainternmed.2013.8998 CrossrefMedlineGoogle Scholar4. Cheng L, Xiong Y, Qin CZ, Zhang W, Chen XP, Li J, et al. HLA-B*58:01 is strongly associated with allopurinol-induced severe cutaneous adverse reactions in Han Chinese patients: a multicentre retrospective case-control clinical study [Letter]. Br J Dermatol. 2015;173:555-8. [PMID: 26104483] doi:10.1111/bjd.13688 CrossrefMedlineGoogle Scholar5. O'Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4:5. [PMID: 25588314] doi:10.1186/2046-4053-4-5 CrossrefMedlineGoogle Scholar Author, Article, and Disclosure InformationAffiliations: From RAND Corporation, Santa Monica, California.Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=L17-0124.This article was published at Annals.org on 13 June 2017. PreviousarticleNextarticle Advertisement FiguresReferencesRelatedDetailsSee AlsoComputer-Aided Systematic Review Screening Comes of Age Brian J. Hemens and Alfonso Iorio Metrics Cited byApplying machine classifiers to update searches: Analysis from two case studiesArtificial Intelligence in HematologyThe automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrationsA rule‐based approach for automatically extracting data from systematic reviews and their updates to model the risk of conclusion changeSafe Healthcare Facilities: A Systematic Review on the Costs of Establishing and Maintaining Environmental Health in Facilities in Low- and Middle-Income CountriesComparison of a traditional systematic review approach with review-of-reviews and semi-automation as strategies to update the evidenceDecoding semi-automated title-abstract screening: findings from a convenience sample of reviewsFuture of evidence ecosystem series: 2. current opportunities and need for better tools and methodsIs it time for computable evidence synthesis?InformationResearchNovel text analytics approach to identify relevant literature for human health risk assessments: A pilot study with health effects of in utero exposuresPotential Technologies Review: A hybrid information retrieval framework to accelerate demand‐pull innovation in biomedical engineeringThe risk of conclusion change in systematic review updates can be estimated by learning from a database of published examplesAutomatic screening using word embeddings achieved high sensitivity and workload reduction for updating living network meta-analysesTrial2rev: Combini

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7326/l17-0124

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Cited by

See all 38

Metadata

Authors
Shekelle, P. G., Shetty, K., Newberry, S., Maglione, M., & Motala, A.
Venue
Annals of Internal Medicine
Published
2017-08-01
Volume
167
Issue
3
Pages
213-215
ISSN
  • 0003-4819 (ISSN-L)
  • 1539-3704
OpenAlex
W2625299527
Cited by
38
References
1
FWCI
1.80
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.