Modernizing the systematic review process to inform comparative effectiveness: tools and methods
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Systematic reviews are being increasingly used to inform all levels of healthcare, from bedside decisions to policy-making. Since they are designed to minimize bias and subjectivity, they are a preferred option to assess the comparative effectiveness and safety of healthcare interventions. However, producing systematic reviews and keeping them up-to-date is becoming increasingly onerous for three reasons. First, the body of biomedical literature is expanding exponentially with no indication of slowing down. Second, as systematic reviews gain wide acceptance, they are also being used to address more complex questions (e.g., evaluating the comparative effectiveness of many interventions together rather than focusing only on pairs of interventions). Third, the standards for performing systematic reviews have become substantially more rigorous over time. To address these challenges, we must carefully prioritize the questions that should be addressed by systematic reviews and optimize the processes of research synthesis. In addition to reducing the workload involved in planning and conducting systematic reviews, we also need to make efforts to increase the transparency, reliability and validity of the review process; these aims can be grouped under the umbrella of ‘modernization‘ of the systematic review process.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The BUGS project: evolution, critique and future directions
Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009) · Statistics in Medicine · doi:10.1002/sim.3680 · Cited by 1,876
Systematic meta-analyses of Alzheimer disease genetic association studies: the AlzGene database
Bertram, L., McQueen, M. B., Mullin, K., Blacker, D., & Tanzi, R. E. (2007) · Nature Genetics · doi:10.1038/ng1934 · Cited by 1,857
Salanti, G. (2012) · Research Synthesis Methods · doi:10.1002/jrsm.1037 · Cited by 1,729
Network meta‐analysis for indirect treatment comparisons
Lumley, T. (2002) · Statistics in Medicine · doi:10.1002/sim.1201 · Cited by 1,206
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
Closing the Gap between Methodologists and End-Users: R as a Computational Back-End
Wallace, B. C., Dahabreh, I. J., Trikalinos, T. A., Lau, J., Trow, P., & Schmid, C. H. (2012) · Journal of Statistical Software · doi:10.18637/jss.v049.i05 · Cited by 1,125
Self-citationHow Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838
Reducing Workload in Systematic Review Preparation Using Automated Citation Classification
Cohen, A. M., Hersh, W. R., Peterson, K., & Yen, P.-Y. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1929 · Cited by 411
Semi-automated screening of biomedical citations for systematic reviews
Wallace, B. C., Trikalinos, T. A., Lau, J., Brodley, C., & Schmid, C. H. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-55 · Cited by 351
Self-citationAutomatic classification of sentences to support Evidence Based Medicine.
Kim, S. N., Martinez, D., Cavedon, L., & Yencken, L. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-s2-s5 · Cited by 214
Cited by
Systematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472
“Proposed decision support systems for systematic reviewers include ones that help the basic tasks of systematic reviews [6,12,13].”
BackgroundMarshall, I. J., Noel‐Storr, A., Kuiper, J., Thomas, J., & Wallace, B. C. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1287 · Cited by 435
Self-citationBougioukas, K. I., Liakos, A., Tsapas, A., Ntzani, E., & Haidich, A.-B. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.002 · Cited by 251
(2016) · PubMed · Cited by 134
Automating Biomedical Evidence Synthesis: RobotReviewer
Marshall, I., Kuiper, J., Banner, E., & Wallace, B. C. (2017) · Proceedings of ACL 2017, System Demonstrations · doi:10.18653/v1/p17-4002 · Cited by 96
“But the expanding primary research base has made producing and maintaining SRs increasingly onerous (Bastian et al., 2010; Wallace et al., 2013).”
BackgroundSelf-citationRobson, R. C., Pham, B., Hwee, J., Thomas, S. M., Rios, P., Page, M. J., & Tricco, A. C. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.10.003 · Cited by 86
Finding better active learners for faster literature reviews
Yu, Z., Kraft, N. A., & Menzies, T. (2018) · Empirical Software Engineering · doi:10.1007/s10664-017-9587-0 · Cited by 83
“Wallace et al. conducted a series of studies with machine learning techniques, especially active learning [66,64,65,63,61,62,45].”
BackgroundFAST2: An intelligent assistant for finding relevant papers
Yu, Z., & Menzies, T. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.11.021 · Cited by 80
“The estimator designed by Wallace et al. in 2013 consists of the following: • Probability prediction: fit a sigmoid function on the current labeled data K with SVM decision function as inputs and labels as outputs. where f i is the SVM decision function and P ( y i | f i ) is the probability…”
MethodAutomating Risk of Bias Assessment for Clinical Trials
Marshall, I. J., Kuiper, J., & Wallace, B. C. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2431314 · Cited by 59
Self-citationPretraining to Recognize PICO Elements from Randomized Controlled Trial Literature.
Tian, K., Shirui, Z., & Chunhua, W. (2019) · Studies in Health Technology and Informatics · doi:10.3233/shti190209 · Cited by 45
“Automatic extraction of PICO statements in the biomedical literature is desired to facilitate evidence retrieval, appraisal and synthesis by clinicians and the public [6; 7].”
Method
Metadata
- Authors
- Wallace, B. C., Dahabreh, I. J., Schmid, C. H., Lau, J., & Trikalinos, T. A.
- Venue
- Journal of Comparative Effectiveness Research
- Publisher
- Becaris Publishing Limited
- Published
- 2013-05
- Volume
- 2
- Issue
- 3
- Pages
- 273-282
- ISSN
- 2042-6305 (ISSN-L)
- 2042-6313
- OpenAlex
- W2078936286
- Cited by
- 51
- References
- 16
- FWCI
- 1.01
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.