Innovations in Data Collection, Management, and Archiving for Systematic Reviews
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Data abstraction is a key step in conducting systematic reviews because data collected from study reports form the basis of appropriate conclusions. Recent methodological standards and expectations highlight several principles for data collection. To support implementation of these standards, this article provides a step-by-step tutorial for selecting data collection tools; constructing data collection forms; and abstracting, managing, and archiving data for systematic reviews. Examples are drawn from recent experience using the Systematic Review Data Repository for data collection and management. If it is done well, data collection for systematic reviews only needs to be done by 1 team and placed into a publicly accessible database for future use. Technological innovations, such as the Systematic Review Data Repository, will contribute to finding trustworthy answers for many health and health care questions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Developing core outcome sets for clinical trials: issues to consider
Williamson, P. R., Altman, D. G., Blazeby, J. M., Clarke, M., Devane, D., Gargon, E., & Tugwell, P. (2012) · Trials · doi:10.1186/1745-6215-13-132 · Cited by 1,661
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188 · cited 1×
“With more than 4000 systematic reviews published each year (25), innovation in methods to improve the efficiency and validity of systematic reviews is likely to have profound implications and will be indispensable for finding trustworthy answers to many health and health care questions.”
BackgroundSingle data extraction generated more errors than double data extraction in systematic reviews
Buscemi, N., Hartling, L., Vandermeer, B., Tjosvold, L., & Klassen, T. P. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.11.010 · Cited by 464 · cited 1×
“There are advantages to data abstractors having complementary skills and expertise (for example, a topic area specialist and a methodologist) (4, 14).”
BackgroundSemi-automated screening of biomedical citations for systematic reviews
Wallace, B. C., Trikalinos, T. A., Lau, J., Brodley, C., & Schmid, C. H. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-55 · Cited by 351 · cited 1×
“We anticipate that tools for automated screening of titles and abstracts will enable seamless data flow and handling from screening of search results to data abstraction (22, 23).”
MethodSelf-citationData Extraction Errors in Meta-analyses That Use Standardized Mean Differences
Gøtzsche, P. C., Hróbjartsson, A., Marić, K., & Tendal, B. (2007) · JAMA · doi:10.1001/jama.298.4.430 · Cited by 238
ExaCT: automatic extraction of clinical trial characteristics from journal publications
Kiritchenko, S., de Bruijn, B., Carini, S., Martin, J., & Sim, I. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-56 · Cited by 188 · cited 1×
“New tools that record the location for each data element and automatically enter validated data into a data collection system will further improve the efficiency and reproducibility of systematic reviews (24).”
Jones, A. P., Remmington, T., Williamson, P. R., Ashby, D., & Smyth, R. L. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.11.024 · Cited by 114
Horton, J., Vandermeer, B., Hartling, L., Tjosvold, L., Klassen, T. P., & Buscemi, N. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.04.007 · Cited by 81 · cited 2×
“Errors during data abstraction are common and have been well-documented (13–17).”
BackgroundToward modernizing the systematic review pipeline in genetics: efficient updating via data mining
Wallace, B. C., Small, K., Brodley, C. E., Lau, J., Schmid, C. H., Bertram, L., Lill, C. M., Cohen, J. T., & Trikalinos, T. A. (2012) · Genetics in Medicine · doi:10.1038/gim.2012.7 · Cited by 62
Self-citation
Cited by
Page, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769
“enhance reproducibility of meta-analytic results and share data collected [151, 152] and report the methods and results of updated SRs [153] and living SRs [154].”
MethodComparative Effectiveness of First-Line Medications for Primary Open-Angle Glaucoma
Li, T., Lindsley, K., Rouse, B., Hong, H., Shi, Q., Friedman, D. S., Wormald, R., & Dickersin, K. (2016) · Ophthalmology · doi:10.1016/j.ophtha.2015.09.005 · Cited by 320
Self-citationMerckx, J., Wali, R., Schiller, I., Caya, C., Gore, G. C., Chartrand, C., Dendukuri, N., & Papenburg, J. (2017) · Annals of Internal Medicine · doi:10.7326/m17-0848 · Cited by 287
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T., Mulrow, C. D., Shamseer, L., & Moher, D. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.11.010 · Cited by 211
“These developments include novel guidance on how to identify studies for inclusion (9-11), synthesise and present findings when meta-analysis is not possible or appropriate (12, 13), report and synthesise intervention characteristics of included studies (14, 15), and enhance the reproducibility of meta-analytic result…”
BackgroundBüchter, R. B., Weise, A., & Pieper, D. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01143-3 · Cited by 151
“It recommends that all members of the team are involved including data extractors, content area experts, statisticians and reviewers with formal training in form design such as epidemiologists [37].”
BackgroundMethodInfluential
Metadata
- Authors
- Li, T., Vedula, S. S., Hadar, N., Parkin, C., Lau, J., & Dickersin, K.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2015-02-17
- Volume
- 162
- Issue
- 4
- Pages
- 287-294
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2004942506
- Cited by
- 117
- References
- 9
- FWCI
- 2.20
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.