The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses
Open accessAbstract
Abstract screening is one important aspect of conducting a high‐quality and comprehensive systematic review and meta‐analysis. Abstract screening allows the review team to conduct the tedious but vital first step to synthesize the extant literature: winnowing down the overwhelming amalgamation of citations discovered through research databases to the citations that should be “full‐text” screened and eventually included in the review. Although it is a critical process, few guidelines have been put forth since the publications of seminal systematic review textbooks. The purpose of this paper, therefore, is to provide a practical set of best practice guidelines to help future review teams and managers. Each of the 10 proposed guidelines is explained using real‐world examples or illustrations from applications. We also delineate recent experiences where a team of abstract screeners double‐screened 14 923 abstracts in 89 days.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120 · cited 1×
“The abstract screening tool is based on the inclusion criteria of the study guiding the review and preferably published in a research protocol created prior to the literature search.(19) The abstract screening tool guides screeners in their decisions about whether a citation is eligible for the review and thus plays a…”
MethodLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
(2000) · Cited by 9,198
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635 · cited 1×
“Double‐screening all available abstracts is not a new practice and has been suggested as a best practice for decades.(18) Guidance from the Cochrane Collaboration,(8) the Institute of Medicine,(10) and the Center for Review and Dissemination(9) all include the importance of independent screening of identified studies…”
MethodThe Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
When and Why Incentives (Don't) Work to Modify Behavior
Gneezy, U., Meier, S., & Rey-Biel, P. (2011) · Journal of Economic Perspectives · doi:10.1257/jep.25.4.191 · Cited by 1,735
Research Synthesis and Meta-Analysis: A Step-by-Step Approach
Cooper, H. (2017) · doi:10.4135/9781071878644 · Cited by 1,716
Social network changes and life events across the life span: A meta-analysis.
Wrzus, C., Hänel, M., Wagner, J., & Neyer, F. J. (2013) · Psychological Bulletin · doi:10.1037/a0028601 · Cited by 1,381
Donepezil for dementia due to Alzheimer’s disease.
Birks, J. S., & Harvey, R. J. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd001190.pub3 · Cited by 1,200
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-012545 · Cited by 737 · cited 1×
“After conducting a comprehensive literature search to identify studies, abstract screening is the next stage in the process where bias could threaten the validity of the identified studies: abstract screening actively eliminates studies from the review and therefore the potential for bias must not be overlooked.(7) Ye…”
BackgroundDeploying an interactive machine learning system in an evidence-based practice center: abstrackr
Wallace, B. C., Small, K., Brodley, C. E., Lau, J., & Trikalinos, T. A. (2012) · Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium · doi:10.1145/2110363.2110464 · Cited by 532
Pigott, T. D. (2012) · doi:10.1007/978-1-4614-2278-5 · Cited by 453 · cited 1×
“Conducting a systematic review and meta‐analysis, large or small, requires dedicated planning, consistent information tracking, and constant managerial oversight.(1) A high‐quality review relies on a team of content and methodological members' expertise combined with knowledge”
BackgroundSelf-citationLiving systematic reviews: 2. Combining human and machine effort
Thomas, J., Noel-Storr, A., Marshall, I., Wallace, B., McDonald, S., Mavergames, C., Glasziou, P., Shemilt, I., Synnot, A., Turner, T., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.011 · Cited by 355
Children’s Creativity: A Theoretical Framework and Systematic Review
Kupers, E., Lehmann-Wermser, A., McPherson, G., & van Geert, P. (2019) · Review of Educational Research · doi:10.3102/0034654318815707 · Cited by 195
Rathbone, J., Hoffmann, T., & Glasziou, P. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0067-6 · Cited by 184
Shemilt, I., Simon, A., Hollands, G. J., Marteau, T. M., Ogilvie, D., O'Mara‐Eves, A., Kelly, M. P., & Thomas, J. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1093 · Cited by 170
Systematic searching for environmental evidence using multiple tools and sources
Livoreil, B., Glanville, J., Haddaway, N. R., Bayliss, H., Bethel, A., de Lachapelle, F. F., Robalino, S., Savilaakso, S., Zhou, W., Petrokofsky, G., et al. (2017) · Environmental Evidence · doi:10.1186/s13750-017-0099-6 · Cited by 156 · cited 1×
“After a thorough discussion of the screening tool, the screeners should independently screen the same subset of abstracts using the screening tool.(20) The Cochrane Handbook(8) suggests pilot screening 10 to 12 abstracts in a training phase.”
MethodCan abstract screening workload be reduced using text mining? User experiences of the tool Rayyan
Olofsson, H., Brolund, A., Hellberg, C., Silverstein, R., Stenström, K., Österberg, M., & Dagerhamn, J. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1237 · Cited by 102
Smith, E. H. B., Liwicki, M., & Peng, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70552-6 · Cited by 5
Cited by
Methodological guidance paper: High-quality meta- analysis in a systematic review
Pigott, T. D., & Polanin, J. R. (2020) · Review of Educational Research · doi:10.3102/0034654319877153 · Cited by 568
Self-citationAutomated Paper Screening for Clinical Reviews Using Large Language Models
Guo, E., Gupta, M., Deng, J., Park, Y.-J., Paget, M., & Naugler, C. (2023) · arXiv · doi:10.48550/arxiv.2305.00844 · Cited by 230
Automated paper screening for clinical reviews using large language models: Data analysis study
Guo, E., Gupta, M., Deng, J., Park, Y.-J., Paget, M., & Naugler, C. (2024) · Journal of Medical Internet Research · doi:10.2196/48996 · Cited by 205
Chai, K. E. K., Lines, R. L. J., Gucciardi, D. F., & Ng, L. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01635-3 · Cited by 190
Polanin, J. R., Espelage, D. L., Grotpeter, J. K., Ingram, K., Michaelson, L., Spinney, E., Valido, A., Sheikh, A. E., Torgal, C., & Robinson, L. (2022) · Prevention Science · doi:10.1007/s11121-021-01259-y · Cited by 171
Self-citation
Metadata
- Authors
- Polanin, J. R., Pigott, T. D., Espelage, D. L., & Grotpeter, J. K.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2019-06-24
- Volume
- 10
- Issue
- 3
- Pages
- 330-342
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W2944878516
- Cited by
- 401
- References
- 22
- FWCI
- 3.86
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.