CoChat

Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses

Open access

The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.

Abstract

Background: Systematic reviews are the cornerstone of evidence-based medicine. However, systematic reviews are time consuming and there is growing demand to produce evidence more quickly, while maintaining robust methods. In recent years, artificial intelligence and active-machine learning (AML) have been implemented into several SR software applications. As some of the barriers to adoption of new technologies are the challenges in set-up and how best to use these technologies, we have provided different situations and considerations for knowledge synthesis teams to consider when using artificial intelligence and AML for title and abstract screening. Methods: We retrospectively evaluated the implementation and performance of AML across a set of ten historically completed systematic reviews. Based upon the findings from this work and in consideration of the barriers we have encountered and navigated during the past 24 months in using these tools prospectively in our research, we discussed and developed a series of practical recommendations for research teams to consider in seeking to implement AML tools for citation screening into their workflow. Results: We developed a seven-step framework and provide guidance for when and how to integrate artificial intelligence and AML into the title and abstract screening process. Steps include: (1) Consulting with Knowledge user/Expert Panel; (2) Developing the search strategy; (3) Preparing your review team; (4) Preparing your database; (5) Building the initial training set; (6) Ongoing screening; and (7) Truncating screening. During Step 6 and/or 7, you may also choose to optimize your team, by shifting some members to other review stages (e.g., full-text screening, data extraction). Conclusion: Artificial intelligence and, more specifically, AML are well-developed tools for title and abstract screening and can be integrated into the screening process in several ways. Regardless of the method chosen, transparent reporting of these methods is critical for future studies evaluating artificial intelligence and AML.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12874-021-01451-2

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Seven-step approach to integrating active machine-learning into title/abstract screening
Fig. 1 Seven-step approach to integrating active machine-learning into title/abstract screeningCC BY · PubMed Central
Fig. 2 Integration of AI into the overall title and abstract screening process
Fig. 2 Integration of AI into the overall title and abstract screening processCC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 41 references

Research works printed but not linked: 3

  1. Cottrell E, Whitlock E, Kato E, Uhl S, Belinson S, Chang C, et al. Defining the Benefits of Stakeholders Engagement in Systematic Reviews. Rockville: Agency for Healthcare Research and Quality (US); 2014.
  2. Thomas J, Graziosi S, Brunton J, Ghouze Z, O’Driscoll P, Bond M. EPPI-Reviewer: advanced software for systematic reviews, maps and evidence synthesis. London: UCL Social Research Institute: EPPI-Centre Software; 2020.
  3. Rice D, Wolfe D, Garritty C, Hersi M, Esmaeilisaraji L, Butler C, et al. Best Practice in Pain Management: Rapid Reviews of Guidelines and Knowledge Syntheses. Ottawa: Ottawa Hospital Research Insitute; 2020.

Cited by

See all 120

Metadata

Authors
Hamel, C., Hersi, M., Kelly, S. E., Tricco, A. C., Straus, S., Wells, G., Pham, B., & Hutton, B.
Affiliations
  • Ottawa Hospital Research Institute (CA)
  • University of Ottawa (CA)
  • St. Michael's Hospital (CA)
Corresponding author
Hamel, Candyce
Venue
BMC Medical Research Methodology
Published
2021-12
Volume
21
Issue
1
Pages
285-285
ISSN
1471-2288 (ISSN-L)
OpenAlex
W4200362342
Cited by
104
References
40
Citations by year
2026: 21 · 2025: 24 · 2024: 29 · 2023: 20 · 2022: 10
FWCI
8.55
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • uO Research (University of Ottawa) (submittedVersion, public-domain)
  • TSpace (University of Toronto) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Canadian Institutes of Health Research (Tier 2)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
artificial intelligence, active learning (machine learning), title and abstract screening, systematic reviews, knowledge synthesis, citation screening, screening workflow, search strategies, reporting transparency
MeSH terms
Artificial Intelligence, Humans, Mass Screening, Research Design, Retrospective Studies, Evidence-Based Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses. https://app.cochat.ai/papers/10.1186/s12874-021-01451-2
The paper
Hamel, C., Hersi, M., Kelly, S. E., Tricco, A. C., Straus, S., Wells, G., Pham, B., & Hutton, B. (2021). Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-021-01451-2

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.