CoChat

The DOI matches its Crossref record; 90 of the 90 references it cites were found in a registry or an index.

Machine learning techniques for the automation of literature reviews and systematic reviews in EFSA

Open access
Jaspers, Stijn, De Troyer, Ewoud, Aerts, Marc · 2018 · EFSA Supporting Publications

Abstract

This Report presents the results from EFSA project RC/EFSA/AMU/2016/01 related to the implementation of machine learning techniques for literature reviews and systematic reviews in EFSA. An overview of the different steps of a systematic review is provided, along with possible ways for automation. Although it was found that most steps could benefit from automation, it was also observed that some steps require more sophisticated methods than those encompassed within the machine learning framework. Availability of data and methodology allowed for the development of an automatic screening tool based on several machine learning techniques. The developed shiny R application can be used for the screening of abstracts and full texts. Properties of machine learning techniques are discussed in this Report together with their most important advantages and disadvantages. The latter discussion includes both general properties, as well as context-specific properties based on their performance in three case studies. Although creating a universal automatic data extraction tool was considered to be infeasible in this stage, this step of the systematic review was addressed to allow the reviewer to scan the uploaded pdf files for certain words or string of words. Based on observations from the performed case studies, recommendations were made regarding which methods are preferred in specific situations. More explicitly, a discussion is made about the performance of the classifiers with respect to the magnitude of the pool of papers to be screened as well as to the amount of imbalance, referring to the proportion of relevant and irrelevant papers. Finally, it was concluded that the results presented in this report provide proof that the developed shiny application could be efficiently used in combination with other software such as DistillerSR.

Overview

Integrity signals

Registry

Crossref · 10.2903/sp.efsa.2018.en-1427

Cites

90 of 90 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 89 references

Research works printed but not linked: 3

  1. Higgins, 2011, Update on the MECIR project: methodological expectations for Cochrane intervention, Cochrane Methodology, 1, 2
  2. Lin, 2011, Automating the process of critical appraisal and assessing the strength of evidence with information extraction technology, Journal of Evaluation in Clinical Practice, 17, 832, 10.1111/j.1365-2753.2011.01712.x · DOI
  3. Mitchell, 1997, Machine Learning, 400

Cited by

See all 36

Metadata

Authors
Jaspers, S., De Troyer, E., & Aerts, M.
Affiliations
  • Hasselt University (BE)
  • UHASSELT Belgium
Venue
EFSA Supporting Publications
Publisher
Wiley
Published
2018-06
Volume
15
Issue
6
ISSN
2397-8325 (ISSN-L)
OpenAlex
W2808716231
Cited by
36
References
89
Citations by year
2025: 2 · 2024: 9 · 2023: 6 · 2022: 4 · 2021: 6 · 2020: 4 · 2019: 4 · 2018: 1
FWCI
0.75
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
EFSA Supporting Publications (publishedVersion, other-oa)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › General Social Sciences › Computational and Text Analysis Methods
  • Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
Keywords
machine learning, systematic reviews, automation, automated data extraction, classification performance, class imbalance, EFSA

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Machine learning techniques for the automation of literature reviews and systematic reviews in EFSA. https://app.cochat.ai/papers/10.2903/sp.efsa.2018.en-1427
The paper
Jaspers, S., De Troyer, E., & Aerts, M. (2018). Machine learning techniques for the automation of literature reviews and systematic reviews in EFSA. EFSA Supporting Publications. https://doi.org/10.2903/sp.efsa.2018.en-1427

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.