Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 1

The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.

Simulation-based active learning for systematic reviews: A scoping review of literature

Teijema, Jelle Jasper, Ribeiro, Guilherme, Seuren, Sofie, Anadria, Daniel, Bagheri, Ayoub, van de Schoot, Rens·2025·Journal of Information Science
DOI 10.1177/01655515251379058
Open this paper in CoChat — free account

Abstract

Background: Active learning is a proposed method for accelerating the screening phase of systematic reviews. While extensively studied, evidence remains scattered across a fragmented body of literature. Objective: This scoping review investigates whether active learning is recommended for systematic review screening and identifies areas needing further research. Design: We screened 1887 records published since 2006 using ASReview, an active learning tool, and included 60 relevant studies. We also analysed 238 of 336 collected datasets for study design, dataset usage, and implementation. Results: All 60 studies recommended active learning as a means to improve screening efficiency. Despite some methodological heterogeneity, consistent endorsement was found across the literature. Conclusion: Active learning shows strong potential to support systematic review screening. Standardising evaluation metrics, encouraging open data practices, and diversifying model configurations are key priorities for advancing this field.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/01655515251379058

Cites

25 of 25 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 25 references

Research works printed but not linked: 59

  1. 10.1186/s13643-019-1074-9 · DOI · Paper page
  2. 10.1002/9781119536604 · DOI · Paper page
  3. 10.1007/978-1-0716-1566-9_2 · DOI · Paper page
  4. 10.1177/02683962211048201 · DOI
  5. 10.1093/bib/bbaa296 · DOI
  6. 10.7507/1672-2531.202012150 · DOI
  7. 10.17605/OSF.IO/G3NKZ · DOI
  8. 10.1016/j.jbi.2014.06.005 · DOI · Paper page
  9. 10.1016/j.jclinepi.2017.08.011 · DOI · Paper page
  10. O’Mara-Eves A. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev 201; 4(1): 1–22.
  11. Teijema JJ. Simulation–based Active Learning for Systematic Reviews: A Scoping Review of the Literature – Repository. 2025. DOI: 10.17605/OSF.IO/T9HGM. url: osf.io/t9hgm. · DOI
  12. Muthu S. The efficiency of machine learning-assisted platform for article screening in systematic reviews in orthopaedics. Int Orthop 2022: 1–6.
  13. 10.1080/00273171.2020.1853501 · DOI
  14. Teijema JJ, de Bruin J, Bagheri A, et al. Large-scale simulation study of active learning models for systematic reviews. PsyArXiv, 1 November, 2023.
  15. Wallace BC Small K Brodley CE et al. Modeling annotation time to reduce workload in comparative effectiveness reviews pp. 28–35. DOI: 10.1145/1882992.1882999. · DOI · Paper page
  16. Wallace BC Small K Brodley CE et al. Active learning for biomedical citation screening pp. 173–181. DOI: 10.1145/1835804.1835829. · DOI · Paper page
  17. Wallace BC Small K Brodley CE et al. Who should label what? instance allocation in multiple expert active learning. pp. 176–187. DOI: 10.1137/1.9781611972818.16. · DOI
  18. 10.1145/2110363.2110464 · DOI · Paper page
  19. 10.3389/fninf.2013.00038 · DOI · Paper page
  20. Cormack GV Grossman MR. Autonomy and Reliability of Continuous Active Learning for Technology-Assisted Review. arXiv:1504.06868 2015. · DOI · Paper page
  21. Timsina P El-Gayar O Liu J. Active learning for the automation of medical systematic review creation.
  22. 10.1016/j.jbi.2016.06.001 · DOI
  23. 10.1145/2983323.2983776 · DOI
  24. Cormack GV Grossman MR. Technology-assisted review in empirical medicine: Waterloo participation in CLEF eHealth 2017 vol. 1866.
  25. Yu Z Menzies T (eds) Data balancing for technologically assisted reviews: undersampling or reweighting vol. 1866.
  26. 10.1145/3084226.3084243 · DOI · Paper page
  27. Kanoulas E Li D Azzopardi L et al. CLEF 2017 Technologically assisted reviews in empirical medicine overview https://ceur-ws.org/Vol-1866/invited_paper_12.pdf
  28. Norman C Leeflang M Névéol A. LIMSI@CLEF eHealth 2018 Task 2: technology assisted reviews by stacking active and static learning vol. 2125.
  29. 10.1007/s10796-016-9724-0 · DOI
  30. Cormack GV Grossman MR. Technology-assisted review in empirical medicine: Waterloo participation in CLEF eHealth 2018. 2125.
  31. Kanoulas E, Li D, Azzopardi L, et al. CLEF 2018 Technologically assisted reviews in empirical medicine overview. In: CLEF 2018 technologically assisted reviews in empirical medicine overview.
  32. Di Nunzio GM. Finding all the needles in a haystack: a system to estimate the costs of e-discovery and systematic reviews vol. 2167 p. 106.
  33. 10.1186/s13643-019-1162-x · DOI
  34. Carvallo A Parra D. Comparing word embeddings for document screening based on active learning. 2414 pp. 100–107.
  35. 10.1007/s10669-019-09717-3 · DOI
  36. 10.1016/j.jval.2019.09.2235 · DOI
  37. 10.1145/3388640 · DOI
  38. 10.1007/s10791-019-09361-0 · DOI
  39. 10.2196/22422 · DOI
  40. 10.1007/s11192-020-03648-6 · DOI
  41. Li D Zafeiriadis P Kanoulas E. APS: an active pubmed search system for technology assisted reviews pp. 2137–2140. DOI: 10.1145/3397271.3401401. · DOI
  42. 10.3389/fpubh.2020.582205 · DOI
  43. 10.3390/ijerph17124509 · DOI
  44. 10.1017/S0266462321000696 · DOI · Paper page
  45. Molinari A, Kanoulas E. Transferring knowledge between topics in systematic reviews. Intel Syst Appl 2022; 16: 200150.
  46. 10.1016/j.eswa.2022.116958 · DOI · Paper page
  47. Hou J, Wang X, Dubois J-J, et al. Extreme systematic reviews: a large literature screening dataset to support environmental policymaking. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022, pp. 4029–4033.
  48. Bianco GD, Duarte D, Gonçalves MA. Reducing the user labeling effort in effective high recall tasks by fine-tuning active learning. J Intel Inform Syst 2023: 1–20.
  49. 10.1016/j.euros.2023.07.005 · DOI
  50. Orel E, Ciglenecki I, Thiabaud A, et al. An automated literature review tool (LiteRev) for streamlining and accelerating research using natural language processing and machine learning: descriptive performance evaluation study. J Med Intern Res 2023; 25: e39736.
  51. Pijls BG. Machine Learning assisted systematic reviewing in orthopaedics. J Orthop, 2023.
  52. 10.1016/j.jval.2023.09.2122 · DOI
  53. 10.1080/14737167.2023.2234639 · DOI
  54. 10.1109/BIBM58861.2023.10385372 · DOI
  55. 10.1145/3539618.3591764 · DOI
  56. 10.1186/s13643-024-02587-0 · DOI · Paper page
  57. 10.1186/s13643-023-02421-z · DOI
  58. 10.1016/j.jbi.2017.07.010 · DOI · Paper page
  59. 10.1186/s13643-024-02472-w · DOI · Paper page

Metadata

Authors
Teijema, J. J., Ribeiro, G., Seuren, S., Anadria, D., Bagheri, A., & van de Schoot, R.
Venue
Journal of Information Science
Publisher
SAGE Publications
Published
2025-12-17
ISSN
  • 0165-5515
  • 1741-6485
DOI
10.1177/01655515251379058
Cited by
1
References
25
Language
en
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Simulation-based active learning for systematic reviews: A scoping review of literature. https://app.cochat.ai/papers/10.1177/01655515251379058
The paper
Teijema, J. J., Ribeiro, G., Seuren, S., Anadria, D., Bagheri, A., & van de Schoot, R. (2025). Simulation-based active learning for systematic reviews: A scoping review of literature. Journal of Information Science. https://doi.org/10.1177/01655515251379058