CoChat

Supporting systematic reviews using LDA-based document representations

Open access
Mo, Yuanhan, Kontonatsios, Georgios, Ananiadou, Sophia · 2015 · Systematic Reviews
Cited by 83

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Identifying relevant studies for inclusion in a systematic review (i.e. screening) is a complex, laborious and expensive task. Recently, a number of studies has shown that the use of machine learning and text mining methods to automatically identify relevant studies has the potential to drastically decrease the workload involved in the screening phase. The vast majority of these machine learning methods exploit the same underlying principle, i.e. a study is modelled as a bag-of-words (BOW). METHODS: We explore the use of topic modelling methods to derive a more informative representation of studies. We apply Latent Dirichlet allocation (LDA), an unsupervised topic modelling approach, to automatically identify topics in a collection of studies. We then represent each study as a distribution of LDA topics. Additionally, we enrich topics derived using LDA with multi-word terms identified by using an automatic term recognition (ATR) tool. For evaluation purposes, we carry out automatic identification of relevant studies using support vector machine (SVM)-based classifiers that employ both our novel topic-based representation and the BOW representation. RESULTS: Our results show that the SVM classifier is able to identify a greater number of relevant studies when using the LDA representation than the BOW representation. These observations hold for two systematic reviews of the clinical domain and three reviews of the social science domain. CONCLUSIONS: A topic-based feature representation of documents outperforms the BOW representation when applied to the task of automatic citation screening. The proposed term-enriched topics are more informative and less ambiguous to systematic reviewers.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s13643-015-0117-0

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 11 references

Research works printed but not linked: 26

  1. Barza M, Trikalinos TA, Lau J. Statistical considerations in meta-analysis. Infect Dis Clin North Am. 2009; 23(2):195–210. · DOI
  2. Demner-Fushman D, Lin J. Answering clinical questions with knowledge-based and statistical techniques. Comput Ling. 2007; 33(1):63–103. · DOI · Paper page
  3. Hunter L, Cohen KB. Biomedical language processing: perspective what’s beyond PubMed?Mol Cell. 2006; 21(5):589. · DOI · Paper page
  4. Cohen AM. Performance of support-vector-machine-based classification on 15 systematic review topics evaluated with the WSS@95 measure. J Am Med Inform Assoc. 2011; 18(1):104–4. · DOI · Paper page
  5. O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Sys Rev. 2015; 4(1). doi: 10.1186/2046-4053-4-5 . Highly Accessed. · DOI · Paper page
  6. Lukins SK, Kraft NA, Etzkorn LH. Source code retrieval for bug localization using latent Dirichlet allocation. In: Reverse Engineering, 2008. WCRE’08. 15th Working Conference On. Antwerp: IEEE: 2008. p. 155–64.
  7. Henderson K, Eliassi-Rad T. Applying latent Dirichlet allocation to group discovery in large graphs. In: Proceedings of the 2009 ACM Symposium on Applied Computing. New York, NY, USA: ACM: 2009. p. 1456–1461.
  8. Maskeri G, Sarkar S, Heafield K. Mining business topics in source code using latent Dirichlet allocation. In: Proceedings of the 1st India Software Engineering Conference. New York, NY, USA: ACM: 2008. p. 113–20.
  9. Linstead E, Lopes C, Baldi P. An application of latent Dirichlet allocation to analyzing software evolution. In: Machine learning and applications, 2008. ICMLA’08. Seventh International Conference On. San Diego, CA: IEEE: 2008. p. 813–8.
  10. Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. Js Mach Learn Res. 2003; 3:993–1022.
  11. Redner RA, Walker HF. Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 1984; 26(2):195–239. · DOI · Paper page
  12. Berry MW, Dumais ST, O’Brien GW. Using linear algebra for intelligent information retrieval. SIAM review. 1995; 37(4):573–595. · DOI · Paper page
  13. Lee M, Mimno D. Low-dimensional embeddings for interpretable anchor-based topic inference. In: Proceedings of Empirical Methods in Natural Language Processing: 2014. · DOI
  14. Arora S, Ge R, Moitra A. Learning topic models—going beyond SVD. In: Foundations of Computer Science (FOCS), 2012 IEEE 53rd Annual Symposium On. New Brunswick, NJ: IEEE: 2012. p. 1–10.
  15. Bekhuis T, Demner-Fushman D. Towards automating the initial screening phase of a systematic review. Stud Health Technol Inform. 2010; 160(Pt 1):146–50.
  16. Octaviano FR, Felizardo KR, Maldonado JC, Fabbri SC. Semi-automatic selection of primary studies in systematic literature reviews: is it reasonable?Empir Softw Eng. 2015; 20(6):1898–1917. · DOI
  17. Cohen AM. Optimizing feature representation for automated systematic review work prioritization. In: AMIA Annual Symposium Proceedings: 2008. p. 121. American Medical Informatics Association.
  18. Romero Felizardo K, Souza SR, Maldonado JC. The use of visual text mining to support the study selection activity in systematic literature reviews: a replication study. In: Replication in empirical software engineering research (RESER), 2013 3rd International Workshop On. Baltimore, MD: IEEE: 2013. p. 91–100.
  19. Fiszman M, Bray BE, Shin D, Kilicoglu H, Bennett GC, Bodenreider O, et al.Combining relevance assignment with quality of the evidence to support guideline development. Stud Health Technol Inform. 2010; 160(01):709.
  20. Frunza O, Inkpen D, Matwin S. Building systematic reviews using automatic text classification techniques. In: Proceedings of the 23rd International Conference on Computational Linguistics: Posters. Beijing, China: Association for Computational Linguistics: 2010. p. 303–11.
  21. Jonnalagadda S, Petitti D. A new iterative method to reduce workload in systematic review process. Int J Comput Biol Drug Des. 2013; 6(1):5–17. · DOI · Paper page
  22. Frantzi K, Ananiadou S, Mima H. Automatic recognition of multi-word terms: the C-value/NC-value method. Int J Digit Libr. 2000; 3(2):115–30. · DOI · Paper page
  23. Chang CC, Lin CJ. LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST). 2011; 2(3):27.
  24. Řehuřek R, Sojka P. Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. Valletta, Malta: ELRA: 2010. p. 45–50. http://is.muni.cz/publication/884893/en .
  25. Kotsiantis S, Kanellopoulos D, Pintelas P. Handling imbalanced datasets: a review. GESTS Int Trans Comput Sci Eng. 2006; 30(1):25–36.
  26. Akbani R, Kwek S, Japkowicz N. Applying support vector machines to imbalanced datasets. In: Machine learning: ECML 2004. Berlin Heidelbeg: Springer: 2004. p. 39–50.

Cited by

See all 83

Metadata

Authors
Mo, Y., Kontonatsios, G., & Ananiadou, S.
Venue
Systematic Reviews
Published
2015-11-26
Volume
4
Issue
1
Pages
172-172
ISSN
2046-4053 (ISSN-L)
OpenAlex
W2180673283
Cited by
83
References
11
FWCI
0.82
Type
journal article
Language
en
Open access
gold

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Supporting systematic reviews using LDA-based document representations. https://app.cochat.ai/papers/10.1186/s13643-015-0117-0
The paper
Mo, Y., Kontonatsios, G., & Ananiadou, S. (2015). Supporting systematic reviews using LDA-based document representations. Systematic Reviews. https://doi.org/10.1186/s13643-015-0117-0

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.