Evidence-based medicine, the essential role of systematic reviews, and the need for automated text mining tools
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
High quality, cost-effective medical care requires consideration of the best available, most appropriate evidence in the care of each patient, a practice known as Evidence-based Medicine (EBM). EBM is dependent upon the wide availability and coverage of accurate, objective syntheses called evidence reports (also called systematic reviews). These are compiled by a time and resource-intensive process that is largely manual, and that has not taken advantage of many of the advances in information processing technologies that have assisted other textual domains. We propose a specific text-mining based pipeline to support the creation and updating of evidence reports that provides support for the literature collection, collation, and triage steps of the systematic review process. The pipeline includes a metasearch engine that covers both bibliographic databases and selected "grey" literature; a module that classifies articles according to study type; a module for grouping studies that are closely related (e.g. that derive from the same underlying clinical trial or same study cohort); and an automated system that ranks publications according to the likelihood that they will meet inclusion criteria for the report. The proposed pipeline will also enable groups performing systematic review to reuse tools and models created by other groups, and will provide a test-bed for further informatics research to develop improved tools in the future. Ultimately, this should increase the rate that high-quality systematic reviews and meta-analyses can be generated, accessed and utilized by clinicians, patients, care-givers, and policymakers, resulting in better and more cost-effective care.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Solving Multiclass Learning Problems via Error-Correcting Output Codes
Dietterich, T. G., & Bakiri, G. (1995) · Journal of Artificial Intelligence Research · doi:10.1613/jair.105 · Cited by 2,739
Reducing Workload in Systematic Review Preparation Using Automated Citation Classification
Cohen, A. M., Hersh, W. R., Peterson, K., & Yen, P.-Y. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1929 · Cited by 411
Self-citationAtkins, D., Fink, K., & Slutsky, J. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-12_part_2-200506211-00002 · Cited by 129
Towards Automatic Recognition of Scientifically Rigorous Clinical Research Evidence
Kilicoglu, H., Demner-Fushman, D., Rindflesch, T. C., Wilczynski, N. L., & Haynes, R. B. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2996 · Cited by 101 · cited 1×
“Kilicoglu et al. ( Kilicoglu, Demner-Fushman, Rindflesch, Wilczynski, & Haynes, 2008, 2009 ) and Aphinyanaphongs et al. (Aphinyanaphongs & Aliferis, 2003; Aphinyanaphongs, Statnikov, & Aliferis, 2006) have trained machine learning systems to identify studies that are high quality according to EBM standards of evidence.”
BackgroundCross-Topic Learning for Work Prioritization in Systematic Review Creation and Update
Cohen, A. M., Ambert, K., & McDonagh, M. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3162 · Cited by 88 · cited 2×
“In this way, the ranking models can be productively used by any potential user, although the models will perform better when customized based on past review work relevant to that user, review team, and systematic review topic (A. M. Cohen, et al., 2009 ).”
MethodSelf-citationAutomatically Identifying Health Outcome Information in MEDLINE Records
Demner-Fushman, D., Few, B., Hauser, S. E., & Thoma, G. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1911 · Cited by 35 · cited 1×
“Similarly, the proposed system does not specifically address the issue of automatically extracting high quality information present within summaries or full-text of individual articles ( Demner-Fushman, Few, Hauser, & Thoma, 2006; Mendonca & Cimino, 2000), nor does it include automated methods of synthesizing and weig…”
Method
Cited by
Systematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472
“The process of creating a systematic review can be broken down into between four and 15 tasks depending on resolution [5,10,13,15].”
BackgroundAutomating data extraction in systematic reviews: a systematic review
Jonnalagadda, S. R., Goyal, P., & Huffman, M. D. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0066-7 · Cited by 287
“Cohen et al. [52] proposed a module for grouping studies that are closely related and an automated system to rank publications according to the likelihood for meeting the inclusion criteria of a systematic review.”
MethodBannach-Brown, A., Przybyła, P., Thomas, J., Rice, A. S. C., Ananiadou, S., Liao, J., & Macleod, M. R. (2019) · Systematic Reviews · doi:10.1186/s13643-019-0942-7 · Cited by 185
“Researchers find it increasingly difficult to keep up with new findings and discoveries even within a single biomedical domain, an issue that has been emerging for a number of years [2].”
BackgroundShemilt, I., Simon, A., Hollands, G. J., Marteau, T. M., Ogilvie, D., O'Mara‐Eves, A., Kelly, M. P., & Thomas, J. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1093 · Cited by 170
Cohen, A. M., Smalheiser, N. R., McDonagh, M. S., Yu, C., Adams, C. E., Davis, J. M., & Yu, P. S. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu025 · Cited by 78
Self-citation
Metadata
- Authors
- Cohen, A. M., Adams, C. E., Davis, J. M., Yu, C., Yu, P. S., Meng, W., Duggan, L., McDonagh, M., & Smalheiser, N. R.
- Venue
- Proceedings of the 1st ACM International Health Informatics Symposium
- Publisher
- ACM
- Published
- 2010-11-11
- Pages
- 376-380
- OpenAlex
- W2087432624
- Cited by
- 45
- References
- 6
- FWCI
- 1.13
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evidence-based medicine, the essential role of systematic reviews, and the need for automated text mining tools. https://app.cochat.ai/papers/10.1145/1882992.1883046
- The paper
- Cohen, A. M., Adams, C. E., Davis, J. M., Yu, C., Yu, P. S., Meng, W., Duggan, L., McDonagh, M., & Smalheiser, N. R. (2010). Evidence-based medicine, the essential role of systematic reviews, and the need for automated text mining tools. Proceedings of the 1st ACM International Health Informatics Symposium. https://doi.org/10.1145/1882992.1883046
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.