Extracting PICO elements from RCT abstracts using 1-2gram analysis and multitask classification.
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
The core of evidence-based medicine is to read and analyze numerous papers in the medical literature on a specific clinical problem and summarize the authoritative answers to that problem. Currently, to formulate a clear and focused clinical problem, the popular PICO framework is usually adopted, in which each clinical problem is considered to consist of four parts: patient/problem (P), intervention (I), comparison (C) and outcome (O). In this study, we compared several classification models that are commonly used in traditional machine learning. Next, we developed a multitask classification model based on a soft-margin SVM with a specialized feature engineering method that combines 1-2gram analysis with TF-IDF analysis. Finally, we trained and tested several generic models on an open-source data set from BioNLP 2018. The results show that the proposed multitask SVM classification model based on 1-2gram TF-IDF features exhibits the best performance among the tested models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“39,54,87,95,98,136”
From Data extraction methods for systematic review (semi)automation: Update of a living systematic review (2025) · cites this paper as [87]
Citations
Cites
Evidence based medicine: what it is and what it isn't
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996) · BMJ · doi:10.1136/bmj.312.7023.71 · Cited by 15,769
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
Evidence-Based Medicine: How to Practice and Teach EBM
Sackett, D. L. (2000) · Cited by 5,950
Yeh, M.-L. (2014) · Journal of Nursing Research · doi:10.1097/jnr.0000000000000065 · Cited by 3,829
Utilization of the PICO framework to improve searching PubMed for clinical questions
Schardt, C., Adams, M. B., Owens, T., Keitz, S., & Fontelo, P. (2007) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-7-16 · Cited by 3,053
The well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931 · cited 3×
“has often been incorporated into the “I” category because “comparative” elements can refer to other interventions or to the decision not to participate in clinical randomized controlled trials (RCTs) [4], which should be addressed in the intervention category. In fact, very few abstracts with comparison labels are fou…”
InfluentialMethley, A. M., Campbell, S., Chew-Graham, C., McNally, R., & Cheraghi-Sohi, S. (2014) · BMC Health Services Research · doi:10.1186/s12913-014-0579-0 · Cited by 2,706 · cited 1×
“compared the PICO search method with several other search methods and found that the PICO method can significantly improve the efficiency of a literature search [5].”
Data Mining and Knowledge Discovery Handbook
(2010) · doi:10.1007/978-0-387-09823-4 · Cited by 2,190
Evidence-based Medicine: How to Practice and Teach EBM
Satya-Murti, S. (1997) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1997.03550020100049 · Cited by 975
Evaluation of PICO as a knowledge representation for clinical questions.
Huang, X., Lin, J., & Demner‐Fushman, D. (2006) · Europe PMC (PubMed Central) · Cited by 898
Caruana, R. (1998) · Learning to Learn · doi:10.1007/978-1-4615-5529-2_5 · Cited by 577
Answering Clinical Questions with Knowledge-Based and Statistical Techniques
Demner-Fushman, D., & Lin, J. (2007) · Computational Linguistics · doi:10.1162/coli.2007.33.1.63 · Cited by 296
Combining classifiers for robust PICO element detection.
Boudin, F., Nie, J.-Y., Bartlett, J. C., Grad, R., Pluye, P., & Dawes, M. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-29 · Cited by 131 · cited 2×
“2010 [11] proposed a combination of multiple classifier models (Multi-Layer Perceptron, MLP) using weighted linear combinations of predicted scores.”
Categorization of sentence types in medical abstracts.
McKnight, L., & Srinivasan, P. (2003) · PubMed · Cited by 126
Sentence retrieval for abstracts of randomized controlled trials.
Chung, G. Y. (2009) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-9-10 · Cited by 83
Screening nonrandomized studies for medical systematic reviews: a comparative study of classifiers
Bekhuis, T., & Demner-Fushman, D. (2012) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2012.05.002 · Cited by 70 · cited 1×
“ed excellent results, this method of pattern matching based on expert annotation has a fatal shortcoming, namely, the requirement for cumbersome manual labeling, which limits the size of the data set [14]. In addition, there may be differences between the annotator and the information seeker in terms of analysis or em…”
Neural Networks for Joint Sentence Classification in Medical Paper Abstracts
Dernoncourt, F., Lee, J. Y., & Szolovits, P. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2110 · Cited by 68
Rathbone, J., Albarqouni, L., Bakhit, M., Beller, E., Byambasuren, O., Hoffmann, T., Scott, A. M., & Glasziou, P. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0629-x · Cited by 64
PICO Element Detection in Medical Text via Long Short-Term Memory Neural Networks.
Jin, D., & Szolovits, P. (2018) · Proceedings of the BioNLP 2018 workshop · doi:10.18653/v1/w18-2308 · Cited by 61 · cited 3×
“The data set used throughout this study is an opensource data set from BioNLP 2018 [11].”
InfluentialThe role of knowledge in conceptual retrieval: a study in the domain of clinical medicine
Lin, J., & Demner-Fushman, D. (2006) · Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1148170.1148191 · Cited by 59
PICO element detection in medical text without metadata: are first sentences enough?
Huang, K.-C., Chiang, I.-J., Xiao, F., Liao, C.-C., Liu, C. C.-H., & Wong, J.-M. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.07.009 · Cited by 52
Chung, G. Y.-C. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.12.011 · Cited by 34 · cited 2×
“2009 [14] proposed an automated text mining method for automatically identifying intervention elements in abstracts in the RCT literature.”
Classification of PICO elements by text features systematically extracted from PubMed abstracts.
Huang, K.-C., Liu, C. C.-H., Yang, S.-S., Xiao, F., Wong, J.-M., Liao, C.-C., & Chiang, I.-J. (2011) · 2011 IEEE International Conference on Granular Computing · doi:10.1109/grc.2011.6122608 · Cited by 26 · cited 8×
“We designed three separate binary classification models for P, I and O classification to mitigate the possible effects of the imbalanced data distribution on the model classification results [23][24] and to avoid sentences with fuzzy labels being assigned as negative samples of non-P/I/O labels, which could affect the…”
Influential
Research works printed but not linked: 2
- Varantola K." Special Language and General Language". ALSED2LSP Newsletter,1986,9 (2): 10220
- Rosita B. Maglie." Understanding the Language of Medicine". Springer Aracne; 2009. p.12--25.
Cited by
Data extraction methods for systematic review (semi)automation: a living systematic review
Schmidt, L., Olorisade, B. K., McGuinness, L. A., Thomas, J., & Higgins, J. P. T. (2021) · F1000Research · doi:10.12688/f1000research.51117.1 · Cited by 106
Santos, Á. O. D., da Silva, E. S., Couto, L. M., Reis, G. V. L., & Belo, V. S. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2023.104389 · Cited by 64
A deep learning classifier for sentence classification in biomedical and computer science abstracts
Gonçalves, S., Cortez, P. C., & Moro, S. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04334-2 · Cited by 68
Data extraction methods for systematic review (semi)automation: Update of a living systematic review
Schmidt, L., Finnerty Mutlu, A. N., Elmore, R., Olorisade, B. K., Thomas, J., & Higgins, J. P. T. (2023) · F1000Research · doi:10.12688/f1000research.51117.2 · Cited by 44
Aceso: PICO-Guided Evidence Summarization on Medical Literature.
Zhang, X., Geng, P., Zhang, T., Lu, Q., Gao, P., & Mei, J. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2984704 · Cited by 25
Metadata
- Authors
- Yuan, X., Xiaoli, L., Shilei, L., Qinwen, S., & Ke, L.
- Affiliations
- School of Life Science & Technology, University of Electronic Science & Technology of China
- University of Electronic Science and Technology of China (CN)
- Venue
- Proceedings of the third International Conference on Medical and Health Informatics 2019 - ICMHI 2019
- Publisher
- ACM Press
- Published
- 2019
- Pages
- 194-199
- OpenAlex
- W2966702003
- Cited by
- 10
- References
- 25
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 2 · 2021: 2 · 2020: 2 · 2019: 1
- FWCI
- 3.77
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- International Conference on Medical and Health Informatics (ICMHI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Text and Document Classification Technologies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- PICO framework, soft-margin SVM, evidence-based medicine, biomedical text classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.