The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Clinical concept extraction: A methodology review
Open accessAbstract
Background Concept extraction, a subdomain of natural language processing (NLP) with a focus on extracting concepts of interest, has been adopted to computationally extract clinical information from text for a wide range of applications ranging from clinical decision support to care quality improvement. Objectives In this literature review, we provide a methodology review of clinical concept extraction, aiming to catalog development processes, available methods and tools, and specific considerations when developing clinical concept extraction applications. Methods Based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, a literature search was conducted for retrieving EHR-based information extraction articles written in English and published from January 2009 through June 2019 from Ovid MEDLINE In-Process & Other Non-Indexed Citations, Ovid MEDLINE, Ovid EMBASE, Scopus, Web of Science, and the ACM Digital Library. Results A total of 6,686 publications were retrieved. After title and abstract screening, 228 publications were selected. The methods used for developing clinical concept extraction applications were discussed in this review.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Deep learning for computer vision: A brief review
Voulodimos, A., Doulamis, N., Doulamis, A., & Protopapadakis, E. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/7068349 · Cited by 3,403
The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
Hochreiter, S. (1998) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488598000094 · Cited by 2,939
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2017) · arXiv · doi:10.48550/arxiv.1707.08114 · Cited by 2,910
A survey of named entity recognition and classification
Nadeau, D., & Sekine, S. (2007) · Lingvisticae Investigationes · doi:10.1075/li.30.1.03nad · Cited by 2,507 · cited 5×
“This is evident as many of the researchers combined conditional random field (CRF) with various deep neural networks (word embeddings input) to improve the performance on NER, such as Bi-LSTM-CRF (44%), Bi-LSTM-Attention-CRF (22%), and CNN-CRF (22%).”
BackgroundInfluentialDeep learning for visual understanding: A review
Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.09.116 · Cited by 2,504
Bias in error estimation when using cross-validation for model selection
Varma, S., & Simon, R. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-91 · Cited by 2,485 · cited 1×
“The nested-cross-validation that uses two independent loops of CV for parameter tuning and error estimation can reduce the bias of the true error estimation (192).”
MethodSavova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967 · cited 2×
“The well-established clinical NLP tools such as MedLEE (113), MetaMap (114), cTAKES (115) and MedTagger (116) can achieve normalization using diverse approaches like a dictionary-lookup method and rule-based methods applying pattern recognition.”
MethodSelf-citationAlsentzer, E., Murphy, J., Boag, W., Weng, W.-H., Jindi, D., Naumann, T., & McDermott, M. (2019) · Proceedings of the 2nd Clinical Natural Language Processing Workshop · doi:10.18653/v1/w19-1909 · Cited by 1,736 · cited 1×
“The current state-of-the-art models use the long short-term memory (LSTM) (40) variant of bidirectional recurrent neural networks (BiRNN) with a subsequent conditional random field (CRF) decoding layer (15, 22, 23, 41).”
MethodThe elements of statistical learning: Data mining, inference, and prediction
Franklin, J. (2005) · The Mathematical Intelligencer · doi:10.1007/bf02985802 · Cited by 1,400
Savova, G. K., Kipper-Schuler, K. C., Hurdle, J. F., & Meystre, S. M. (2008) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638592 · Cited by 895 · cited 1×
“A review done by Meystre et al. in 2008 observed an increasing utilization of NLP in the clinical domain and a major challenge in advancing clinical NLP due to the unavailability of a large amount of clinical text (11).”
BackgroundThe epistemology of a rule-based expert system —a framework for explanation
Clancey, W. J. (1983) · Artificial Intelligence · doi:10.1016/0004-3702(83)90008-5 · Cited by 712 · cited 1×
“Symbolic rule-based concept extraction methodologies use a comprehensive set of rules and keyword-based features to identify pre-defined patterns in text (58, 121).”
MethodInterpretable Machine Learning in Healthcare
Ahmad, M. A., Eckert, C., & Teredesai, A. (2018) · Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics · doi:10.1145/3233547.3233667 · Cited by 583 · cited 1×
“In a practice setting, these explanations may serve as important criteria for safety and bias evaluations, and are usually referred to as a key factor of “user trust” (203, 204).”
BackgroundGated Feedback Recurrent Neural Networks
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.02367 · Cited by 583 · cited 1×
“RNNs are neural networks which explicitly model connections along a sequence, making RNNs uniquely suited for tasks that require long-term dependency to be captured (181, 182).”
BackgroundAchieving a Nationwide Learning Health System
Friedman, C. P., Wong, A. K., & Blumenthal, D. (2010) · Science Translational Medicine · doi:10.1126/scitranslmed.3001456 · Cited by 539
Health Information Technology: An Updated Systematic Review With a Focus on Meaningful Use
Jones, S. S., Rudin, R. S., Perry, T., & Shekelle, P. G. (2014) · Annals of Internal Medicine · doi:10.7326/m13-1531 · Cited by 475
Chapman, W. W., Nadkarni, P. M., Hirschman, L., D'Avolio, L. W., Savova, G. K., & Uzuner, O. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000465 · Cited by 296 · cited 1×
“However, these questions are further challenged by the stringent privacy and security requirements posed by the regulations surrounding protected health information, thereby limiting ability to share data or produce fruitful collaborations (215).”
BackgroundA neural network multi-task learning approach to biomedical named entity recognition
Crichton, G., Pyysalo, S., Chiu, B., & Korhonen, A. (2017) · BMC Bioinformatics · doi:10.1186/s12859-017-1776-8 · Cited by 229
Patrick, J., & Li, M. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.003939 · Cited by 159
Wen, A., Fu, S., Moon, S., El Wazir, M., Rosenbaum, A., Kaggal, V. C., Liu, S., Sohn, S., Liu, H., & Fan, J. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0208-8 · Cited by 142 · cited 1×
“…the model into their institutional IT infrastructure, which includes an ETL process for document or HL7 message retrieval, a parser that does document pre-processing, an engine to host and run rule-based or traditional machine learning models, and a database to store extracted results (214).”
BackgroundSelf-citationA hybrid system for temporal information extraction from clinical text
Tang, B., Wu, Y., Jiang, M., Chen, Y., Denny, J. C., & Xu, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001635 · Cited by 117
Selected articles from the BioCreative/OHNLP challenge 2018
Liu, S., Wang, Y., & Liu, H. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0994-6 · Cited by 10
Self-citation
Cited by
Artificial Intelligence in the Intensive Care Unit
Greco, M., Caruso, P. F., & Cecconi, M. (2021) · Seminars in Respiratory and Critical Care Medicine · doi:10.1055/s-0040-1719037 · Cited by 59
Fu, S., Vassilaki, M., Ibrahim, O. A., Petersen, R. C., Pagali, S., St Sauver, J., Moon, S., Wang, L., Fan, J. W., Liu, H., et al. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.958539 · Cited by 15
Self-citationFedFSA: Hybrid and federated framework for functional status ascertainment across institutions
Fu, S., Jia, H., Vassilaki, M., Keloth, V. K., Dang, Y., Zhou, Y., Garg, M., Petersen, R. C., St Sauver, J., Moon, S., et al. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104623 · Cited by 17
Self-citationvan Slobbe, R., Herrmannova, D., Boeke, D. J., Lima-Walton, E. S., Abu-Hanna, A., & Vagliano, I. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105815 · Cited by 7
Metadata
- Authors
- Fu, S., Chen, D., He, H., Liu, S., Moon, S., Peterson, K. J., Shen, F., Wang, L., Wang, Y., Wen, A., Zhao, Y., Sohn, S., & Liu, H.
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2020-09
- Volume
- 109
- Pages
- 103526
- ISSN
- 1532-0464 (ISSN-L)
- OpenAlex
- W3048179169
- Cited by
- 183
- References
- 28
- FWCI
- 6.36
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.