The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
A study of machine-learning-based approaches to extract clinical entities and their assertions from discharge summaries
Open accessAbstract
OBJECTIVE: The authors' goal was to develop and evaluate machine-learning-based approaches to extracting clinical entities-including medical problems, tests, and treatments, as well as their asserted status-from hospital discharge summaries written using natural language. This project was part of the 2010 Center of Informatics for Integrating Biology and the Bedside/Veterans Affairs (VA) natural-language-processing challenge. DESIGN: The authors implemented a machine-learning-based named entity recognition system for clinical text and systematically evaluated the contributions of different types of features and ML algorithms, using a training corpus of 349 annotated notes. Based on the results from training data, the authors developed a novel hybrid clinical entity extraction system, which integrated heuristic rule-based modules with the ML-base named entity recognition module. The authors applied the hybrid system to the concept extraction and assertion classification tasks in the challenge and evaluated its performance using a test data set with 477 annotated notes. MEASUREMENTS: Standard measures including precision, recall, and F-measure were calculated using the evaluation script provided by the Center of Informatics for Integrating Biology and the Bedside/VA challenge organizers. The overall performance for all three types of clinical entities and all six types of assertions across 477 annotated notes were considered as the primary metric in the challenge. RESULTS AND DISCUSSION: Systematic evaluation on the training set showed that Conditional Random Fields outperformed Support Vector Machines, and semantic information from existing natural-language-processing systems largely improved performance, although contributions from different types of features varied. The authors' hybrid entity extraction system achieved a maximum overall F-score of 0.8391 for concept extraction (ranked second) and 0.9313 for assertion classification (ranked fourth, but not statistically different than the first three systems) on the test data set in the challenge.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Advances in kernel methods: support vector learning
Schölkopf, B., Burges, C. J. C., & Smola, A. J. (1999) · doi:10.5555/299094 · Cited by 5,789
Making Large-Scale SVM Learning Practical
Joachims, T. (1998) · RePEc: Research Papers in Economics · doi:10.17877/de290r-14262 · Cited by 4,411
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967 · cited 1×
“are a few of the earliest NLP systems developed for clinical domain. More recently, open source clinical NLP systems such as cTAKES”
An overview of MetaMap: historical perspective and recent advances
Aronson, A. R., & Lang, F.-M. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.002733 · Cited by 1,519
The Unified Medical Language System
Humphreys, B. L., McCray, A. T., & Lindberg, D. A. B. (1993) · Methods of Information in Medicine · doi:10.1055/s-0038-1634945 · Cited by 1,182
A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries
Chapman, W. W., Bridewell, W., Hanbury, P., Cooper, G. F., & Buchanan, B. G. (2001) · Journal of Biomedical Informatics · doi:10.1006/jbin.2001.1029 · Cited by 1,118
A General Natural-language Text Processor for Clinical Radiology
Friedman, C., Alderson, P. O., Austin, J. H. M., Cimino, J. J., & Johnson, S. B. (1994) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1994.95236146 · Cited by 778
MedEx: a medication information extraction system for clinical narratives
Xu, H., Stenner, S. P., Doan, S., Johnson, K. B., Waitman, L. R., & Denny, J. C. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3378 · Cited by 531
Self-citationChunking with support vector machines
Kudo, T., & Matsumoto, Y. (2001) · Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001 - NAACL '01 · doi:10.3115/1073336.1073361 · Cited by 520
Text categorization with support vector machines
Joachims, T. (1997) · Technische Universität Dortmund Eldorado (Technische Universität Dortmund) · doi:10.17877/de290r-5097 · Cited by 507
The Unified Medical Language System
Humphreys, B. L., McCray, A. T., & Lindberg, D. A. B. (1993) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1637976 · Cited by 507
Extracting medication information from clinical text
Uzuner, Ö., Solti, I., & Cadag, E. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.003947 · Cited by 466
Sang, E. F. T. K., & Veenstra, J. (1999) · Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics - · doi:10.3115/977035.977059 · Cited by 456
Text Categorization with Support Vector Machines. How to Represent Texts in Input Space?
Leopold, E., & Kindermann, J. (2002) · Machine Learning · doi:10.1023/a:1012491419635 · Cited by 403
Example selection for bootstrapping statistical parsers
Steedman, M., Hwa, R., Clark, S., Osborne, M., Sarkar, A., Hockenmaier, J., Ruhlen, P., Baker, S., & Crim, J. (2003) · Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology - NAACL '03 · doi:10.3115/1073445.1073476 · Cited by 364
Zeng, Q. T., Goryachev, S., Weiss, S., Sordo, M., Murphy, S. N., & Lazarus, R. (2006) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-6-30 · Cited by 360
ProMiner: rule-based protein and gene entity recognition
Hanisch, D., Fundel, K., Mevissen, H.-T., Zimmer, R., & Fluck, J. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-s1-s14 · Cited by 346
Unlocking Clinical Data from Narrative Reports: A Study of Natural Language Processing
Hripcsak, G., Friedman, C., Alderson, P. O., DuMouchel, W., Johnson, S. B., & Clayton, P. D. (1995) · Annals of Internal Medicine · doi:10.7326/0003-4819-122-9-199505010-00007 · Cited by 339
LIBLINEAR: A Library for Large Linear Classification
FanRong-En, ChangKai-Wei, HsiehCho-Jui, WangXiang-Rui, & LinChih-Jen (2008) · Journal of Machine Learning Research · doi:10.5555/1390681.1442794 · Cited by 328
Term identification in the biomedical literature
Krauthammer, M., & Nenadic, G. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.08.004 · Cited by 304
Use of support vector learning for chunk identification
Kudoh, T., & Matsumoto, Y. (2000) · Conference on Computational Natural Language Learning · doi:10.3115/1117601.1117635 · Cited by 287
Tuning support vector machines for biomedical named entity recognition
Kazama, J., Makino, T., Ohta, Y., & Tsujii, J. (2002) · doi:10.3115/1118149.1118150 · Cited by 258
Recognizing names in biomedical texts: a machine learning approach
Zhou, G., Zhang, J., Su, J., Shen, D., & Tan, C. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth060 · Cited by 251
Mutalik, P. G., Deshpande, A., & Nadkarni, P. M. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080598 · Cited by 249
MedPost: a part-of-speech tagger for bioMedical text
Smith, L., Rindflesch, T., & Wilbur, W. J. (2004) · Bioinformatics · doi:10.1093/bioinformatics/bth227 · Cited by 224
Hripcsak, G., Austin, J. H. M., Alderson, P. O., & Friedman, C. (2002) · Radiology · doi:10.1148/radiol.2241011118 · Cited by 200
Medical Language Processing: Computer Management of Narrative Data
Sager, N., Friedman, C., & Lyman, M. S. (1987) · Medical Entomology and Zoology · Cited by 170
Patrick, J., & Li, M. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.003939 · Cited by 159
Evaluation of a Method to Identify and Categorize Section Headers in Clinical Documents
Denny, J. C., Spickard, A., Johnson, K. B., Peterson, N. B., Peterson, J. F., & Miller, R. A. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3037 · Cited by 125
Self-citationBio-medical entity extraction using support vector machines
Takeuchi, K., & Collier, N. (2004) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.07.019 · Cited by 113
Recognition of protein/gene names from text using an ensemble of classifiers
Zhou, G., Shen, D., Zhang, J., Su, J., & Tan, S. (2005) · BMC Bioinformatics · doi:10.1186/1471-2105-6-s1-s7 · Cited by 110
Li, D., Schuler, K. K., & Savova, G. (2008) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.3115/1572306.1572326 · Cited by 91
BioTagger-GM: A Gene/Protein Name Recognition System
Torii, M., Hu, Z., Wu, C. H., & Liu, H. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2844 · Cited by 81
Experience with a mixed semantic/syntactic parser.
Haug, P. J., Koehler, S., Lau, L. M. S., Wang, P., Rocha, R. A., & Huff, S. M. (1995) · PubMed Central · Cited by 75
Development and evaluation of a clinical note section header terminology.
(2008) · PubMed Central · Cited by 65
Lancet: a high precision medication event extraction system for clinical text
Li, Z., Liu, F., Antieau, L. D., Cao, Y.-G., & Yu, H. Q. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.004077 · Cited by 55
Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing
Demner‐Fushman, D. (2008) · Cited by 54
A natural language parsing system for encoding admitting diagnoses.
Haug, P. J., Christensen, L. M., Gundersen, M. L., Clemons, B., Koehler, S., & Bauer, K. (1997) · PubMed Central · Cited by 51
Automatic identification of pneumonia related concepts on chest x-ray reports.
Marcelo Fiszman, B., Chapman, W. W., Evans, S. R., & Haug, P. J. (1999) · PubMed · Cited by 42
Biological entity recognition with conditional random fields.
He, Y., & Kayaalp, M. (2008) · PubMed · Cited by 30
Syntactically-informed semantic category recognition in discharge summaries.
Sibanda, T. C., He, T. Y., Peter Szolovits, B., & Uzuner, Ö. (2006) · Europe PMC (PubMed Central) · Cited by 29
Cited by
Shickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2767063 · Cited by 1,688
Large language models are few-shot clinical information extractors
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.130 · Cited by 358
Improving large language models for clinical named entity recognition via prompt engineering
Hu, Y., Chen, Q., Du, J., Peng, X., Keloth, V. K., Zuo, X., Zhou, Y., Li, Z., Jiang, X., Lu, Z., et al. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad259 · Cited by 314
“35 Regarding the GPT models, we used the specific versions GPT-3.”
Self-citationElectronic health records-driven phenotyping: challenges, recent advances, and perspectives
Pathak, J., Kho, A. N., & Denny, J. C. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002428 · Cited by 274
Self-citationData Processing and Text Mining Technologies on Electronic Medical Records: A Review
Sun, W., Cai, Z., Li, Y., Liu, F., Fang, S., & Wang, G. (2018) · Journal of Healthcare Engineering · doi:10.1155/2018/4302425 · Cited by 287
Metadata
- Authors
- Jiang, M., Chen, Y., Liu, M., Rosenbloom, S. T., Mani, S., Denny, J. C., & Xu, H.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2011-09
- Volume
- 18
- Issue
- 5
- Pages
- 601-606
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2107435951
- Cited by
- 291
- References
- 42
- FWCI
- 5.76
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.