Automatic Detection of Acute Bacterial Pneumonia from Chest X-ray Reports
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: To evaluate the performance of a natural language processing system in extracting pneumonia-related concepts from chest x-ray reports. DESIGN: Four physicians, three lay persons, a natural language processing system, and two keyword searches (designated AAKS and KS) detected the presence or absence of three pneumonia-related concepts and inferred the presence or absence of acute bacterial pneumonia from 292 chest x-ray reports. Gold standard: Majority vote of three independent physicians. Reliability of the gold standard was measured. OUTCOME MEASURES: Recall, precision, specificity, and agreement (using Finn's R: statistic) with respect to the gold standard. Differences between the physicians and the other subjects were tested using the McNemar test for each pneumonia concept and for the disease inference of acute bacterial pneumonia. RESULTS: Reliability of the reference standard ranged from 0.86 to 0.96. Recall, precision, specificity, and agreement (Finn R:) for the inference on acute bacterial pneumonia were, respectively, 0.94, 0.87, 0.91, and 0.84 for physicians; 0.95, 0.78, 0.85, and 0.75 for natural language processing system; 0.46, 0.89, 0.95, and 0.54 for lay persons; 0.79, 0.63, 0.71, and 0.49 for AAKS; and 0.87, 0.70, 0.77, and 0.62 for KS. The McNemar pairwise comparisons showed differences between one physician and the natural language processing system for the infiltrate concept and between another physician and the natural language processing system for the inference on acute bacterial pneumonia. The comparisons also showed that most physicians were significantly different from the other subjects in all pneumonia concepts and the disease inference. CONCLUSION: In extracting pneumonia related concepts from chest x-ray reports, the performance of the natural language processing system was similar to that of physicians and better than that of lay persons and keyword searches. The encoded pneumonia information has the potential to support several pneumonia-related applications used in our institution. The applications include a decision support system called the antibiotic assistant, a computerized clinical protocol for pneumonia, and a quality assurance application in the radiology department.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Fiszman et al showed that introducing NLP for reading the chest X-ray reports would assist the antibiotic assistant system to alert physicians for the possible need for anti-infective therapy.43”
From Artificial intelligence in healthcare: past, present and future (2017) · cites this paper as [43]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1988) · Cited by 16,582
Generalizability theory: a primer
(1992) · Choice Reviews Online · doi:10.5860/choice.29-4192 · Cited by 1,298
Natural Language Understanding
(2021) · AI Assistants · doi:10.7551/mitpress/12761.003.0007 · Cited by 1,271
A Computer-Assisted Management Program for Antibiotics and Other Antiinfective Agents
Evans, R. S., Pestotnik, S. L., Classen, D. C., Clemmer, T. P., Weaver, L. K., Orme, J. F., Lloyd, J. F., & Burke, J. P. (1998) · New England Journal of Medicine · doi:10.1056/nejm199801223380406 · Cited by 1,061
Self-citationNiederman, M. S., Bass, J. B., Campbell, G. D., Fein, A. M., Grossman, R. F., Mandell, L. A., Marrie, T. J., Sarosi, G. A., Torres, A., & Yu, V. L. (1993) · American Review of Respiratory Disease · doi:10.1164/ajrccm/148.5.1418 · Cited by 1,045
A General Natural-language Text Processor for Clinical Radiology
Friedman, C., Alderson, P. O., Austin, J. H. M., Cimino, J. J., & Johnson, S. B. (1994) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1994.95236146 · Cited by 778
A COMPUTER-ASSISTED MANAGEMENT PROGRAM FOR ANTIBIOTICS AND OTHER ANTIINFECTIVE AGENTS
Vans, R. S. C. E., Estotnik, S. T. L. P., Lassen, D. A. C. C., Lemmer, T., Eaver, L. I. K. W., Rme, J. A. F. O., Loyd, J. A. F. L., & Urke, J. O. P. B. (1998) · Cited by 692
The Reliability of Clinical Methods, Data and Judgments: (First of Two Parts)
Koran, L. M. (1975) · New England Journal of Medicine · doi:10.1056/nejm197509252931307 · Cited by 649
Niederman (1993) · Medical Entomology and Zoology · Cited by 624
BRENNER, H., & GEFELLER, O. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970515)16:9<981::aid-sim510>3.0.co;2-n · Cited by 488
Research works printed but not linked: 6
- Friedman C, Alderson PO, Austin JH, Cimino JJ, Johnson SB. A general natural-language text processor for clinical radiology. J Am Med Inform Assoc. 1994;1:161–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sager N, Lyman M, Bucknall C, Nhan N, Tick LJ. Natural language processing and the representation of clinical data. J Am Med Inform Assoc. 1994;1:142–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hughes, 1988, M.D. computing : computers in medical practice, 5, 22
- Hripcsak G, Kuperman GJ, Friedman C, Heitjan DF. A reliability study for evaluating information extraction from radiology reports. J Am Med Inform Assoc. 1999;6:143–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zar JH. Biostatistical Analysis. Englewood Cliffs, N.J.: Prentice Hall, 1974.
- 1984, The American Psychologist, 39, 22, 10.1037/0003-066x.39.1.22 · doi:10.1037/0003-066x.39.1.22 · Paper page
Cited by
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
“For example, Fiszman et al showed that introducing NLP for reading the chest X-ray reports would assist the antibiotic assistant system to alert physicians for the possible need for anti-infective therapy.(43) Miller et al used NLP to automatically monitor the laboratory-based adverse effects.”
MethodSavova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967
A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries
Chapman, W. W., Bridewell, W., Hanbury, P., Cooper, G. F., & Buchanan, B. G. (2001) · Journal of Biomedical Informatics · doi:10.1006/jbin.2001.1029 · Cited by 1,118
Self-citationSavova, G. K., Kipper-Schuler, K. C., Hurdle, J. F., & Meystre, S. M. (2008) · Yearbook of Medical Informatics · doi:10.1055/s-0038-1638592 · Cited by 895
“They have been used to extract interpretations of lung scans [86], to detect pneumonia [87], and to detect mentions of central venous catheters [88].”
MethodWhat can natural language processing do for clinical decision support?
Demner-Fushman, D., Chapman, W. W., & McDonald, C. J. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.08.007 · Cited by 688
“mial infections [19], to biosurveillance [82].”
MethodResultInfluentialSelf-citationNatural language processing in radiology: a systematic review
Pons, E., Braun, L. M. M., Hunink, M. G. M., & Kors, J. A. (2016) · Radiology · doi:10.1148/radiol.16142770 · Cited by 573
A review of approaches to identifying patient phenotype cohorts using electronic health records
Shivade, C., Raghavan, P., Fosler-Lussier, E., Embi, P. J., Elhadad, N., Johnson, S. B., & Lai, A. M. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001935 · Cited by 531
MedEx: a medication information extraction system for clinical narratives
Xu, H., Stenner, S. P., Doan, S., Johnson, K. B., Waitman, L. R., & Denny, J. C. (2010) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3378 · Cited by 531
Rindflesch, T. C., & Fiszman, M. (2003) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2003.11.003 · Cited by 518
“SymText has been evaluated for clinical applications [6,30,31].”
BackgroundDetecting Adverse Events Using Information Technology
Bates, D. W., Evans, R. S., Murff, H., Stetson, P. D., Pizziferri, L., & Hripcsak, G. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1074 · Cited by 467
Self-citationDetecting adverse events for patient safety research: a review of current methodologies
Murff, H. J., Patel, V. L., Hripcsak, G., & Bates, D. W. (2003) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2003.08.003 · Cited by 357
Automated Detection of Adverse Events Using Natural Language Processing of Discharge Summaries
Melton, G. B., & Hripcsak, G. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1794 · Cited by 297
Unsupervised biomedical named entity recognition: Experiments with clinical and biological texts
Zhang, S., & Elhadad, N. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.08.004 · Cited by 249
“Examples of BM-NER systems include extracting clinical information from radiology reports [1–3], identifying diseases and drug names in discharge summaries [4–6], detecting gene and protein mentions in biomedical paper abstracts [7–9].”
BackgroundArtificial Intelligence in Healthcare: Review, Ethics, Trust Challenges & Future Research Directions
Kumar, P., Chauhan, S., & Awasthi, L. K. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105894 · Cited by 218
khan, Z. F., & Alotaibi, S. R. (2020) · Journal of Healthcare Engineering · doi:10.1155/2020/8894694 · Cited by 235
“Convolutional neural network [109] Combination of AI and natural language processing It reads the chest X-ray reports of patients and assists the antibiotic assistant system to alert physicians for anti-infective therapy.”
Natural Language Processing Technologies in Radiology Research and Clinical Applications
Cai, T., Giannopoulos, A. A., Yu, S., Kelil, T., Ripley, B., Kumamaru, K. K., Rybicki, F. J., & Mitsouras, D. (2016) · RadioGraphics · doi:10.1148/rg.2016150080 · Cited by 214
“Fiszman et al (37) used a system called SymText to detect acute bacterial infection in chest radiography reports to replace a keyword search (“pne-moni * ,” “aspirati * ,” “infiltr * ”) previously used in a CDS system that aids in selection of appropriate antibiotics in conjunction with laboratory…”
MethodResultInfluentialNatural Language Processing for EHR-Based Computational Phenotyping
Zeng, Z., Deng, Y., Li, X., Naumann, T., & Luo, Y. (2019) · IEEE/ACM Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2018.2849968 · Cited by 209
“[75] Self-defined rules SymText Chest x-ray reports Set of augmented transition network grammars and a lexicon derived from the specialist Lexicon Garla et al.”
BackgroundRea, S., Pathak, J., Savova, G., Oniki, T. A., Westberg, L., Beebe, C. E., Tao, C., Parker, C. G., Haug, P. J., Huff, S. M., et al. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.01.009 · Cited by 206
“Traditionally, IE and NLP have been applied to the domain of medicine for specific use cases where only a limited set of signals are to be discovered, such as discovering pneumonia from chest X-ray reports[23, 24] and peripheral arterial disease from radiology notes.”
BackgroundSelf-citationHripcsak, G., Austin, J. H. M., Alderson, P. O., & Friedman, C. (2002) · Radiology · doi:10.1148/radiol.2241011118 · Cited by 200
Evaluation of negation phrases in narrative clinical reports.
Chapman, W. W., Bridewell, W. B., Hanbury, P., Cooper, G. F., & Buchanan, B. G. (2001) · PubMed · Cited by 175
Self-citationPakhomov, S. V. S., Buntrock, J. D., & Chute, C. G. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2077 · Cited by 161
Computer Algorithms To Detect Bloodstream Infections
Trick, W. E., Zagorski, B. M., Tokars, J. I., Vernon, M. O., Welbel, S. F., Wisniewski, M. F., Richards, C., & Weinstein, R. A. (2004) · Emerging Infectious Diseases · doi:10.3201/eid1009.030978 · Cited by 162
“The specificity of automated primary bloodstream infection detection could be improved by interpreting radiology reports or using International Classification of Disease codes to automate pneumonia detection (14).”
MethodNotice of Retraction: AI Techniques for COVID-19
Hussain, A. A., Bouachir, O., Al-Turjman, F., & Aloqaily, M. (2020) · IEEE Access · doi:10.1109/access.2020.3007939 · Cited by 166
“Authors in [109] utilized this method in screening the research center based on unfavorable impacts.”
MethodRetractedTemporal reasoning with medical data—A review with emphasis on medical natural language processing
Zhou, L., & Hripcsak, G. M. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2006.12.009 · Cited by 153
“Its later version, Symbolic Text Processor (SymText) [215], has been applied to encode admission diagnoses [216] and extract pneumonia-related concepts from chest x-ray reports [217].”
Chapman, B. E., Lee, S., Kang, H. P., & Chapman, W. W. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.03.011 · Cited by 152
“Statistical, symbolic, and hybrids have been applied to text classification of clinical reports [32,6,36,7,18,39].”
BackgroundSelf-citationRule-based information extraction from patients’ clinical data
Mykowiecka, A., Marciniak, M., & Kupść, A. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.07.007 · Cited by 149
“, [17,25,21].”
Chapman, W. W., Christensen, L. M., D, M. M. W. M., Haug, P. J., Ivanov, O., Dowling, J., & Olszewski, R. T. (2004) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.04.001 · Cited by 146
“Mþ has previously been applied to the domains of chest radiography [8] and brain CT scans.”
Self-citationNatural language-based machine learning models for the annotation of clinical radiology reports
Zech, J., Pain, M., Titano, J., Badgeley, M., Schefflein, J., Su, A., Costa, A., Bederson, J., Lehar, J., & Oermann, E. K. (2018) · Radiology · doi:10.1148/radiol.2018171093 · Cited by 142
ConText: an algorithm for identifying contextual features from clinical text
Chapman, W. W., Chu, D., & Dowling, J. N. (2007) · Proceedings of the Workshop on BioNLP 2007 Biological, Translational, and Clinical Language Processing - BioNLP '07 · doi:10.3115/1572392.1572408 · Cited by 136
“forming decision support, quality assurance, and biosurveillance [1-6].”
BackgroundSelf-citationPakhomov, S., Weston, S. A., Jacobsen, S. J., Chute, C. G., Meverden, R. A., & Roger, V. L. (2007) · PubMed · Cited by 122
Chapman, W. W., Fizman, M., Chapman, B. E., & Haug, P. J. (2001) · Journal of Biomedical Informatics · doi:10.1006/jbin.2001.1000 · Cited by 127
“We describe elsewhere how well one of the expert systems (the rules) performed on SymText’s uncorrected parsed data [33].”
MethodSelf-citationAutomation of a problem list using natural language processing
Meystre, S., & Haug, P. J. (2005) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-5-30 · Cited by 112
“Here we have focused on reports for chest radiographs [58-60] with a further focus on the data in these reports that supports the diagnosis of pneumonia [ 64 ,65].”
Self-citationMendonça, E. A., Haas, J., Shagina, L., Larson, E., & Friedman, C. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.02.003 · Cited by 111
Mahajan, A., Vaidya, T. P., Gupta, A., Rane, S. U., & Gupta, S. (2019) · Cancer Research Statistics and Treatment · doi:10.4103/crst.crst_50_19 · Cited by 113
“[25] Another example is the use of introducing natural language processing for extracting pneumonia‐related concepts from chest X‐ray reports which would then assist the antibiotic assistant system to alert physicians to the possible need for anti‐infective therapy.[26,27] Clinical medicine requires doctors to handle…”
BackgroundHazlehurst, B. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1771 · Cited by 102
Afzal, N., Sohn, S., Abram, S., Scott, C. G., Chaudhry, R., Liu, H., Kullo, I. J., & Arruda-Olson, A. M. (2017) · Journal of Vascular Surgery · doi:10.1016/j.jvs.2016.11.031 · Cited by 103
Natural language processing of clinical notes for identification of critical limb ischemia
Afzal, N., Mallipeddi, V. P., Sohn, S., Liu, H., Chaudhry, R., Scott, C. G., Kullo, I. J., & Arruda-Olson, A. M. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2017.12.024 · Cited by 94
Turchin, A., Kolatkar, N. S., Grant, R. W., Makhni, E. C., Pendergrass, M. L., & Einbinder, J. S. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2078 · Cited by 98
Machine Learning and Rule-based Approaches to Assertion Classification
Uzuner, Ö., Zhang, X., & Sibanda, T. C. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2950 · Cited by 99
“They found that SymText’s performance on this task “was similar to that of physi-cians” 1 ; its performance was better than that of keyword search systems when these systems considered a concept to be present unless it was accompanied by an explicit negation.”
BackgroundAutomatic Prediction of Rheumatoid Arthritis Disease Activity from the Electronic Medical Records
Lin, C., Karlson, E. W., Canhao, H., Miller, T. A., Dligach, D., Chen, P. J., Perez, R. N. G., Shen, Y., Weinblatt, M. E., Shadick, N. A., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0069932 · Cited by 98
“There are tools built for specific tasks such as SymText utilized in identifying pneumonia-related concepts and finding pneumonia-supported reports [13,14].”
BackgroundMethodChapman, W. W., Dowling, J., & Wagner, M. M. (2005) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2005.04.012 · Cited by 105
“The most frequent syndromic classification was respiratory syndrome ( 34 ,916), and the least frequent was botulinic (1,961).”
BackgroundSelf-citationHripcsak, G., & Wilcox, A. (2002) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2002.0090001 · Cited by 88
Natural Language Processing in Biomedicine: A Unified System Architecture Overview
Doan, S., Conway, M., Phuong, T. M., & Ohno-Machado, L. (2014) · Methods in Molecular Biology · doi:10.1007/978-1-4939-0847-9_16 · Cited by 65
“SymText has been used in several applications such as mapping chief complaints into ICD-9 codes [29] and extracting pneumonia-related findings from chest radiograph reports [30, 31].”
MethodInfluentialCoreference resolution: A review of general methodologies and applications in the clinical domain
Zheng, J., Chapman, W. W., Crowley, R. S., & Savova, G. (2011) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.08.006 · Cited by 107
“During the past two decades, several systems have been developed to extract named entities (NEs) from clinical narrative, first specialized in certain report types [16] [17] [18] [19], and later more general purpose [20] [3] [21] [22].”
Self-citationMining complex clinical data for patient safety research: a framework for event discovery
Hripcsak, G. M., Henry, S. B., Stetson, P. D., & Patel, V. L. (2003) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2003.08.001 · Cited by 97
“At least two independent groups have demonstrated that natural language processing can be as accurate as expert human coders for coding radiographic reports, as well as more accurate than simple text-based methods, such as searching for relevant phrases in the reports [45,51,52].”
BackgroundFacilitating Cancer Research using Natural Language Processing of Pathology Reports
Hua, X., Kristin, A., Grann Victor, R., & Carol, F. (2004) · Studies in Health Technology and Informatics · doi:10.3233/978-1-60750-949-3-565 · Cited by 88
Automated quantification of COVID-19 severity and progression using chest CT images
Pu, J., Leader, J. K., Bandos, A. I., Shi, K., Wang, J., Shi, J., Du, P., Guo, Y., Wenzel, S. E., Fuhrman, C. R., et al. (2020) · European Radiology · doi:10.1007/s00330-020-07156-2 · Cited by 91
FitzHenry, F., Murff, H. J., Matheny, M. E., Gentry, N., Fielstein, E. M., Brown, S. H., Reeves, R. M., Aronsky, D., Elkin, P. L., Messina, V. P., et al. (2013) · Medical Care · doi:10.1097/mlr.0b013e31828d1210 · Cited by 85
Self-citationLuo, Y., Uzuner, Ö., & Szolovits, P. (2017) · Briefings in Bioinformatics · doi:10.1093/bib/bbw001 · Cited by 78
A temporal constraint structure for extracting temporal information from clinical narrative
Zhou, L., Melton, G. B., Parsons, S., & Hripcsak, G. M. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.07.002 · Cited by 83
Denny, J. C., Choma, N. N., Peterson, J. F., Miller, R. A., Bastarache, L. A., Li, M., & Peterson, N. B. (2011) · Medical Decision Making · doi:10.1177/0272989x11400418 · Cited by 85
Lee, R. Y., Brumback, L. C., Lober, W. B., Sibley, J., Nielsen, E. L., Treece, P. D., Kross, E. K., Loggers, E. T., Fausto, J. A., Lindvall, C., et al. (2021) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2020.08.024 · Cited by 80
“While NLP techniques have been efficacious in identifying a variety of distinct clinical entities (23–30), identifying goals-of-care discussions—a broad construct most often represented by descriptive free-text rather than precise signifiers—poses unique challenges to machine approaches (45).”
BackgroundNegation recognition in medical narrative reports
Rokach, L., Romano, R., & Maimon, O. (2008) · Information Retrieval · doi:10.1007/s10791-008-9061-0 · Cited by 90
“Researchers in medical informatics have suggested methods for automatically extracting information contained in narrative reports for decision support ( Fiszman et al. 2000 ), guideline implementation (Fiszman and Haug 2000), and detection and management of epidemics (Hripcsak et al. 1999).”
MethodLi, L., Chase, H. S., Patel, C. O., Friedman, C., & Weng, C. (2008) · PubMed · Cited by 81
Recent Trends in AI-Based Intelligent Sensing
Sharma, A., Sharma, V., Jaiswal, M., Wang, H.-C., Jayakody, D. N. K., Basnayaka, C. M. W., & Muthanna, A. (2022) · Electronics · doi:10.3390/electronics11101661 · Cited by 96
Opportunities and challenges in application of artificial intelligence in pharmacology
Kumar, M., Nguyen, T. P. N., Kaur, J., Singh, T. G., Soni, D., Singh, R., & Kumar, P. (2023) · Pharmacological Reports · doi:10.1007/s43440-022-00445-1 · Cited by 92
“Natural language processing (NLP), the first reading device to provides excellent flexibility for doctors to study the descriptors of X-rays on the chest are also used to treat infectious diseases [29] and least square support vector machine (LSSVM) used in cancer diagnosis were introduced based on AI [30].”
MethodSavova, G., Fan, J., Ye, Z., Murphy, S. P., Zheng, J., Chute, C. G., & Kullo, I. J. (2010) · PubMed · Cited by 95
Extracting timing and status descriptors for colonoscopy testing from electronic medical records
Denny, J. C., Peterson, J. F., Choma, N. N., Xu, H., Miller, R. A., Bastarache, L., & Peterson, N. B. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.004804 · Cited by 76
“…on tasks such as the identi fi cation of adverse events from discharge summaries, 12 the recognition of important fi ndings from radiology reports, 31 or recent Informatics for Integrating Biology and the Bedside (i2b2) ‘ NLP challenges ’ regarding the detection of smoking status, 32 obesity and…”
BackgroundArtificial Intelligence in Global Health —A Framework and Strategy for Adoption and Sustainability
Hadley, T. D., Pettit, R. W., Malik, T., Khoei, A. A., & Salihu, H. M. (2020) · International Journal of Maternal and Child Health and AIDS (IJMA) · doi:10.21106/ijma.296 · Cited by 66
Zhou, L., Plasek, J. M., Mahoney, L. M., Karipineni, N., CHANG, F. Y., Xuemin, Y., Chang, F., Dimaggio, D., Goldman, D. S., & Rocha, R. A. (2011) · PubMed · Cited by 85
Fever detection from free-text clinical records for biosurveillance
Chapman, W. W., Dowling, J., & D, M. M. W. M. (2004) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2004.03.002 · Cited by 75
Self-citationA Natural Language Processing Tool for Large-Scale Data Extraction from Echocardiography Reports
Nath, C., Albaghdadi, M. S., & Jonnalagadda, S. R. (2016) · PLOS ONE · doi:10.1371/journal.pone.0153749 · Cited by 69
Pneumonia identification using statistical feature selection
Bejan, C. A., Xia, F., Vanderwende, L., Wurfel, M. M., & Yetisgen-Yildiz, M. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000752 · Cited by 67
Cui, L., Bozorgi, A., Lhatoo, S. D., Zhang, G., & Sahoo, S. S. (2012) · Europe PMC (PubMed Central) · Cited by 78
Natural Language Processing to identify pneumonia from radiology reports
Dublin, S., Baldwin, E., Walker, R. L., Christensen, L. M., Haug, P. J., Jackson, M. L., Nelson, J. C., Ferraro, J., Carrell, D., & Chapman, W. W. (2013) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.3418 · Cited by 65
“17 ONYX is based on a family of NLP systems 12,18 developed largely for chest radiograph interpretation.”
Self-citationUsing automatically extracted information from mammography reports for decision-support
Bozkurt, S., Gimenez, F., Burnside, E. S., Gülkesen, K. H., & Rubin, D. L. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.07.001 · Cited by 68
“This was the case in the Antibiotic Assistant [49], and the authors needed to implement a simpler keyword-based approach to NLP.”
Identifying QT prolongation from ECG impressions using a general-purpose Natural Language Processor
Denny, J. C., Miller, R. A., Waitman, L. R., Arrieta, M. A., & Peterson, J. F. (2008) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2008.09.001 · Cited by 63
“[12] Researchers have used NLP systems to identify clinical syndromes and common biomedical concepts from radiology reports[13–15], clinical notes[16–18], problem lists[19], nursing documentation[20], and medical education documents[21].”
MethodDeléger, L., Grouin, C., & Zweigenbaum, P. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.003962 · Cited by 63
McMaster, C., Chan, J., Liew, D. F. L., Su, E., Frauman, A. G., Chapman, W. W., & Pires, D. E. V. (2022) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104265 · Cited by 71
Self-citationAnaphoric relations in the clinical narrative: corpus creation
Savova, G. K., Chapman, W. W., Zheng, J., & Crowley, R. S. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000108 · Cited by 57
Self-citationComparison of computerized surveillance and manual chart review for adverse events
Tinoco, A., Evans, R. S., Staes, C. J., Lloyd, J. F., Rothschild, J. M., & Haug, P. J. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000187 · Cited by 53
“Prior studies on CSS that use information in physician narratives have focused on specific types of documents, such as discharge summaries.”
BackgroundSelf-citationApplication of Artificial Intelligence in Modern Healthcare System
Datta, S., Barua, R., & Das, J. (2020) · IntechOpen eBooks · doi:10.5772/intechopen.90454 · Cited by 63
Haas, J. P., Mendonça, E. A., Ross, B. G., Friedman, C., & Larson, E. (2005) · American Journal of Infection Control · doi:10.1016/j.ajic.2005.06.008 · Cited by 55
Selective Automated Indexing of Findings and Diagnoses in Radiology Reports
Hersh, W., Mailhot, M., Arnott-Smith, C., & Lowe, H. (2001) · Journal of Biomedical Informatics · doi:10.1006/jbin.2001.1025 · Cited by 54
Liu, B., Zhou, X., Wang, Y., Hu, J., He, L., Zhang, R., Chen, S., & Guo, Y. (2011) · Statistics in Medicine · doi:10.1002/sim.4417 · Cited by 59
“Although dissecting sentences and extracting the meaning completely from free-text clinical documents is still difficult [43], text mining tools are becoming increasingly used and valued for simplified tasks [44–46].”
BackgroundUnder-documentation of chronic kidney disease in the electronic health record in outpatients
Chase, H. S., Radhakrishnan, J., Shirazian, S., Rao, M. K., & Vawdrey, D. K. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001396 · Cited by 53
“Agreement in the experts’ classifications was analyzed using the k statistic.(28) In the initial rounds of pair-wise validation there was moderate agreement (k1⁄40.”
MethodFriedlin, J., & McDonald, C. J. (2006) · Europe PMC (PubMed Central) · Cited by 53
Lederman, A., Lederman, R., & Verspoor, K. M. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac121 · Cited by 60
Annotation of a Large Clinical Entity Corpus
Patel, P., Davey, D., Panchal, V., & Pathak, P. S. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1228 · Cited by 58
“For example, (Fiszman et al., 2000) annotated chest x-ray reports for automatic identification of acute bacterial pneumonia; (South et al., 2009) manually annotated clinical records to identify phenotypic information for inflammatory bowel disease; (Koeling et al., 2011) have annotated oncology…”
BackgroundConvolutional Neural Network for Automatic Pneumonia Detection in Chest Radiography
Khoiriyah, S. A., Basofi, A., & Fariza, A. (2020) · International Electronics Symposium (IES) · doi:10.1109/ies50839.2020.9231540 · Cited by 50
“The automated method was developed as a decision support tool to help doctors diagnose pneumonia [3].”
MethodValidation Study of an Automated Electronic Acute Lung Injury Screening Tool
Azzam, H. C., Khalsa, S. S., Urbani, R., Shah, C. V., Christie, J. D., Lanken, P. N., & Fuchs, B. D. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3120 · Cited by 49
“Natural language processing has been shown to be superior to text-based methods that rely on detection of relevant phrases in the diagnosis of pneumonia from radiology reports,”
Kukafka, R., Bales, M. E., Burkhardt, A., & Friedman, C. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2107 · Cited by 45
“detecting acute bacterial pneumonia from radiology reports,”
Detecting possible vaccine adverse events in clinical notes of the electronic medical record
Hazlehurst, B. L., Naleway, A. L., & Mullooly, J. P. (2009) · Vaccine · doi:10.1016/j.vaccine.2009.01.105 · Cited by 45
Chapman, W. W., Cooper, G. F., Hanbury, P., Chapman, B. E., Harrison, L. H., & D, M. M. W. M. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1330 · Cited by 45
“and have been used to help detect patients with findings consistent with pneumonia,”
MethodSelf-citationWang, Y., Sun, L., & Jin, Q. (2019) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2019.2911947 · Cited by 45
“…the traditional computer-aided diagnostic methods of medical images, medical image processing based on in-depth learning has currently achieved optimal performance [15], [16] and has reached the level of professional doctors for some situations such as the detection of malignant melanomas [17].”
BackgroundHassanpour, S., Bay, G., & Langlotz, C. (2017) · Journal of digital imaging · doi:10.1007/s10278-016-9931-8 · Cited by 45
Escudié, J.-B., Rance, B., Malamut, G., Khater, S., Burgun, A., Cellier, C., & Jannot, A. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0537-y · Cited by 53
“Disorder terms are indeed present in various types of clinical narratives, such as radiology reports [5, 6], medical observations [7, 8], nurse narratives [9]– and generally every document produced for healthcare activity.”
BackgroundKarystianis, G., Sheppard, T., Dixon, W. G., & Nenadić, G. (2015) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-016-0255-x · Cited by 43
“It relies on various lexical, syntactic and semantic techniques in order to recognise and link various concepts [12 – 14].”
BackgroundJones, B. E., South, B. C., Shao, Y., Lu, C. C., Leng, J., Sauer, B. C., Gundlapalli, A., Samore, M., & Zeng, Q. (2018) · Applied Clinical Informatics · doi:10.1055/s-0038-1626725 · Cited by 44
“6 It has previously been applied to radiology reports for radiographic concepts of pneumonia, 7,8 and resulting tools have been used for both surveillance 9,10 andprovidercognitivesupport.”
Complex epilepsy phenotype extraction from narrative clinical discharge summaries
Cui, L., Sahoo, S. S., Lhatoo, S. D., Garg, G., Rai, P., Bozorgi, A., & Zhang, G. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.06.006 · Cited by 46
“Two salient examples are the Cancer Text Information System (caTIES) [1] and SymText [2].”
BackgroundAutomated Computer-Assisted Categorization of Radiology Reports
Thomas, B. J., Ouellette, H. A., Halpern, E. F., & Rosenthal, D. I. (2005) · American Journal of Roentgenology · doi:10.2214/ajr.184.2.01840687 · Cited by 41
“Manual coding (“results coding”) by trained personnel is rarely performed because it is time-consuming and costly [ 9 ], and coding by radiologists has been generally resisted by the radiology community [9].”
BackgroundUsing an ensemble system to improve concept extraction from clinical records
Kang, N., Afzal, Z., Singh, B., van Mulligen, E. M., & Kors, J. A. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2011.12.009 · Cited by 46
“, detection of adverse events in medical records of hospitalized patients [10], extraction of family history from discharge summaries [11], and detection of signs of pneumonia in radiology reports [12], to name a few.”
BackgroundClinical Decision Support Systems: A Survey of NLP-Based Approaches from Unstructured Data
Reyes‐Ortiz, J. A., González-Beltrán, B. A., & Gallardo-López, L. (2015) · International Workshop on Database and Expert Systems Applications (DEXA) · doi:10.1109/dexa.2015.47 · Cited by 40
DeLisle, S., Kim, B., Deepak, J., Siddiqui, T., Gundlapalli, A. V., Samore, M. H., & DʼAvolio, L. (2013) · PLoS ONE · doi:10.1371/journal.pone.0070944 · Cited by 40
“Over the last 20 years, various combinations of approaches, including natural language processing [28] [29] [30] [31] , expert rules [32, 33] , Bayesian [32, 34] or neural networks [35] and machine-learning [33] , have held their own compared to physicians for their ability to find pneumonia-related concepts in report…”
BackgroundIdentifying adverse drug reactions from free‐text electronic hospital health record notes
Wasylewicz, A. T. M., van de Burgt, B. W. M., Weterings, A., Jessurun, N. T., Korsten, E. H. H. M., Egberts, T. C. G., Bouwman, R. A., Kerskes, M., Grouls, R. J. E., & van der Linden, C. M. J. (2021) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15068 · Cited by 39
Automated Outcome Classification of Emergency Department Computed Tomography Imaging Reports
Yadav, K., Sarioglu, E., Smith, M., & Choi, H.-A. (2013) · Academic Emergency Medicine · doi:10.1111/acem.12174 · Cited by 38
Natural language processing in radiology: Clinical applications and future directions
Bobba, P. S., Sailer, A., Pruneski, J. A., Beck, S., Mozayan, A., Mozayan, S., Arango, J., Cohan, A., & Chheang, S. (2023) · Clinical Imaging · doi:10.1016/j.clinimag.2023.02.014 · Cited by 37
eQuality: Electronic Quality Assessment From Narrative Clinical Reports
Brown, S. H., Speroff, T., Fielstein, E. M., Bauer, B. A., Wahner‐Roedler, D. L., Greevy, R. A., & Elkin, P. L. (2006) · Mayo Clinic Proceedings · doi:10.4065/81.11.1472 · Cited by 34
Buchlak, Q. D., Esmaili, N., Bennett, C. C., & Farrokhi, F. R. (2021) · Acta neurochirurgica. Supplementum · doi:10.1007/978-3-030-85292-4_32 · Cited by 39
Automatic Classification of Foot Examination Findings Using Clinical Notes and Machine Learning
Pakhomov, S., Hanson, P., Bjornsen, S. S., & Smith, S. A. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2585 · Cited by 34
Metadata
- Authors
- Fiszman, M., Chapman, W. W., Aronsky, D., Evans, R. S., & Haug, P. J.
- Affiliations
- University of Utah (US)
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2000-11-01
- Volume
- 7
- Issue
- 6
- Pages
- 593-604
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC129668
- OpenAlex
- W2125726375
- Cited by
- 250
- References
- 34
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 10 · 2023: 15 · 2022: 12 · 2021: 13 · 2020: 17 · 2019: 12 · 2018: 4 · 2017: 13 · 2016: 9 · 2015: 8 · 2014: 6 · 2013: 10 · 2012: 12
- FWCI
- 2.41
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (5r01hl053427-03, 5r01lm006539-03)
- National Heart, Lung, and Blood Institute (HL53427)
- U.S. National Library of Medicine (LM06539)
- Topics
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Epidemiology › Pneumonia and Respiratory Infections
- Keywords
- acute bacterial pneumonia, chest X-ray reports, natural language processing, radiology report interpretation, infiltrates, keyword search, physician agreement, quality assurance in radiology
- MeSH terms
- Acute Disease, Algorithms, Diagnosis, Computer-Assisted, Humans, Lung, Natural Language Processing, Radiography, Thoracic, Reproducibility of Results, Pneumonia, Bacterial
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.