Natural Language Processing for the Identification of Silent Brain Infarcts From Neuroimaging Reports
Open accessThe DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Silent brain infarction (SBI) is defined as the presence of 1 or more brain lesions, presumed to be because of vascular occlusion, found by neuroimaging (magnetic resonance imaging or computed tomography) in patients without clinical manifestations of stroke. It is more common than stroke and can be detected in 20% of healthy elderly people. Early detection of SBI may mitigate the risk of stroke by offering preventative treatment plans. Natural language processing (NLP) techniques offer an opportunity to systematically identify SBI cases from electronic health records (EHRs) by extracting, normalizing, and classifying SBI-related incidental findings interpreted by radiologists from neuroimaging reports. OBJECTIVE: This study aimed to develop NLP systems to determine individuals with incidentally discovered SBIs from neuroimaging reports at 2 sites: Mayo Clinic and Tufts Medical Center. METHODS: Both rule-based and machine learning approaches were adopted in developing the NLP system. The rule-based system was implemented using the open source NLP pipeline MedTagger, developed by Mayo Clinic. Features for rule-based systems, including significant words and patterns related to SBI, were generated using pointwise mutual information. The machine learning models adopted convolutional neural network (CNN), random forest, support vector machine, and logistic regression. The performance of the NLP algorithm was compared with a manually created gold standard. The gold standard dataset includes 1000 radiology reports randomly retrieved from the 2 study sites (Mayo and Tufts) corresponding to patients with no prior or current diagnosis of stroke or dementia. 400 out of the 1000 reports were randomly sampled and double read to determine interannotator agreements. The gold standard dataset was equally split to 3 subsets for training, developing, and testing. RESULTS: Among the 400 reports selected to determine interannotator agreement, 5 reports were removed due to invalid scan types. The interannotator agreements across Mayo and Tufts neuroimaging reports were 0.87 and 0.91, respectively. The rule-based system yielded the best performance of predicting SBI with an accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 0.991, 0.925, 1.000, 1.000, and 0.990, respectively. The CNN achieved the best score on predicting white matter disease (WMD) with an accuracy, sensitivity, specificity, PPV, and NPV of 0.994, 0.994, 0.994, 0.994, and 0.994, respectively. CONCLUSIONS: We adopted a standardized data abstraction and modeling process to developed NLP techniques (rule-based and machine learning) to detect incidental SBIs and WMDs from annotated neuroimaging reports. Validation statistics suggested a high feasibility of detecting SBIs and WMDs from EHRs using NLP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Kernan, W. N., Ovbiagele, B., Black, H. R., Bravata, D. M., Chimowitz, M. I., Ezekowitz, M. D., Fang, M. C., Fisher, M., Furie, K. L., Heck, D. V., et al. (2014) · Stroke · doi:10.1161/str.0000000000000024 · Cited by 5,170
Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages
McNemar, Q. (1947) · Psychometrika · doi:10.1007/bf02295996 · Cited by 4,754
Silent brain infarcts: a systematic review
Vermeer, S. E., Longstreth, W. T., & Koudstaal, P. J. (2007) · The Lancet Neurology · doi:10.1016/s1474-4422(07)70170-9 · Cited by 1,221 · cited 1×
“SBIs are more common than stroke and can be detected on MRI in 20% of healthy elderly [1-3].”
PheWAS: demonstrating the feasibility of a phenome-wide scan to discover gene–disease associations
Denny, J. C., Ritchie, M. D., Basford, M. A., Pulley, J. M., Bastarache, L., Brown-Gentry, K., Wang, D., Masys, D. R., Roden, D. M., & Crawford, D. C. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq126 · Cited by 1,219
Chart reviews in emergency medicine research: Where are the methods?
Gilbert, E. H., Lowenstein, S. R., Koziol-McLain, J., Barta, D. C., & Steiner, J. (1996) · Annals of Emergency Medicine · doi:10.1016/s0196-0644(96)70264-0 · Cited by 1,165
Clinical information extraction applications: A literature review
Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., Liu, S., Zeng, Y., Mehrabi, S., Sohn, S., et al. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.11.011 · Cited by 910
Self-citationGouw, A. A., van der Flier, W. M., Fazekas, F., van Straaten, E. C. W., Pantoni, L., Poggesi, A., Inzitari, D., Erkinjuntti, T., Wahlund, L. O., Waldemar, G., et al. (2008) · Stroke · doi:10.1161/strokeaha.107.498535 · Cited by 384
A comparison of word embeddings for the biomedical natural language processing
Wang, Y., Liu, S., Afzal, N., Rastegar-Mojarad, M., Wang, L., Shen, F., Kingsbury, P., & Liu, H. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.09.008 · Cited by 370
Self-citation
Cited by
Wen, A., Fu, S., Moon, S., El Wazir, M., Rosenbaum, A., Kaggal, V. C., Liu, S., Sohn, S., Liu, H., & Fan, J. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0208-8 · Cited by 142
“Through iterative refinement conducted by a neurologist and neuroradiologist, 36 SBI-related terms were generated and grouped into three semantic concepts.(16) User feedback indicated that NLPaaS was easily adoptable and usable due to being distributed as a standalone executable with easy-to-follow instructions.”
Self-citationGupta, S., Belouali, A., Shah, N. J., Atkins, M. B., & Madhavan, S. (2021) · JCO Clinical Cancer Informatics · doi:10.1200/cci.20.00109 · Cited by 36
Fu, S., Vassilaki, M., Ibrahim, O. A., Petersen, R. C., Pagali, S., St Sauver, J., Moon, S., Wang, L., Fan, J. W., Liu, H., et al. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.958539 · Cited by 15
Self-citation
Metadata
- Authors
- Fu, S., Leung, L. Y., Wang, Y., Raulli, A.-O., Kallmes, D. F., Kinsman, K. A., Nelson, K. B., Clark, M. S., Luetmer, P. H., Kingsbury, P. R., Kent, D. M., & Liu, H.
- Venue
- JMIR Medical Informatics
- Publisher
- JMIR Publications Inc.
- Published
- 2019-05-03
- Volume
- 7
- Issue
- 2
- Pages
- e12109
- ISSN
- 2291-9694 (ISSN-L)
- OpenAlex
- W2927599034
- Cited by
- 62
- References
- 11
- FWCI
- 4.37
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.