The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review
Open accessAbstract
Objective: Natural language processing (NLP) of symptoms from electronic health records (EHRs) could contribute to the advancement of symptom science. We aim to synthesize the literature on the use of NLP to process or analyze symptom information documented in EHR free-text narratives. Materials and Methods: Our search of 1964 records from PubMed and EMBASE was narrowed to 27 eligible articles. Data related to the purpose, free-text corpus, patients, symptoms, NLP methodology, evaluation metrics, and quality indicators were extracted for each study. Results: Symptom-related information was presented as a primary outcome in 14 studies. EHR narratives represented various inpatient and outpatient clinical specialties, with general, cardiology, and mental health occurring most frequently. Studies encompassed a wide variety of symptoms, including shortness of breath, pain, nausea, dizziness, disturbed sleep, constipation, and depressed mood. NLP approaches included previously developed NLP tools, classification methods, and manually curated rule-based processing. Only one-third (n = 9) of studies reported patient demographic characteristics. Discussion: NLP is used to extract information from EHR free-text narratives written by a variety of healthcare providers on an expansive range of symptoms across diverse clinical specialties. The current focus of this field is on the development of methods to extract symptom information and the use of symptom information for disease classification tasks rather than the examination of symptoms themselves. Conclusion: Future NLP studies should concentrate on the investigation of symptoms and symptom documentation in EHR free-text narratives. Efforts should be undertaken to examine patient characteristics and make symptom-related NLP algorithms or pipelines and vocabularies openly available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Artificial intelligence in healthcare: past, present and future
Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H., & Wang, Y. (2017) · Stroke and Vascular Neurology · doi:10.1136/svn-2017-000101 · Cited by 4,837
circlize implements and enhances circular visualization in R
Gu, Z., Gu, L., Eils, R., Schlesner, M., & Brors, B. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu393 · Cited by 4,246
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967
2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text
Uzuner, Ö., South, B. R., Shen, S., & DuVall, S. L. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000203 · Cited by 1,252
Clinical information extraction applications: A literature review
Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., Liu, S., Zeng, Y., Mehrabi, S., Sohn, S., et al. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.11.011 · Cited by 910
How Open Science Helps Researchers Succeed
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., McDougall, D., Nosek, B. A., Ram, K., Soderberg, C. K., et al. (2016) · eLife · doi:10.7554/elife.16800 · Cited by 798
Self-citationKreimeyer, K., Foster, M., Pandey, A., Arya, N., Halford, G., Jones, S. F., Forshee, R., Walderhaug, M., & Botsis, T. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.07.012 · Cited by 616
Natural language processing in radiology: a systematic review
Pons, E., Braun, L. M. M., Hunink, M. G. M., & Kors, J. A. (2016) · Radiology · doi:10.1148/radiol.16142770 · Cited by 573
Miaskowski, C., Barsevick, A., Berger, A., Casagrande, R., Grady, P. A., Jacobsen, P., Kutner, J., Patrick, D., Zimmerman, L., Xiao, C., et al. (2017) · Journal of the National Cancer Institute · doi:10.1093/jnci/djw253 · Cited by 498
Concurrence of big data analytics and healthcare: a systematic review
Mehta, N., & Pandit, A. (2018) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2018.03.013 · Cited by 494
Evaluating temporal relations in clinical text: 2012 i2b2 Challenge
Sun, W., Rumshisky, A., & Uzuner, O. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001628 · Cited by 473
Ford, E., Carroll, J. A., Smith, H. E., Scott, D., & Cassell, J. A. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv180 · Cited by 408
Natural Language Processing in Oncology: A Review
Yim, W.-W., Yetisgen, M., Harris, W. P., & Kwan, S. W. (2016) · JAMA Oncology · doi:10.1001/jamaoncol.2016.0213 · Cited by 298
Wang, X., Hripcsak, G., Markatou, M., & Friedman, C. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3028 · Cited by 288
Agreement between patient-reported symptoms and their documentation in the medical record.
Pakhomov, S., Jacobsen, S. J., Chute, C. G., & Roger, V. L. (2008) · PubMed · Cited by 281
Recognizing Obesity and Comorbidities in Sparse Data
Uzuner, O. (2009) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m3115 · Cited by 231
Jackson, R. G., Patel, R., Jayatilleke, N., Kolliakou, A., Ball, M., Gorrell, G., Roberts, A., Dobson, R. J., & Stewart, R. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-012012 · Cited by 221
Text summarization in the biomedical domain: A systematic review of recent research
Mishra, R., Bian, J., Fiszman, M., Weir, C. R., Jonnalagadda, S., Mostafa, J., & Del Fiol, G. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.06.009 · Cited by 200
Application of text mining in the biomedical domain
Fleuren, W. W. M., & Alkema, W. (2015) · Methods · doi:10.1016/j.ymeth.2015.01.015 · Cited by 193
“Big Data” and the Electronic Health Record
Ross, M. K., Wei, W., & Ohno-Machado, L. (2014) · Yearbook of Medical Informatics · doi:10.15265/iy-2014-0003 · Cited by 190
Cancer Symptom Cluster Management
Kwekkeboom, K. L. (2016) · Seminars in Oncology Nursing · doi:10.1016/j.soncn.2016.08.004 · Cited by 169
Byrd, R. J., Steinhubl, S. R., Sun, J., Ebadollahi, S., & Stewart, W. F. (2013) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2012.12.005 · Cited by 148
Patel, R., Lloyd, T., Jackson, R., Ball, M., Shetty, H., Broadbent, M. T. M., Geddes, J. R., Stewart, R. J., McGuire, P. K., & Taylor, M. J. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-007504 · Cited by 140
Stubbs, A., Kotfila, C., Xu, H., & Uzuner, Ö. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.07.001 · Cited by 130
Vijayakrishnan, R., Steinhubl, S. R., Ng, K., Sun, J., Byrd, R. J., Daar, Z. S., Williams, B. A., deFilippi, C. R., Ebadollahi, S., & Stewart, W. F. (2014) · Journal of Cardiac Failure · doi:10.1016/j.cardfail.2014.03.008 · Cited by 103
When will ‘open science’ become simply ‘science’?
Watson, M. (2015) · Genome biology · doi:10.1186/s13059-015-0669-2 · Cited by 90
Chase, H. S., Mitrani, L. R., Lu, G. G., & Fulgieri, D. J. (2017) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-017-0418-4 · Cited by 83
Heintzelman, N., Taylor, R. J., Simonsen, L., Lustig, R. L., Anderko, D., Haythornthwaite, J. A., Childs, L. C., & Bova, G. S. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001076 · Cited by 76
Friedman, C., Knirsch, C. A., Shagina, L., & Hripcsak, G. M. (1999) · PubMed Central · Cited by 74
Exploring the Ability of Natural Language Processing to Extract Data From Nursing Narratives
Hyun, S., Johnson, S. B., & Bakken, S. R. (2009) · CIN Computers Informatics Nursing · doi:10.1097/ncn.0b013e3181a91b58 · Cited by 70
Self-citationAutomated knowledge acquisition from clinical narrative reports.
Wang, X., Chused, A. E., Elhadad, N., Friedman, C., & Markatou, M. (2008) · PubMed Central · Cited by 68
Epidemiology of angina pectoris: Role of natural language processing of the medical record
Pakhomov, S. S. V., Hemingway, H., Weston, S. A., Jacobsen, S. J., Rodeheffer, R. J., & Roger, V. L. (2007) · American Heart Journal · doi:10.1016/j.ahj.2006.12.022 · Cited by 68
Elkin, P. L., Froehling, D. A., Wahner‐Roedler, D. L., Brown, S. H., & Bailey, K. R. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-156-1-201201030-00003 · Cited by 67
Identifying Patients with Depression Using Free-text Clinical Documents
Zhou, L., Baughman, A. W., Lei, V. J., Lai, K. H., Navathe, A. S., CHANG, F. Y., Margarita, S., Topaz, M., Zhong, F., Murrali, M., et al. (2015) · Studies in health technology and informatics · doi:10.3233/978-1-61499-564-7-629 · Cited by 55
Iqbal, E., Mallah, R., Rhodes, D. R., Wu, H., Romero, A., Chang, N., Dzahini, O., Pandey, C., Broadbent, M. T. M., Stewart, R. J., et al. (2017) · PLoS ONE · doi:10.1371/journal.pone.0187121 · Cited by 54
Greenwald, J. L., Cronin, P. R., Carballo, V., Danaei, G., & Choy, G. (2017) · Medical Care · doi:10.1097/mlr.0000000000000651 · Cited by 47
Detecting possible vaccine adverse events in clinical notes of the electronic medical record
Hazlehurst, B. L., Naleway, A. L., & Mullooly, J. P. (2009) · Vaccine · doi:10.1016/j.vaccine.2009.01.105 · Cited by 45
Automatable algorithms to identify nonmedical opioid use using electronic data: a systematic review
Canan, C. E., Polinski, J. M., Alexander, G. C., Kowal, M. K., Brennan, T. A., & Shrank, W. H. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx066 · Cited by 44
Matheny, M. E., FitzHenry, F., Speroff, T., Green, J. K., Griffith, M. L., Vasilevskis, E. E., Fielstein, E. M., Elkin, P. L., & Brown, S. H. (2012) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2011.11.005 · Cited by 43
Mining the Electronic Health Record for Disease Knowledge
Chen, E. S., & Sarkar, I. N. (2014) · Methods in molecular biology · doi:10.1007/978-1-4939-0709-0_15 · Cited by 40
Evaluation of preprocessing techniques for chief complaint classification
Dara, J., Dowling, J., Travers, D. A., Cooper, G. F., & Chapman, W. W. (2007) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2007.11.004 · Cited by 36
Weissman, G. E., Harhay, M. O., Lugo, R. M., Fuchs, B. D., Halpern, S. D., & Mikkelsen, M. E. (2016) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201602-131oc · Cited by 35
Envisioning the future in symptom science
Corwin, E. J., Berg, J. A., Armstrong, T. S., Dabbs, A. D., Lee, K. A., Meek, P., & Redeker, N. S. (2014) · Nursing Outlook · doi:10.1016/j.outlook.2014.06.006 · Cited by 35
Detecting Unplanned Care From Clinician Notes in Electronic Health Records
Tamang, S. R., Patel, M. I., Blayney, D. W., Kuznetsov, J. L., Finlayson, S. G., Vetteth, Y., & Shah, N. H. (2015) · Journal of Oncology Practice · doi:10.1200/jop.2014.002741 · Cited by 34
Gundlapalli, A. V., Divita, G., Redd, A. M., Carter, M. E., Ko, D., Rubin, M. A., Samore, M. H., Strymish, J. M., Krein, S. L., Gupta, K., et al. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.07.012 · Cited by 31
Yuan, L., Pan, X., Li, G., & Hu, X. T. (2015) · IEEE Transactions on NanoBioscience · doi:10.1109/tnb.2015.2422612 · Cited by 26
Gundlapalli, A. V., South, B. R., Phansalkar, S., Kinney, A. Y., Shen, S., DeLisle, S., Perl, T. M., & Samore, M. H. (2008) · PubMed Central · Cited by 22
Tang, H., Solti, I., Kirkendall, E. S., Zhai, H., Lingren, T., Meller, J., & Ni, Y. (2017) · Biomedical Informatics Insights · doi:10.1177/1178222617713018 · Cited by 19
General Symptom Extraction from VA Electronic Medical Notes
Divita, G., Luo, G., Tran, L., Workman, T. E., Gundlapalli, A. V., & Samore, M. H. (2017) · Studies in health technology and informatics · doi:10.3233/978-1-61499-830-3-356 · Cited by 18
Nunes, A. P., Loughlin, A. M., Qiao, Q., Ezzy, S. M., Yochum, L., Clifford, C. R., Gately, R. V., Dore, D. D., & Seeger, J. D. (2017) · Diabetes Therapy · doi:10.1007/s13300-017-0314-z · Cited by 15
Cited by
Lee, E. E., Torous, J., De Choudhury, M., Depp, C. A., Graham, S. A., Kim, H.-C., Paulus, M. P., Krystal, J. H., & Jeste, D. V. (2021) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2021.02.001 · Cited by 429
Hossain, E., Rana, R., Higgins, N., Soar, J., Barua, P. D., Pisani, A. R., & Turner, K. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106649 · Cited by 407
Artificial intelligence surgery: how do we get to autonomous actions in surgery?
Gumbs, A. A., Frigerio, I., Spolverato, G., Croner, R., Illanes, A., Chouillard, E., & Elyan, E. (2021) · Sensors · doi:10.3390/s21165526 · Cited by 161
“analyzed the available methods, employing NLP to interpret symptoms from EHRs of inpatients and outpatients, finding possible future applications for NLP in the normalization of symptoms to controlled vocabularies, in order to avoid overlapping of different words for the same concept [21].”
MethodYan, M. Y., Gustad, L. T., & Nytrø, Ø. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab236 · Cited by 119
A survey on clinical natural language processing in the United Kingdom from 2007 to 2022
Wu, H., Wang, M., Wu, J., Francis, F., Chang, Y.-H., Shavick, A., Dong, H., Poon, M. T. C., Fitzpatrick, N., Levine, A. P., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00730-6 · Cited by 101
Metadata
- Authors
- Koleck, T. A., Dreisbach, C., Bourne, P. E., & Bakken, S.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-02-06
- Volume
- 26
- Issue
- 4
- Pages
- 364-379
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- OpenAlex
- W2912971066
- Cited by
- 408
- References
- 50
- FWCI
- 2.23
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.