Application of a Natural Language Processing Algorithm to Asthma Ascertainment. An Automated Chart Review
Open accessAbstract
RATIONALE: Difficulty of asthma ascertainment and its associated methodologic heterogeneity have created significant barriers to asthma care and research. OBJECTIVES: We evaluated the validity of an existing natural language processing (NLP) algorithm for asthma criteria to enable an automated chart review using electronic medical records (EMRs). METHODS: The study was designed as a retrospective birth cohort study using a random sample of 500 subjects from the 1997-2007 Mayo Birth Cohort who were born at Mayo Clinic and enrolled in primary pediatric care at Mayo Clinic Rochester. Performance of NLP-based asthma ascertainment using predetermined asthma criteria was assessed by determining both criterion validity (chart review of EMRs by abstractor as a gold standard) and construct validity (association with known risk factors for asthma, such as allergic rhinitis). MEASUREMENTS AND MAIN RESULTS: After excluding three subjects whose respiratory symptoms could be attributed to other conditions (e.g., tracheomalacia), among the remaining eligible 497 subjects, 51% were male, 77% white persons, and the median age at last follow-up date was 11.5 years. The asthma prevalence was 31% in the study cohort. Sensitivity, specificity, positive predictive value, and negative predictive value for NLP algorithm in predicting asthma status were 97%, 95%, 90%, and 98%, respectively. The risk factors for asthma (e.g., allergic rhinitis) that were identified either by NLP or the abstractor were the same. CONCLUSIONS: Asthma ascertainment through NLP should be considered in the era of EMRs because it can enable large-scale clinical studies in a more time-efficient manner and improve the recognition and care of childhood asthma in practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Computational phenotyping methods that leverage complex electronic health record data have been validated in different practice settings.
“To assist in the early identification and diagnosis of asthma, there are currently available innovative computational phenotyping methods that leverage complex electronic health record data that have been validated in different practice settings (32, 33).”
From Epidemiology of Asthma in Children and Adults (2019) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Aletaha, D., Neogi, T., Silman, A. J., Funovits, J., Felson, D. T., Bingham, C. O., Birnbaum, N. S., Burmester, G. R., Bykerk, V. P., Cohen, M. D., et al. (2010) · Arthritis & Rheumatism · doi:10.1002/art.27584 · Cited by 9,631 · see citing works
Asher, M. I., Montefort, S., Björkstén, B., Lai, C. K., Strachan, D. P., Weiland, S. K., & Williams, H. (2006) · The Lancet · doi:10.1016/s0140-6736(06)69283-0 · Cited by 4,223 · see citing works
The State of US Health, 1990-2010: Burden of Diseases, Injuries, and Risk Factors
Murray, C. J. L. (2013) · JAMA · doi:10.1001/jama.2013.13805 · Cited by 2,438 · see citing works
Identification of Asthma Phenotypes Using Cluster Analysis in the Severe Asthma Research Program
Moore, W. C., Meyers, D. A., Wenzel, S. E., Teague, W. G., Li, H., Li, X., D'Agostino, R., Castro, M., Curran‐Everett, D., Fitzpatrick, A. M., et al. (2009) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200906-0896oc · Cited by 2,131 · see citing works
Cluster analysis and clinical asthma phenotypes
Haldar, P., Pavord, I. D., Shaw, D. E., Berry, M. A., Thomas, M., Brightling, C. E., Wardlaw, A. J., & Green, R. H. (2008) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200711-1754oc · Cited by 1,987 · see citing works
Summary health statistics for U.S. adults: national health interview survey, 2012.
Blackwell, D. L., Lucas, J. W., & Clarke, T. C. (2014) · PubMed · Cited by 1,889 · see citing works
Treatment of Childhood Atopic Dermatitis and Economic Burden of Illness in Asia Pacific Countries
Lee, B. W., & Detzel, P. R. (2015) · Annals of Nutrition and Metabolism · doi:10.1159/000370221 · Cited by 1,671 · see citing works
A Community-based Study of the Epidemiology of Asthma: Incidence Rates, 1964–1983
Yunginger, J. W., Reed, C. E., O’Connell, E. J., Melton, L. J., O’Fallon, W. M., & Silverstein, M. D. (1992) · American Review of Respiratory Disease · doi:10.1164/ajrccm/146.4.888 · Cited by 724 · see citing works
Caesarean delivery and risk of atopy and allergic disesase: meta‐analyses
Bager, P., Wohlfahrt, J., & Westergaard, T. (2008) · Clinical & Experimental Allergy · doi:10.1111/j.1365-2222.2008.02939.x · Cited by 503 · see citing works
Fitzpatrick, A. M., Teague, W. G., Meyers, D. A., Peters, S. P., Li, X., Li, H., Wenzel, S. E., Aujla, S. J., Castro, M., Bacharier, L. B., et al. (2011) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2010.11.015 · Cited by 443 · see citing works
Identification of IL6R and chromosome 11q13.5 as risk loci for asthma
Ferreira, M. A., Matheson, M. C., Duffy, D. L., Marks, G. B., Hui, J., Le Souëf, P., Danoy, P., Baltic, S., Nyholt, D. R., Jenkins, M., et al. (2011) · The Lancet · doi:10.1016/s0140-6736(11)60874-x · Cited by 381 · see citing works
Zeng, Q. T., Goryachev, S., Weiss, S., Sordo, M., Murphy, S. N., & Lazarus, R. (2006) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-6-30 · Cited by 360 · see citing works
The Childhood Asthma Management Program (CAMP)
Childhood Asthma Management Program Research Group (1999) · Controlled Clinical Trials · doi:10.1016/s0197-2456(98)00044-0 · Cited by 352 · see citing works
(2011) · PubMed · Cited by 338 · see citing works
Oral Prednisolone for Preschool Children with Acute Virus-Induced Wheezing
Panickar, J. R., Lakhanpaul, M., Lambert, P. C., et al. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0804897 · Cited by 329 · see citing works
Electronic medical records for discovery research in rheumatoid arthritis
Liao, K. P., Cai, T., Gainer, V. S., Goryachev, S. D., Zeng‐Treitler, Q., Raychaudhuri, S., Peter Szolovits, B., Churchill, S. E., Murphy, S. N., Kohane, I. S., et al. (2010) · Arthritis Care & Research · doi:10.1002/acr.20184 · Cited by 319 · see citing works
Genome-wide association study of asthma identifies RAD50-IL13 and HLA-DR/DQ regions
Li, X., Howard, T. D., Zheng, S. L., Haselkorn, T., Peters, S. P., Meyers, D. A., & Bleecker, E. R. (2010) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2009.11.018 · Cited by 318 · see citing works
Preemptive Use of High-Dose Fluticasone for Virus-Induced Wheezing in Young Children
Ducharme, F. M., Lemire, C., Noya, F., Davis, G. M., Alos, N., Leblond, H., Savdie, C., Collet, J., Khomenko, L., Rivard, G. E., et al. (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0808907 · Cited by 305 · see citing works
Natural Language Processing and the Representation of Clinical Data
Sager, N., Lyman, M. S., Bucknall, C. E., Nhàn, N. T., & Tick, L. J. (1994) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1994.95236145 · Cited by 217 · see citing works
Azad, M. B., Coneys, J. G., Kozyrskyj, A. L., Field, C. J., Ramsey, C. D., Becker, A. B., Friesen, C., Abou-Setta, A., & Zarychanski, R. (2013) · BMJ · doi:10.1136/bmj.f6471 · Cited by 206 · see citing works
Incidence and Outcomes of Asthma in the Elderly
Bauer, B. A., Reed, C. E., Yunginger, J. W., Wollan, P. C., & Silverstein, M. D. (1997) · CHEST Journal · doi:10.1378/chest.111.2.303 · Cited by 175 · see citing works
Lazic, N., Roberts, G., Custovic, A., Belgrave, D., Bishop, C. M., Winn, J., Curtin, J. A., Hasan Arshad, S., & Simpson, A. (2013) · Allergy · doi:10.1111/all.12134 · Cited by 168 · see citing works
An information extraction framework for cohort identification using electronic health records.
Liu, H., Bielinski, S. J., Sohn, S., Murphy, S. P., Wagholikar, K. B., Jonnalagadda, S., Ravikumar, K. E., Wu, S., Kullo, I. J., & Chute, C. G. (2013) · PubMed Central · Cited by 164 · see citing works
Self-citationYawn, B. P., Yunginger, J. W., Wollan, P. C., Reed, C. E., Silverstein, M. D., & Harris, A. G. (1999) · Journal of Allergy and Clinical Immunology · doi:10.1016/s0091-6749(99)70525-7 · Cited by 163 · see citing works
Attained adult height after childhood asthma: Effect of glucocorticoid therapy
Silverstein, M. D., Yunginger, J. W., Reed, C. E., Petterson, T. M., Zimmerman, D., Li, J. C., & OFALLON, W. (1997) · Journal of Allergy and Clinical Immunology · doi:10.1016/s0091-6749(97)70072-1 · Cited by 162 · see citing works
Increased risk of serious pneumococcal disease in patients with asthma
Juhn, Y. J., Kita, H., Yawn, B. P., Boyce, T. G., Yoo, K. H., McGree, M. E., Weaver, A. L., Wollan, P. C., & Jacobson, R. M. (2008) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2008.07.029 · Cited by 157 · see citing works
Self-citationSummary health statistics for u.s. Adults: national health interview survey, 2011.
Schiller, J. S., Lucas, J. W., & Peregoy, J. A. (2012) · PubMed · Cited by 154 · see citing works
Long-Term Survival of a Cohort of Community Residents with Asthma
Silverstein, M. D., Reed, C. E., O’Connell, E. J., Melton, L. J., O’Fallon, W. M., & Yunginger, J. W. (1994) · New England Journal of Medicine · doi:10.1056/nejm199412083312301 · Cited by 148 · see citing works
Accuracy of the Death Certificate in a Population-Based Study of Asthmatic Patients
Hunt, L. W. (1993) · JAMA · doi:10.1001/jama.1993.03500150059027 · Cited by 137 · see citing works
Summary health statistics for u.s. Adults: national health interview survey, 2003.
Lethbridge‐Çejku, M., & Vickerie, J. L. (2005) · PubMed · Cited by 120 · see citing works
Mode of Delivery Is Associated With Asthma and Allergy Occurrences in Children
Salam, M. T., Margolis, H. G., McConnell, R., McGregor, J., Avol, E., & Gilliland, F. D. (2005) · Annals of Epidemiology · doi:10.1016/j.annepidem.2005.06.054 · Cited by 117 · see citing works
Safarova, M. S., Liu, H., & Kullo, I. (2016) · Journal of clinical lipidology · doi:10.1016/j.jacl.2016.08.001 · Cited by 102 · see citing works
Self-citationMode of delivery at birth and development of asthma: A population-based cohort study
Juhn, Y. J., Weaver, A. L., Katusic, S. K., & Yunginger, J. W. (2005) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2005.05.043 · Cited by 99 · see citing works
Self-citationJuhn, Y. J., Sauver, J. L. S., Katusic, S. K., Chanes, D. V., Weaver, A. L., & Yunginger, J. W. (2005) · Social Science & Medicine · doi:10.1016/j.socscimed.2004.11.034 · Cited by 97 · see citing works
Self-citationThe impact of requiring patient authorization for use of data in medical records research.
Yawn, B. P., Yawn, R. A., Gr, G., et al. (1998) · PubMed · Cited by 91 · see citing works
Increased risk of pertussis in patients with asthma
Capili, C., Hettinger, A., Rigelman-Hedberg, N. E., Fink, L., Boyce, T. G., Lahr, B. D., & Juhn, Y. J. (2011) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2011.11.020 · Cited by 86 · see citing works
Self-citationAdherence to Asthma Guidelines in Children, Tweens, and Adults in Primary Care Settings
Yawn, B. P., Rank, M. A., Cabana, M. D., et al. (2016) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2016.01.010 · Cited by 84 · see citing works
Wu, S., Sohn, S., Ravikumar, K. E., Wagholikar, K. B., Jonnalagadda, S., Liu, H., & Juhn, Y. J. (2013) · Annals of Allergy Asthma & Immunology · doi:10.1016/j.anai.2013.07.022 · Cited by 79 · see citing works
Self-citationInterobserver variability in medical record review: An epidemiological study of asthma
Beard, C. M., Yunginger, J. W., Reed, C. E., O’Connell, E. J., & Silverstein, M. D. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90117-6 · Cited by 74 · see citing works
Zheng, L., Wang, Y., Hao, S., Shin, A. Y., Jin, B., Ngo, A. D., Jackson-Browne, M. S., Feller, D. J., Fu, T., Zhang, K., et al. (2016) · JMIR Medical Informatics · doi:10.2196/medinform.6328 · Cited by 57 · see citing works
Genetics of asthma and allergy: What have we learned?
Meyers, D. A. (2010) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2010.07.012 · Cited by 51 · see citing works
Juhn, Y. J., Qin, R., Urm, S.-H., Katusic, S. K., & Chanes, D. V. (2010) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2009.12.998 · Cited by 51 · see citing works
Self-citationValidation of Claims Data Algorithms to Identify Nonmelanoma Skin Cancer
Eide, M. J., Tuthill, J. M., Krajenta, R. J., et al. (2012) · Journal of Investigative Dermatology · doi:10.1038/jid.2012.98 · Cited by 34 · see citing works
Childhood asthma and measles vaccine response
Juhn, Y. J., Kita, H., Lee, L. A., et al. (2006) · Annals of Allergy Asthma & Immunology · doi:10.1016/s1081-1206(10)60937-4 · Cited by 32 · see citing works
Self-citationHimes, B. E., Kohane, I. S., Ramoni, M. F., et al. (2008) · PubMed · Cited by 28 · see citing works
Patient-level temporal aggregation for text-based asthma status ascertainment
Wu, S. T., Juhn, Y. J., Sohn, S., et al. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002463 · Cited by 28 · see citing works
Self-citationAssessment of the association between atopic conditions and tympanostomy tube placement in children
Bjur, K. A., Lynch, R. L., Fenta, Y. A., Yoo, K. H., Jacobson, R. M., Li, X., & Juhn, Y. J. (2012) · Allergy and Asthma Proceedings · doi:10.2500/aap.2012.33.3529 · Cited by 27 · see citing works
Self-citationFord, E., Carroll, J., Smith, H., Davies, K., Koeling, R., Petersen, I., Rait, G., & Cassell, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010393 · Cited by 26 · see citing works
Risk of herpes zoster in children with asthma
Wi, C., Kim, B., Mehra, S., Yawn, B. P., Park, M. A., & Juhn, Y. J. (2015) · Allergy and Asthma Proceedings · doi:10.2500/aap.2015.36.3864 · Cited by 23 · see citing works
Self-citationWi, C., Park, M. A., & Juhn, Y. J. (2014) · Journal of Asthma · doi:10.3109/02770903.2014.952438 · Cited by 17 · see citing works
Self-citationAutomated application of clinical practice guidelines for asthma management.
Ertle, A. R., Campbell, E. M., & Hersh, W. (1996) · PubMed · Cited by 15 · see citing works
What accounts for the association between late preterm births and risk of asthma?
Voge, G. A., Carey, W. A., Ryu, E., King, K. S., Wi, C., & Juhn, Y. J. (2017) · Allergy and Asthma Proceedings · doi:10.2500/aap.2017.38.4021 · Cited by 12 · see citing works
Self-citationWi, C., Sohn, S., Ryu, E., et al. (2016) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2015.12.771 · Cited by 8 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Epidemiology of Asthma in Children and Adults
Dharmage, S. C., Perret, J. L., & Custovic, A. (2019) · Frontiers in Pediatrics · doi:10.3389/fped.2019.00246 · Cited by 1,200 · see citing works
Juhn, Y. J., Ryu, E., Wi, C.-I., King, K. S., Malik, M., Romero-Brufau, S., Weng, C., Sohn, S., Sharp, R. R., & Halamka, J. D. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac052 · Cited by 83 · see citing works
Self-citationNakatani, H., Nakao, M., Uchiyama, H., Toyoshiba, H., & Ochiai, C. (2020) · JMIR Medical Informatics · doi:10.2196/16970 · Cited by 62 · see citing works
- Authors
- Wi, C.-I., Sohn, S., Rolfes, M. C., Seabright, A., Ryu, E., Voge, G., Bachman, K. A., Park, M. A., Kita, H., Croghan, I. T., Liu, H., & Juhn, Y. J.
- Affiliations
- Department of Pediatric and Adolescent Medicine
- Division of Biomedical Statistics and Informatics, and
- Mayo Clinic (US)
- Asthma Epidemiology Research Unit
- Children's Minnesota (US)
- Corresponding author
- Juhn, Young J.
- ORCID
- Venue
- American Journal of Respiratory and Critical Care Medicine
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-08
- Volume
- 196
- Issue
- 4
- Pages
- 430-437
- ISSN
- 1073-449X (ISSN-L)
- 1535-4970
- PMC
- PMC5564673
- OpenAlex
- W2604265436
- Cited by
- 88
- References
- 54
- Citations by year
- 2026: 4 · 2025: 11 · 2024: 7 · 2023: 14 · 2022: 8 · 2021: 16 · 2020: 8 · 2019: 7 · 2018: 7 · 2017: 6
- FWCI
- 5.66
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- American Journal of Respiratory and Critical Care Medicine (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (UL1TR000135, TR000135)
- National Institute on Aging (R01AG034676, R01-AG34676)
- National Heart, Lung, and Blood Institute (R01HL126667)
- National Institute of Allergy and Infectious Diseases (R21AI116839)
- National Center for Advancing Translational Sciences (TR000135)
- Topics
- Health Sciences › Medicine › Physiology › Asthma and respiratory diseases
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- natural language processing, electronic health records, childhood asthma, asthma prevalence, allergic rhinitis, asthma risk factors, algorithm validation
- MeSH terms
- Adolescent, Asthma, Child, Child, Preschool, Female, Humans, Male, Minnesota, Natural Language Processing, Retrospective Studies, Risk Factors, Sensitivity and Specificity, Reproducibility of Results, Cohort Studies, Prevalence, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
