CoChat
Sign Up
Cited by 82Asthma and respiratory diseasesElectronic Health Records SystemsArtificial Intelligence in Healthcare and Education

Artificial intelligence-assisted clinical decision support for childhood asthma management: A randomized clinical trial

Open access
Seol, Hee Yun, Shrestha, Pragya, Muth, Joy Fladager, Wi, Chung-Il, Sohn, Sunghwan, Ryu, Euijung, Park, Miguel, Ihrke, Kathy +9 more·2021·PLOS ONE
DOI 10.1371/journal.pone.0255261PMC PMC8328289
Open this paper in CoChat — free accountOpen PDF

Abstract

Rationale: Clinical decision support (CDS) tools leveraging electronic health records (EHRs) have been an approach for addressing challenges in asthma care but remain under-studied through clinical trials. Objectives: To assess the effectiveness and efficiency of Asthma-Guidance and Prediction System (A-GPS), an Artificial Intelligence (AI)-assisted CDS tool, in optimizing asthma management through a randomized clinical trial (RCT). Methods: This was a single-center pragmatic RCT with a stratified randomization design conducted for one year in the primary care pediatric practice of the Mayo Clinic, MN. Children (<18 years) diagnosed with asthma receiving care at the study site were enrolled along with their 42 primary care providers. Study subjects were stratified into three strata (based on asthma severity, asthma care status, and asthma diagnosis) and were blinded to the assigned groups. Measurements: Intervention was a quarterly A-GPS report to clinicians including relevant clinical information for asthma management from EHRs and machine learning-based prediction for risk of asthma exacerbation (AE). Primary endpoint was the occurrence of AE within 1 year and secondary outcomes included time required for clinicians to review EHRs for asthma management. Main results: Out of 555 participants invited to the study, 184 consented for the study and were randomized (90 in intervention and 94 in control group). Median age of 184 participants was 8.5 years. While the proportion of children with AE in both groups decreased from the baseline (P = 0.042), there was no difference in AE frequency between the two groups (12% for the intervention group vs. 15% for the control group, Odds Ratio: 0.82; 95%CI 0.374–1.96; P = 0.626) during the study period. For the secondary end points, A-GPS intervention, however, significantly reduced time for reviewing EHRs for asthma management of each participant (median: 3.5 min, IQR: 2–5), compared to usual care without A-GPS (median: 11.3 min, IQR: 6.3–15); p<0.001). Mean health care costs with 95%CI of children during the trial (compared to before the trial) in the intervention group were lower than those in the control group (-$1,036 [-$2177, $44] for the intervention group vs. +$80 [-$841, $1000] for the control group), though there was no significant difference (p = 0.12). Among those who experienced the first AE during the study period (n = 25), those in the intervention group had timelier follow up by the clinical care team compared to those in the control group but no significant difference was found (HR = 1.93; 95% CI: 0.82–1.45, P = 0.10). There was no difference in the proportion of duration when patients had well-controlled asthma during the study period between the intervention and the control groups. Conclusions: While A-GPS-based intervention showed similar reduction in AE events to usual care, it might reduce clinicians’ burden for EHRs review resulting in efficient asthma management. A larger RCT is needed for further studying the findings. Trial registration: ClinicalTrials.gov Identifier: NCT02865967.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0255261

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Mayo Clinic researchers implemented RCTs to assess effectiveness and efficiency of ML models in predicting 1-year occurrence of asthma exacerbation.

    “In the realm of prognostic ML models, researchers from the Mayo Clinic implemented RCTs to assess the effectiveness and efficiency of ML models in predicting 1‐year occurrence of asthma exacerbation [28].”

    From Toward real‐world deployment of machine learning for health care: External validation, continual monitoring, and randomized clinical trials (2024) · cites this paper as [28]

    The Abstract (Methods and Measurements; page not provided) describes a one-year Mayo Clinic RCT of an asthma decision-support report that included an ML-based prediction of exacerbation risk. The Methods (Endpoints; page not provided) define one-year asthma exacerbation as an effectiveness outcome, with efficiency assessed through measures such as clinician time. The trial assessed the decision-support intervention, not the effectiveness or efficiency of the ML model at predicting exacerbations. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 56 · Cited by 82

Research works printed but not linked: 54

  1. [1]Martin-Sanchez F, Verspoor K: Big data in medicine is driving big changes. Yearb Med Inform 2014, 9(1): 14– 20. 25123716 PMC4287083
  2. [2]Gardner RL, Cooper E, Haskell J, Harris DA, Poplau S, Kroth PJ, et al: Physician stress and burnout: the impact of health information technology. Journal of the American Medical Informatics Association: JAMIA 2019, 26(2): 106– 114. doi: 10.1093/jamia/ocy145 30517663 PMC7647171 · DOI · Paper page
  3. [4]Linzer M, Bitton A, Tu SP, Plews-Ogan M, Horowitz KR, Schwartz MD, et al: The End of the 15–20 Minute Primary Care Visit. Journal of general internal medicine 2015, 30(11): 1584– 1586. doi: 10.1007/s11606-015-3341-3 25900539 PMC4617939 · DOI
  4. [5]Holmgren AJ, Downing NL, Bates DW, Shanafelt TD, Milstein A, Sharp CD, et al: Assessment of Electronic Health Record Use Between US and Non-US Health Systems. JAMA Internal Medicine 2020. PMC7737152 33315048
  5. [6]Dzau VJ, McClellan MB, McGinnis JM, Burke SP, Coye MJ, Diaz A, et al: Vital Directions for Health and Health Care: Priorities From a National Academy of Medicine Initiative. JAMA 2017, 317(14): 1461– 1470. doi: 10.1001/jama.2017.1964 28324029 · DOI
  6. [7]Bell LM, Grundmeier R, Localio R, Zorc J, Fiks AG, Zhang X, et al: Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics 2010, 125(4): e770– 777. doi: 10.1542/peds.2009-1385 20231191 · DOI
  7. [8]Shiffman RN, Freudigman M, Brandt CA, Liaw Y, Navedo DD: A guideline implementation system using handheld computers for office management of asthma: effects on adherence and patient outcomes. Pediatrics 2000, 105(4 Pt 1): 767– 773. doi: 10.1542/peds.105.4.767 10742318 · DOI
  8. [9]McCowan C, Neville RG, Ricketts IW, Warner FC, Hoskins G, Thomas GE: Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Medical informatics and the Internet in medicine 2001, 26(3): 191– 201. doi: 10.1080/14639230110067890 11706929 · DOI
  9. [10]Li A, Walling J, Ahn S, Kotliarov Y, Su Q, Quezado M, et al: Unsupervised analysis of transcriptomic profiles reveals six glioma subtypes. Cancer Res 2009, 69(5): 2091– 2099. doi: 10.1158/0008-5472.CAN-08-2100 19244127 PMC2845963 · DOI
  10. [11]Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al: Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. Jama 2016, 316(22): 2402– 2410. doi: 10.1001/jama.2016.17216 27898976 · DOI · Paper page
  11. [12]Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA: The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med 2018, 24(11): 1716– 1720. doi: 10.1038/s41591-018-0213-5 30349085 · DOI · Paper page
  12. [14]Angus DC: Randomized Clinical Trials of Artificial Intelligence. JAMA 2020, 323(11): 1043– 1045. doi: 10.1001/jama.2020.1039 32065828 · DOI · Paper page
  13. [16]Assurance NCfQ: Technical Specifications for Health Plans. The Healthcare Effectiveness Data and Information Set (HEDIS), vol. 2; 2016.
  14. [17]National Asthma E, Prevention P: Expert Panel Report 3 (EPR-3): Guidelines for the Diagnosis and Management of Asthma-Summary Report 2007. J Allergy Clin Immunol 2007, 120(5 Suppl): S94– 138. 17983880
  15. [18]Bisgaard H, Szefler S: Prevalence of asthma-like symptoms in young children. Pediatr Pulmonol 2007, 42(8): 723– 728. doi: 10.1002/ppul.20644 17598172 · DOI
  16. [19]Molis WE, Bagniewski S, Weaver AL, Jacobson RM, Juhn YJ: Timeliness of diagnosis of asthma in children and its predictors. Allergy 2008, 63(11): 1529– 1535. doi: 10.1111/j.1398-9995.2008.01749.x 18925889 · DOI
  17. [20]Yoo KH, Johnson SK, Voigt RG, Campeau LJ, Yawn BP, Juhn YJ: Characterization of asthma status by parent report and medical record review. The Journal of allergy and clinical immunology 2007, 120(6): 1468– 1469. doi: 10.1016/j.jaci.2007.09.008 17981319 · DOI
  18. [21]Juhn Y, Kung A, Voigt R, Johnson S: Characterisation of children’s asthma status by ICD-9 code and criteria-based medical record review. Primary care respiratory journal: journal of the General Practice Airways Group 2011, 20(1): 79– 83. doi: 10.4104/pcrj.2010.00076 21063669 PMC6602167 · DOI
  19. [22]Yunginger JW, Reed CE, O’Connell EJ, Melton L, O’Fallon WM, Silverstein MD: A community-based study of the epidemiology of asthma. Am Rev Respir Dis 1992, 146(4): 888– 894. doi: 10.1164/ajrccm/146.4.888 1416415 · DOI
  20. [23]Ducharme FM, Dell SD, Radhakrishnan D, Grad RM, Watson WTA, Yang CL, et al: Diagnosis and management of asthma in preschoolers: A Canadian Thoracic Society and Canadian Paediatric Society position paper. Canadian Respiratory Journal: Journal of the Canadian Thoracic Society 2015, 22(3): 135– 143. PMC4470545 25893310
  21. [24]Wi CI, Sohn S, Rolfes MC, Seabright A, Ryu E, Voge G, et al: Application of a Natural Language Processing Algorithm to Asthma Ascertainment. An Automated Chart Review. American journal of respiratory and critical care medicine 2017, 196(4): 430– 437. doi: 10.1164/rccm.201610-2006OC 28375665 PMC5564673 · DOI · Paper page
  22. [25]Wu ST, Juhn YJ, Sohn S, Liu H: Patient-level temporal aggregation for text-based asthma status ascertainment. Journal of the American Medical Informatics Association: JAMIA 2014, 21(5): 876– 884. doi: 10.1136/amiajnl-2013-002463 24833775 PMC4147607 · DOI
  23. [26]Wu ST, Sohn S, Ravikumar KE, Wagholikar K, Jonnalagadda SR, Liu H, et al: Automated chart review for asthma cohort identification using natural language processing: an exploratory study. Annals of allergy, asthma & immunology: official publication of the American College of Allergy, Asthma, & Immunology 2013, 111(5): 364– 369. PMC3839107 24125142
  24. [27]Yunginger JW, Reed CE, O’Connell EJ, Melton LJ, 3rd, O’Fallon WM, Silverstein MD: A community-based study of the epidemiology of asthma. Incidence rates, 1964–1983. Am Rev Respir Dis 1992, 146(4): 888– 894. doi: 10.1164/ajrccm/146.4.888 1416415 · DOI
  25. [28]Beard CM, Yunginger JW, Reed CE, O’Connell EJ, Silverstein MD: Interobserver variability in medical record review: an epidemiological study of asthma. J Clin Epidemiol 1992, 45(9): 1013– 1020. doi: 10.1016/0895-4356(92)90117-6 1432015 · DOI
  26. [29]Hunt LW Jr., Silverstein MD, Reed CE, O’Connell EJ, O’Fallon WM, Yunginger JW: Accuracy of the death certificate in a population-based study of asthmatic patients. Jama 1993, 269(15): 1947– 1952. 8464126
  27. [30]Silverstein MD, Reed CE, O’Connell EJ, Melton LJ, 3rd, O’Fallon WM, Yunginger JW: Long-term survival of a cohort of community residents with asthma. N Engl J Med 1994, 331(23): 1537– 1541. doi: 10.1056/NEJM199412083312301 7969322 · DOI
  28. [31]Bauer BA, Reed CE, Yunginger JW, Wollan PC, Silverstein MD: Incidence and outcomes of asthma in the elderly. A population-based study in Rochester, Minnesota. Chest 1997, 111(2): 303– 310. doi: 10.1378/chest.111.2.303 9041973 · DOI
  29. [32]Silverstein MD, Yunginger JW, Reed CE, Petterson T, Zimmerman D, Li JT, et al: Attained adult height after childhood asthma: effect of glucocorticoid therapy. J Allergy Clin Immunol 1997, 99(4): 466– 474. doi: 10.1016/s0091-6749(97)70072-1 9111490 · DOI
  30. [33]Juhn YJ, Qin R, Urm S, Katusic S, Vargas-Chanes D: The influence of neighborhood environment on the incidence of childhood asthma: a propensity score approach. J Allergy Clin Immunol 2010, 125(4): 838– 843 e832. doi: 10.1016/j.jaci.2009.12.998 20236695 PMC2962617 · DOI
  31. [34]Juhn YJ, Sauver JS, Katusic S, Vargas D, Weaver A, Yunginger J: The influence of neighborhood environment on the incidence of childhood asthma: a multilevel approach. Soc Sci Med 2005, 60(11): 2453– 2464. doi: 10.1016/j.socscimed.2004.11.034 15814171 · DOI
  32. [35]Juhn YJ, Weaver A, Katusic S, Yunginger J: Mode of delivery at birth and development of asthma: A population-based cohort study. J Allergy Clin Immun 2005, 116(3): 510– 516. doi: 10.1016/j.jaci.2005.05.043 16159617 · DOI
  33. [36]Yawn BP, Yunginger JW, Wollan PC, Reed CE, Silverstein MD, Harris AG: Allergic rhinitis in Rochester, Minnesota residents with asthma: frequency and impact on health care charges. J Allergy Clin Immunol 1999, 103(1 Pt 1): 54– 59. doi: 10.1016/s0091-6749(99)70525-7 9893185 · DOI
  34. [37]Seol HY, Rolfes MC, Chung W, Sohn S, Ryu E, Park MA, et al: Expert artificial intelligence-based natural language processing characterises childhood asthma. BMJ Open Respiratory Research 2020, 7(1). doi: 10.1136/bmjresp-2019-000524 33371009 PMC7011897 · DOI
  35. [38]Seol HY, Sohn S, Liu H, Wi CI, Ryu E, Park MA, et al: Early Identification of Childhood Asthma: The Role of Informatics in an Era of Electronic Health Records. Frontiers in pediatrics 2019, 7: 113. doi: 10.3389/fped.2019.00113 31001500 PMC6454104 · DOI
  36. [39]Juhn YJ: Risks for infection in patients with asthma (or other atopic conditions): Is asthma more than a chronic airway disease? The Journal of allergy and clinical immunology 2014, 134(2): 247– 257 e243. doi: 10.1016/j.jaci.2014.04.024 25087224 PMC4122981 · DOI · Paper page
  37. [40]St Sauver JL, Grossardt BR, Yawn BP, Melton LJ, 3rd, Pankratz JJ, Brue SM, et al: Data resource profile: the Rochester Epidemiology Project (REP) medical records-linkage system. Int J Epidemiol 2012, 41(6): 1614– 1624. doi: 10.1093/ije/dys195 23159830 PMC3535751 · DOI
  38. [41]St. Sauver JL, Grossardt BR, Leibson CL, Yawn BP, Melton Iii LJ, Rocca WA: Generalizability of Epidemiological Findings and Public Health Decisions: An Illustration From the Rochester Epidemiology Project. Mayo Clinic Proceedings 2012, 87(2): 151– 160. doi: 10.1016/j.mayocp.2011.11.009 22305027 PMC3538404 · DOI
  39. [42]Yawn BP, Wollan P, Kurland M, Scanlon P: A longitudinal study of the prevalence of asthma in a community population of school-age children. Journal of Pediatrics 2002, 140(5): 576– 581. 12032525
  40. [43]Zhong W, Finnie DM, Shah ND, Wagie AE, St. Sauver JL, Jacobson DJ, et al: Effect of Multiple Chronic Diseases on Health Care Expenditures in Childhood. J Prim Care Community Health 2015, 6(1): 2– 9. doi: 10.1177/2150131914540916 25001922 · DOI
  41. [44]Busse WW, Morgan WJ, Taggart V, Togias A: Asthma outcomes workshop: Overview. The Journal of allergy and clinical immunology 2012, 129(3): S1– S8. doi: 10.1016/j.jaci.2011.12.985 22386504 PMC4259286 · DOI
  42. [45]Lee MS, Ray KN, Mehrotra A, Giboney P, Yee HF Jr, Barnett ML: Primary care practitioners’ perceptions of electronic consult systems: A qualitative analysis. JAMA Internal Medicine 2018. doi: 10.1001/jamainternmed.2018.0738 29801079 PMC6145753 · DOI · Paper page
  43. [46]Visscher SL, Naessens JM, Yawn BP, Reinalda MS, Anderson SS, Borah BJ: Developing a standardized healthcare cost data warehouse. BMC Health Serv Res 2017, 17(1): 396. doi: 10.1186/s12913-017-2327-8 28606088 PMC5469019 · DOI
  44. [48]Cajigal S, Wells KE, Peterson EL, Ahmedani BK, Yang JJ, Kumar R, et al: Predictive Properties of the Asthma Control Test and Its Component Questions for Severe Asthma Exacerbations. The journal of allergy and clinical immunology In practice 2017, 5(1): 121– 127.e122. doi: 10.1016/j.jaip.2016.06.025 27544712 PMC5222777 · DOI
  45. [49]Murphy KR, Zeiger RS, Kosinski M, Chipps B, Mellon M, Schatz M, et al: Test for respiratory and asthma control in kids (TRACK): a caregiver-completed questionnaire for preschool-aged children. J Allergy Clin Immunol 2009, 123(4): 833– 839.e839. doi: 10.1016/j.jaci.2009.01.058 19348922 · DOI
  46. [50]Zeiger RS, Mellon M, Chipps B, Murphy KR, Schatz M, Kosinski M, et al: Test for Respiratory and Asthma Control in Kids (TRACK): Clinically meaningful changes in score. The Journal of allergy and clinical immunology 2011, 128(5): 983– 988. doi: 10.1016/j.jaci.2011.08.010 21906790 · DOI
  47. [51]Findley S, Rosenthal M, Bryant-Stephens T, Damitz M, Lara M, Mansfield C, et al: Community-based care coordination: practical applications for childhood asthma. Health Promot Pract 2011, 12(6 Suppl 1): 52s– 62s. doi: 10.1177/1524839911404231 22068360 · DOI
  48. [52]Gorin AA, Lenz EM, Cornelius T, Huedo-Medina T, Wojtanowski AC, Foster GD: Randomized Controlled Trial Examining the Ripple Effect of a Nationally Available Weight Management Program on Untreated Spouses. Obesity (Silver Spring, Md) 2018, 26(3): 499– 504. doi: 10.1002/oby.22098 29388385 PMC5838794 · DOI
  49. [53]Jackson SE, Steptoe A, Wardle J: The influence of partner’s behavior on health behavior change: the English Longitudinal Study of Ageing. JAMA Intern Med 2015, 175(3): 385– 392. doi: 10.1001/jamainternmed.2014.7554 25599511 · DOI
  50. [54]White E, Hurlich M, Thompson RS, Woods MN, Henderson MM, Urban N, et al: Dietary Changes Among Husbands of Participants in a Low-Fat Dietary Intervention. American Journal of Preventive Medicine 1991, 7(5): 319– 325. 1790038
  51. [55]Katheryn Courville PD: How a smartphone application changes the behavior, thinking and attitudes of interprofessional team members: finding, mechanisms using realist evaluaiton. In: 2018 Clinical Informatics Conference: 2018; Scottsdale, AZ; 2018.
  52. [56]Kaur H, Sohn S, Wi CI, Ryu E, Park MA, Bachman K, et al: Automated chart review utilizing natural language processing algorithm for asthma predictive index. BMC pulmonary medicine 2018, 18(1): 34. doi: 10.1186/s12890-018-0593-9 29439692 PMC5812028 · DOI
  53. [57]Lindgren P, Johnson J, Williams A, Yawn B, Pratt GC: Asthma exacerbations and traffic: examining relationships using link-based traffic metrics and a comprehensive patient database. Environ Health 2016, 15(1): 102. doi: 10.1186/s12940-016-0184-2 27809853 PMC5094142 · DOI
  54. [58]Sbihi H, Koehoorn M, Tamburic L, Brauer M: Asthma Trajectories in a Population-based Birth Cohort. Impacts of Air Pollution and Greenness. American journal of respiratory and critical care medicine 2017, 195(5): 607– 613. doi: 10.1164/rccm.201601-0164OC 27606967 · DOI

Authors
Seol, H. Y., Shrestha, P., Muth, J. F., Wi, C.-I., Sohn, S., Ryu, E., Park, M., Ihrke, K., Moon, S., King, K., Wheeler, P., Borah, B., Moriarty, J., Rosedahl, J., Liu, H., McWilliams, D. B., & Juhn, Y. J.
Affiliations
Mayo Clinic (US)
Corresponding author
Juhn, Young J.
ORCID
Venue
PLOS ONE
Publisher
Public Library of Science (PLoS)
Published
2021-08-02
Volume
16
Issue
8
Pages
e0255261
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0255261
PMC
PMC8328289
OpenAlex
W3191204346
Cited by
82
References
56
Citations by year
2026: 14 · 2025: 24 · 2024: 20 · 2023: 13 · 2022: 11
FWCI
6.91
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Foundation for the National Institutes of Health (R01 HL126667; R21AI142702)
  • Genentech (Innovative Methods to Improve Asthma Disease Management Program)
  • National Institutes of Health (GM008685-22)
Topics
  • Health Sciences › Medicine › Physiology › Asthma and respiratory diseases
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
childhood asthma, clinical decision support, artificial intelligence, asthma exacerbation prediction, electronic health records, primary care, asthma management, health care costs, machine learning prediction, pediatric asthma management, health care utilization
MeSH terms
Adolescent, Artificial Intelligence, Asthma, Child, Child, Preschool, Female, Humans, Male, Primary Health Care, Decision Support Systems, Clinical, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review