CoChat
Sign Up
Cited by 32Electronic Health Records SystemsClinical Reasoning and Diagnostic SkillsArtificial Intelligence in Healthcare and Education

Impact of Mobile Device-Based Clinical Decision Support Tool on Guideline Adherence and Mental Workload

Open access
Richardson, Katherine M., Fouquet, Sarah D., Kerns, Ellen, McCulloh, Russell J.·2019·Academic Pediatrics
DOI 10.1016/j.acap.2019.03.001PMC PMC6732014
Open this paper in CoChat — free account

Abstract

Objective To evaluate the individual-level impact of an electronic clinical decision support (ECDS) tool, PedsGuide, on febrile infant clinical decision making and cognitive load. Methods A counterbalanced, prospective, crossover simulation study was performed among attending and trainee physicians. Participants performed simulated febrile infant cases with use of PedsGuide and with standard reference text. Cognitive load was assessed using the NASA-Task Load Index (NASA-TLX), which determines mental, physical, temporal demand, effort, frustration, and performance. Usability was assessed with the System Usability Scale (SUS). Scores on cases and NASA-TLX scores were compared between condition states. Results A total of 32 participants completed the study. Scores on febrile infant cases using PedsGuide were greater compared with standard reference text (89% vs 72%, P = .001). NASA-TLX scores were lower (ie, more optimal) with use of PedsGuide versus control (mental 6.34 vs 11.8, P < .001; physical 2.6 vs 6.1, P = .001; temporal demand 4.6 vs 8.0, P = .003; performance 4.5 vs 8.3, P < .001; effort 5.8 vs 10.7, P < .001; frustration 3.9 vs 10, P < .001). The SUS had an overall score of 88 of 100 with rating of acceptable on the acceptability scale. Conclusions Use of PedsGuide led to increased adherence to guidelines and decreased cognitive load in febrile infant management when compared with the use of a standard reference tool. This study employs a rarely used method of assessing ECDS tools using a multifaceted approach (medical decision-making, assessing usability, and cognitive workload,) that may be used to assess other ECDS tools in the future.

Verified 2026-10-05 · 2 references not checked yet

Integrity signals

Registry

Crossref · 10.1016/j.acap.2019.03.001

Cites

38 of 40 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 23 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    In this simulation study, mean scores on febrile infant cases were higher when physicians used PedsGuide than when they used the standard reference text (89% vs 72%, p=0.001).

    “Scores on febrile infant cases using PedsGuide were higher compared to standard reference text (89% vs. 72%, p=0.001).”
    • Also stated in: § Results · plain
      “Mean scores on the febrile infant cases were significantly higher with use of PedsGuide versus use of standard reference tool 89% versus 72%, (p=0.001) (Fig. 2).”
    Quote in the paperSubstantial claim
  • Findingown data

    NASA-TLX performance scores were lower, indicating better self-rated performance, with PedsGuide than with control (4.5 vs 8.3, p<0.001; results 4.47 vs 8.34).

    “NASA-TLX scores were lower (i.e. more optimal) with use of PedsGuide vs. control (mental 6.34 vs. 11.8, p<0.001; physical 2.6 vs. 6.1, p=0.001; temporal demand 4.6 vs. 8.0, p=0.003; performance 4.5 vs. 8.3, p<0.001; effort 5.8 vs. 10.7, p<0.001; frustration (3.9 vs. 10, p<0.001).”
    • Also stated in: § Results · plain
      “Respondents also self-reported better performance when completing simulations with PedsGuide (4.47 vs 8.34, p<0.001).”
    Quote in the paperSubstantial claim
  • Findingown data

    NASA-TLX physical demand scores were lower with PedsGuide than with control (2.6 vs 6.1, p=0.001).

    “NASA-TLX scores were lower (i.e. more optimal) with use of PedsGuide vs. control (mental 6.34 vs. 11.8, p<0.001; physical 2.6 vs. 6.1, p=0.001; temporal demand 4.6 vs. 8.0, p=0.003; performance 4.5 vs. 8.3, p<0.001; effort 5.8 vs. 10.7, p<0.001; frustration (3.9 vs. 10, p<0.001).”
    • Also stated in: § Results · plain
      “Analysis of NASA-TLX responses revealed that participants reported experiencing less frustration (p<0.001), mental demand (p<0.001), physical demand (p=0.001), temporal demand (p=0.003), and overall effort (p<0.001) with ECDS versus the standard reference tool (Fig. 3).”
    Quote in the paperSubstantial claim
  • Findingown data

    NASA-TLX mental demand scores were lower with PedsGuide than with control (6.34 vs 11.8, p<0.001).

    “NASA-TLX scores were lower (i.e. more optimal) with use of PedsGuide vs. control (mental 6.34 vs. 11.8, p<0.001; physical 2.6 vs. 6.1, p=0.001; temporal demand 4.6 vs. 8.0, p=0.003; performance 4.5 vs. 8.3, p<0.001; effort 5.8 vs. 10.7, p<0.001; frustration (3.9 vs. 10, p<0.001).”
    • Also stated in: § Results · plain
      “Analysis of NASA-TLX responses revealed that participants reported experiencing less frustration (p<0.001), mental demand (p<0.001), physical demand (p=0.001), temporal demand (p=0.003), and overall effort (p<0.001) with ECDS versus the standard reference tool (Fig. 3).”
    Quote in the paperSubstantial claim
  • Findingown data

    NASA-TLX temporal demand scores were lower with PedsGuide than with control (4.6 vs 8.0, p=0.003).

    “NASA-TLX scores were lower (i.e. more optimal) with use of PedsGuide vs. control (mental 6.34 vs. 11.8, p<0.001; physical 2.6 vs. 6.1, p=0.001; temporal demand 4.6 vs. 8.0, p=0.003; performance 4.5 vs. 8.3, p<0.001; effort 5.8 vs. 10.7, p<0.001; frustration (3.9 vs. 10, p<0.001).”
    • Also stated in: § Results · plain
      “Analysis of NASA-TLX responses revealed that participants reported experiencing less frustration (p<0.001), mental demand (p<0.001), physical demand (p=0.001), temporal demand (p=0.003), and overall effort (p<0.001) with ECDS versus the standard reference tool (Fig. 3).”
    Quote in the paperSubstantial claim
  • May combine two claims. 13
  • Classified as probably not a substantial claim. 20

  • May combine two claims.

    Findingcitation

    The average converted SUS score for PedsGuide was 88 out of 100, corresponding to acceptable on the acceptability scale, excellent on the adjective scale, and B on the grade scale.

    “The SUS had an overall score of 88 out of 100 with rating of acceptable on the acceptability scale.”
    • Also stated in: § Results · plain
      “The average converted scale derived from the SUS was 88 out of 100, which translated to a “B” on the overall grade scale, “excellent” on the adjective ratings scale, and “acceptable” on the acceptability ranges.34”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] Brooke, 2013, SUS: a retrospective, J Usability Stud

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Many participants felt PedsGuide helped with being thorough about all aspects of febrile infant care and made them feel safer.

    “Many participants felt that PedsGuide helped with being “thorough about all aspects [of febrile infant care],” and expressed that PedsGuide “makes me feel safer,” and”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants reported strong agreement that PedsGuide was easy to use and could be learned quickly, and disagreed that it required technical support or prior knowledge.

    “There was strong agreement with favorable factors such as the app being easy to use and that use could be learned quickly. Users disagreed that PedsGuide required technical support or prior knowledge to be able to use the app (Fig. 4).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A total of 32 participants completed the study.

    “A total of 32 participants completed the study.”
    • Also stated in: § Results · plain
      “A total of 290 resident and attending physicians were sent the recruitment email; 32 participants completed the study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Twenty (63%) participants had used PedsGuide prior to participating in the study.

    “A total of 20 (63%) participants had used PedsGuide prior to participating in the study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Feedback from participants regarding PedsGuide usability varied.

    “Feedback from this study varied in regards to usability.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Use of PedsGuide led to increased adherence to guidelines and decreased cognitive load in febrile infant management during standardized scenarios compared with a standard non-electronic reference tool.

    “Use of PedsGuide led to increased adherence to guidelines and decreased cognitive load in febrile infant management when compared to use of a standard reference tool.”
    • Also stated in: § Discussion · boosted
      “This study demonstrates that use of the ECDS tool PedsGuide led to an increase in adherence to evidence-based recommendations as demonstrated by higher scores on simulation cases when PedsGuide was used as compared to a standard reference tool.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The study demonstrates a multi-faceted approach combining medical decision-making, usability, and cognitive workload assessment that may be used to evaluate other ECDS tools in the future.

    “This study employs a rarely used method of assessing ECDS tools using a multi-faceted approach (medical decision-making, assessing usability, and cognitive workload,) that may be utilized to assess other ECDS tools in the future.”
    • Also stated in: § Discussion · boosted
      “Our study demonstrates how usability assessment, cognitive effort/performance assessment, and simulation can be utilized to assess the impact of ECDS tool use on a clinician’s medical decision-making and cognitive workload.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ECDS tools are one way to disseminate CPGs on a large scale, as evidenced by the deployment of PedsGuide in a national standardization project.

    “ECDS tools are one such way to disseminate CPGs on a large scale, as evidenced by the recent deployment of PedsGuide in a national standardization project.21”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] McCulloh, 2018, Development and implementation of a mobile device-based pediatric electronic decision support tool as part of a national practice standardization project, J Am Med Informatics Assoc source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Counterbalance was completed in blocks to ensure equal distribution of condition states for simulation condition and equal numbers of resident and attending physician slots.

    “Counterbalance was completed in blocks to ensure equal distribution of condition states for simulation condition, and an equal number of slots was allocated to resident and attending physicians.31”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Lin, 2015, The pursuit of balance: an overview of covariate-adaptive randomization techniques in clinical trials, Contemp Clin Trials source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The multi-faceted approach described has the potential to enhance future ECDS tool development if utilized prior to release.

    “Our approach described in the present study has the potential to enhance future ECDS tool development, if utilized prior to release of the ECDS tool.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Use of The Harriet Lane Handbook as control rather than an alternative electronic health resource may not provide an optimal comparison of ECDS tools.

    “Lastly, use of The Harriet Lane Handbook as a control, rather than an alternative electronic health resource, may not provide an optimal comparison of ECDS tools.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    ECDS tools may be used to recognize clinical conditions or condition states, but this does not always translate to desired clinical effect.

    “ECDS tools may be used to recognize clinical conditions or condition states, however this does not always translate to desired clinical effect.36”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [36] Kharbanda, 2018, Evaluation of an electronic clinical decision support tool for incident elevated BP in adolescents, Acad Pediatr source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Increased information assimilation results in increased cognitive load, which can lead to medical errors.

    “Increased information assimilation results in increased cognitive load which can lead to medical errors.2”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [2] Wilbanks, 2018, A review of measuring the cognitive workload of electronic health records, Comput Inform Nurs

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Use of clinical vignettes without high-fidelity simulation did not provide a real-world experience for participants.

    “First, the use of clinical vignettes without high-fidelity simulation did not provide a real-world experience for participants.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The study was a counterbalanced, prospective, cross-over simulation study.

    “This study was a counterbalanced, prospective cross-over simulation study to assess the individual-level impact on performance and cognitive load of PedsGuide on febrile infant management.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Some ECDS tools evaluated in the past have not been suitable for the task designed, have had errors or violations that could cause harm, or have not been user-friendly.

    “Some ECDS tools evaluated in the past have not been suitable for the task designed,8 have had errors or violations that could cause harm to the patient, or have not been user-friendly.20”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [8] Kim, 2016, Exploring the usability of mobile apps supporting radiologists’ training in diagnostic decision making, J Am Coll Radiol source

      Not checked in the free run.

    • cites [20] Beuscart-Zéphir, 2007, IMIA Yearbook of Medical Informatics

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation

    NASA-TLX is a validated subjective workload assessment tool measuring six domains of cognitive workload.

    “After each case was performed each participant completed the NASA-Task Load Index (NASA-TLX), a validated subjective workload assessment tool which measures 6 different domains impacting cognitive workload.32”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32] Hart, 1988, Development of NASA-TLX (Task Load Index): results of empirical and theoretical research, Adv Psychol source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    When a clinical practice guideline is adopted by an institution, more uniform care is provided to febrile infants, which can result in more appropriate use of diagnostic tests, antibiotics, hospitalization, and lower health care costs.

    “However, when a clinical practice guideline (CPG) is adopted by an institution there is more uniform care provided to febrile infants,24 which can result in more appropriate use of diagnostic tests, antibiotics, hospitalization,25 and lower health care costs.26”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [24] Aronson, 2015, Association of clinical practice guidelines with emergency department management of febrile infants ≤56 days of age, J Hosp Med source

      Not checked in the free run.

    • cites [25] Byington, 2012, Costs and infant outcomes after implementation of a care process model for febrile infants, Pediatrics source

      Not checked in the free run.

    • cites [26] Baker, 1993, Outpatient management without antibiotics of fever in selected infants, J Pediatr

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Implementation of CPGs poses significant benefits but dissemination and implementation on a large scale can be a challenge.

    “Thus, implementation of CPGs to help guide febrile infant management poses significant benefits to patients, families, and the healthcare system; however dissemination and implementation of CPGs on a large scale can be a challenge.27”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] McCulloh, 2012, The challenge-and promise-of local clinical practice guidelines, Pediatrics source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The Harriet Lane Handbook was determined to be the most appropriate comparator given its broad availability across clinical settings and the lack of a single universal electronic reference/tool.

    “However, given the broad availability of The Harriet Lane Handbook across clinical settings, and the lack of a single, universal electronic reference/tool, The Harriet Lane Handbook was determined to be the most appropriate comparator.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    The answer key for the cases was created in accordance with previously developed febrile risk stratification criteria, especially modified Rochester criteria.

    “An answer key was created for these cases in accordance with previously developed febrile risk stratification criteria, especially modified Rochester criteria.26,29”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [26] Baker, 1993, Outpatient management without antibiotics of fever in selected infants, J Pediatr

      Not checked in the free run.

    • cites [29] Baraff, 2008, Management of infants and young children with fever without source, Pediatr Ann

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    SUS is a subjective usability measurement whose applicability has been extensively documented.

    “The SUS is a subjective measurement of the usability of an application and was used in this study to assess participants’ perceptions of the usability of PedsGuide. The SUS was first developed in the 1980s, its applicability to determine usability has been extensively documented in the literature.34”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [33] Brooke, 1996, SUS-A "quick and dirty" usability scale

      Not checked in the free run.

    • cites [34] Brooke, 2013, SUS: a retrospective, J Usability Stud

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductreasoning

    In a simulated setting there may be a decrease in stress or cognitive load because it is not a real patient being assessed.

    “Additionally, in a simulated setting there may be a decrease in stress or cognitive load as it is not a real patient being assessed.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Medical knowledge and information is increasing faster than the human mind can assimilate.

    “Medical knowledge and information is increasing at a rate faster than the human mind can assimilate.1”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] Densen, 2011, Challenges and opportunities facing medical education, Trans Am Clin Climatol Assoc

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation

    Mobile applications can be particularly useful in settings where providers lack access to expertise.

    “Given that a majority of providers use their cell phone in clinical practice, mobile applications can be particularly useful in settings where such providers lack access to expertise. 4,5”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [4] Ginsburg, 2015, MPneumonia: development of an innovative mHealth application for diagnosing and treating childhood pneumonia and other childhood illnesses in low-resource settings, PLoS One source

      Not checked in the free run.

    • cites [5] Wallace, 2012, “It's on my iPhone”: attitudes to the use of mobile computing devices in medical education, a mixed-methods study, BMJ Open source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    An a priori power analysis indicated a minimum of 30 participants for 95% power at a .05 two-tailed significance criterion and effect size 0.7.

    “An a priori power analysis indicated a minimum of 30 participants in order to have 95% power for .05 criterion (two-tailed) of statistical significance at an effect size of 0.7.”
    Quote in the paperSubstantial claim
27 of 27

  • The paper relies on this without support.

    The functionality of PedsGuide is identical across Android and iPhone platforms.

    “The functionality of the app is identical across all platforms (Android or iPhone).”

  • The paper relies on this without support.

    The two febrile infant simulation cases are comparable in difficulty, so that counterbalancing adequately controls for case differences.

    “Two febrile infant scenarios were created by the research team. These scenarios included a low risk infant presenting with fever at 40 days of age (case 1) and a higher risk infant presenting at 28 days of age (case 2).”
    “Half of the participants started with case number 1 and half started with case number 2.”

  • The paper relies on this without support.

    ECDS tools can mitigate mental fatigue and disseminate knowledge.

    “One way to mitigate mental fatigue and disseminate knowledge is through electronic clinical decision support (ECDS) tools.”

  • The paper relies on this without support.

    Variation in management of young febrile infants is due in part to the availability of different criteria for determining an infant’s risk of SBI.

    “Variation in the management of young febrile infants is due in part to the availability of different criteria for determining an infant’s risk of SBI.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Environmental factors (interruptions), task demand (task time, task difficulty), technology (system usability) and individual characteristics (professional competency, self-efficacy and mood) are key variables that influence psychophysiological responses, workload modification and performance in human-machine interaction tasks.

    “In this framework, environmental factors (interruptions), along with the factors of task demand (task time, task difficulty) [19], technology (system usability) [20] and individual characteristics (professional competency, self-efficacy and mood), are the key variables that influence psychophysiological responses, workload modification and performance in human–machine interaction tasks [21].”

    From Workflow interruption and nurses’ mental workload in electronic health record tasks: An observational study (2023) · cites this paper as [20]

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. 2015, Mobile medical applications: guidance for industry and Food and Drug Administration Staff, Food Drug Adm, 1

Cited by

Sorted by citations

Related

Authors
Richardson, K. M., Fouquet, S. D., Kerns, E., & McCulloh, R. J.
Affiliations
Children's Mercy Hospital (US)
Corresponding author
Richardson, Katherine M.
ORCID
Venue
Academic Pediatrics
Publisher
Elsevier BV
Published
2019-09
Volume
19
Issue
7
Pages
828-834
ISSN
1876-2859 (ISSN-L)
DOI
10.1016/j.acap.2019.03.001
PMC
PMC6732014
OpenAlex
W2921222672
Cited by
32
References
38
Citations by year
2026: 5 · 2024: 2 · 2023: 9 · 2022: 8 · 2021: 3 · 2020: 3 · 2019: 1 · 2018: 1
FWCI
4.09
Type
journal article
Language
en
Open access
Licence
Locations
  • Academic Pediatrics (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Institutes of Health
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
electronic clinical decision support, febrile infants, guideline adherence, mental workload, usability, clinical decision-making, simulated clinical scenarios, NASA-TLX, System Usability Scale, pediatric emergency care
MeSH terms
Adult, Cognition, Mental Fatigue, Female, Fever, Humans, Infant, Infant, Newborn, Male, Middle Aged, Workload, Patient Simulation, Guideline Adherence, Decision Support Systems, Clinical, Cell Phone, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-05.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-05.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review