Cited by 6Attention Deficit Hyperactivity DisorderDevelopmental and Educational NeuropsychologyBehavioral and Psychological Studies

Sensitivity and Specificity of an Executive Function Screener at Identifying Children with ADHD and Reading Disability

Open access
Karr, Justin E., Kibby, Michelle Y., Jagger-Rickels, Audreyana C., Garcia-Barrera, Mauricio A.·2021·Journal of Attention Disorders

Abstract

Objective: This study evaluated the sensitivity/specificity of a global sum score (GSS) from the Behavior Assessment System for Children, Second Edition, Executive Function screener (BASC-2-EF) at classifying children with/without ADHD and/or reading disability (RD). Method: The BASC-2 Teacher/Parent Rating Scales (TRS/PRS) were completed for children (8-12 years old; 43.1% female) with no diagnosis ( n = 53), RD ( n = 34), ADHD ( n = 85), co-morbid RD/ADHD ( n = 36), and other diagnoses ( n = 15). Receiver operating characteristic (ROC) curve analyses evaluated the sensitivity/specificity of the BASC-2-EF GSS at discriminating between children with/without ADHD or RD. Results: Area under the curve (AUC) scores indicated the sensitivity/specificity of the BASC-2-EF GSS at discriminating between children with/without ADHD (TRS: AUC = .831, p < .001; PRS: AUC = .919, p < .001), with/without RD (TRS: AUC = .724, p = .001; PRS: AUC = .615, p = .101), and with ADHD or RD through post hoc analysis (TRS: AUC = .674, p = .006; PRS: AUC = .819, p < .001). Conclusion: The findings support utilizing the BASC-2-EF GSS when differentiating ADHD from RD and typical development.

Integrity signals

Registry

Crossref · 10.1177/1087054718763878

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

4 of 20 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Correlations between cognitive and behavioral measures often present as non-significant.

    “considering that correlations between cognitive and behavioral measures often present as non-significant (Toplak et al., 2013).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Toplak et al., 2013 source

      The section “Summary of the measurable associations between performance-based and rating measures of executive function” reports that only 68 of 286 correlations (24%) were statistically significant. This supports the claim for performance-based and behavioral-rating measures of executive function, but the source does not establish it for cognitive and behavioral measures generally. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • Interpretationcitation

    The discrepancies between teacher and parent AUC magnitudes could result from rater biases and differences in the settings where raters interact with children.

    “They also could result from both the biases of raters and the differences in settings in which the raters interact with children (Narad et al., 2015).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Narad et al., 2015 source

      The paper says parent–teacher rating discrepancies may result from reporter bias and differences between home and school settings (Abstract; Discussion, page unavailable). It does not address AUC magnitudes specifically, so it supports the proposed explanation for rating differences but not its application to AUC differences. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Findingown data

    For the parent rating scale, the four-factor model had the best statistical fit (CFI=.960, RMSEA=.080).

    “In terms of statistical fit, the four-factor model showed the best fit for both scales (TRS-C: CFI=.942, RMSEA=.087; PRS-C: CFI=.960, RMSEA=.080).”
    Quote in the paperNot a substantial claim
  • Findingown data

    For the parent rating scale, internal consistency for all BASC-2-EF items was excellent (Cronbach's α=.913).

    “With these final item assignments, the internal consistency (i.e., Cronbach’s α) for all items showed excellent reliability (TRS-C=.952; PRS-C=.913)”
    Quote in the paperNot a substantial claim
  • Findingown data

    Lower reliability of parent ratings was observed in the current findings.

    “Lower reliability of parent ratings also was observed in the current findings”
    Quote in the paperNot a substantial claim
  • May combine two claims. 13
  • Classified as probably not a substantial claim. 11
  • Not yet classified. 1
  • May combine two claims.

    Premisecitation

    Parents have been shown to have better diagnostic determination for ADHD than teachers, although a composite from multiple raters is better diagnostically than relying on one rater.

    “Parents have been shown to have better diagnostic determination for ADHD than teachers (despite bias and reliability issues), although utilizing a composite from multiple raters is better diagnostically than relying on one rater (O’Neill, Schneiderman, Rajendran, Marks, & Halperin, 2014; Willcutt, 2012).”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites O’Neill, Schneiderman, Rajendran, Marks, & Halperin, 2014

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Willcutt, 2012 source

      The 'Diagnostic Algorithms Used by Studies Included in the Meta-Analysis' section (page unavailable) says parent and teacher raters identify partly nonoverlapping groups, supporting the value of considering more than one rater. The 'Algorithms to Combine Multiple Sources of Clinical Information' section (page unavailable) reports that one combined-rating rule had high positive predictive power, but missed many cases and concludes that no single algorithm is best for every purpose. The paper does not establish that parents are diagnostically better than teachers or that a multi-rater composite is generally better than one rater. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Classified as probably not a substantial claim.

    Premisecitation

    Ratings of executive behavior may measure a different type of executive function than performance-based measures examine in ADHD assessment.

    “it has been proposed that ratings of executive behavior may measure a different type of executive function than what performance-based measures examine in ADHD assessment (Toplak, West, & Stanovich, 2013)”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Toplak, West, & Stanovich, 2013 source
  • Not yet classified.

    Finding

    For children with RD versus children without RD, the BASC-2-EF GSS from the parent rating scale had an AUC of .615 (95% CI .481-.748), p=.101.

    “and children with and without RD (TRS-C: AUC=.724 [.612, .835], p=.001; PRS-C: AUC=.615 [.481, .748], p=.101)”
    Quote in the paperNot yet classified
  • May combine two claims.

    Findingown data

    For ADHD assessment, both teacher and parent raters had highly significant AUC values.

    “For ADHD assessment, both raters had highly significant AUC values”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    When comparing children with and without RD, teacher ratings showed greater executive dysfunction among RD children, whereas parent ratings did not.

    “however, when comparing children with and without RD, the teacher ratings showed greater executive dysfunction among the RD children, whereas the parent ratings did not.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Both teacher and parent ratings produced significant AUC values across all ROC curve analyses.

    “Although both the teacher and parent ratings resulted in significant AUC values, there were slight differences in the magnitude of AUC values between the two raters across all ROC curve analyses.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The BASC-2-EF is a four-factor screener embedded in the TRS-C and PRS-C that effectively measured problem solving, attentional control, behavioral control, and emotional control.

    “a four-factor screener embedded in each form that effectively measured four executive-related constructs: problem solving, attentional control, behavioral control, and emotional control (Karr & Garcia-Barrera, 2017).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Karr & Garcia-Barrera, 2017 source
  • May combine two claims.

    Interpretationown data

    The ROC results met standards for good and excellent AUC values for the teacher and parent rating scales, respectively, when differentiating ADHD from typical development, showing strong sensitivity and specificity of the BASC-2-EF GSS.

    “The results of these analyses met standards for good and excellent AUC values for the TRS-C and PRS-C, respectively, when differentiating ADHD from typical development, evidencing strong sensitivity and specificity of the BASC-2-EF GSS.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The behavioral assessment of executive functions has become increasingly popular in both research and clinical practice.

    “The behavioral assessment of executive functions has become increasingly popular in both research (e.g., Barkley & Fischer, 2011; Sadeh et al., 2012) and clinical practice (Rabin et al., 2016)”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Barkley & Fischer, 2011 source
    • cites Sadeh et al., 2012 source
    • cites Rabin et al., 2016 source
  • May combine two claims.

    Premisecitation

    The BASC-2-EF has only been derived using the BASC-2 normative data, which consisted entirely of typically developing children, and it has not yet been clinically validated.

    “the BASC-2-EF has only been derived using the BASC-2 normative data, consisting entirely of typically developing children (Karr & Garcia-Barrera, 2017), and it has yet to be clinically validated.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Karr & Garcia-Barrera, 2017 source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The four-factor measurement model of the BASC-2-EF showed optimal statistical fit, and each factor had excellent internal consistency.

    “The four-factor measurement model of the BASC-2-EF showed optimal statistical fit, and each factor had excellent internal consistency.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Karr & Garcia-Barrera, 2017 source
  • May combine two claims.

    Interpretationown data

    A teacher may provide a more reliable rating of executive functions, whereas a parent may provide a rating with better diagnostic utility for ADHD.

    “A teacher may provide a more reliable rating of executive functions, whereas a parent may provide a rating with better diagnostic utility for ADHD.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The discrepancies between teacher and parent AUC magnitudes could result from the ADHD diagnostic process, in which a parent interview and DSM-IV questionnaire were the primary sources of diagnostic information.

    “These discrepancies could result from the diagnostic process for ADHD, where a clinical interview and DSM-IV questionnaire completed with the parent were the primary sources of information to determine a diagnosis.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    Neither parent nor teacher ratings on the BASC-2-EF are independently diagnostic of ADHD.

    “Similarly, neither the parent nor teacher ratings on the BASC-2-EF are independently diagnostic of ADHD”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Within a classroom, a child with RD may more quickly disengage or become frustrated with a reading task, which may be perceived as reduced attentional or emotional control.

    “Within a classroom setting, a child with RD may more quickly disengage or become frustrated with a reading task, which may be perceived as reduced attentional or emotional control, respectively.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Differences in parent and teacher ratings have been historically evident, such as more inconsistent and biased ratings provided by parents than teachers.

    “Differences in parent and teacher ratings have been historically evident, such as more inconsistent and biased ratings provided by parents than teachers (Hartman, Rhee, Willcutt, & Pennington, 2007; Lochman, 1995).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Hartman, Rhee, Willcutt, & Pennington, 2007 source
    • cites Lochman, 1995 source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Parent raters may provide more one-on-one support to their children when reading at home, and thus may not observe as many of these behaviors.

    “In contrast, parent raters may provide more one-on-one support to their children when reading at home, and, thus, may not observe as many of these behaviors.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    The current findings will more likely support diagnostic decision making in archival datasets.

    “whereas the current findings will more likely support diagnostic decision making in archival datasets.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The BASC-2-EF and later iterations could assist with diagnostic decision making.

    “the BASC-2-EF and later iterations (i.e., BASC-3) could assist with such diagnostic decision making”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Premisecitation

    ADHD status was a significant predictor of all four factors measured by the BASC-EF.

    “ADHD status was a significant predictor of all four factors measured by the BASC-EF (Garcia-Barrera et al., 2015).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Garcia-Barrera et al., 2015 source
  • May combine two claims.

    Premisecitation

    The BASC was not originally developed as a diagnostic measure for ADHD, but it has demonstrated clinical utility in aiding its diagnosis.

    “Although the BASC was not originally developed as a diagnostic measure for ADHD, the instrument has demonstrated clinical utility in aiding its diagnosis (e.g., Jarratt, Riccio, & Siekierski, 2005; Pineda et al., 2005).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Jarratt, Riccio, & Siekierski, 2005 source
    • cites Pineda et al., 2005 source
21 of 21

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Post-hoc teacher confidence interval conflicts with its p-value

    Minor

    The reported 95% confidence interval for the teacher-rated ADHD-versus-RD AUC includes 0.5, the no-discrimination value, yet its reported p-value is .006. These give conflicting indications of statistical significance for the same comparison.

  • Post-hoc cutoff omitted despite the stated cutoff rule

    Minor

    The methods say a cutoff was calculated whenever an analysis had an AUC of at least .80. The post-hoc parent analysis had an AUC of .819, but the results explicitly say no post-hoc cutoff was calculated.

2 of 2

Cites

Flagged first, then by citations

1 of the 36 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. Garcia-Barrera M. A., 2006, Psychodiagnostic assessment of children: Dimensional and categorical methods, 319
  2. Steiger J. H., 1989, EzPATH: A supplementary module for SYSTAT and SYGRAPH

Cited by

Sorted by citations

Authors
Karr, J. E., Kibby, M. Y., Jagger-Rickels, A. C., & Garcia-Barrera, M. A.
Affiliations
  • University of Victoria (CA)
  • Southern Illinois University Carbondale (US)
  • University of Victoria, British Columbia, Canada
Venue
Journal of Attention Disorders
Published
2018-03-22
Volume
25
Issue
1
Pages
134-140
ISSN
  • 1087-0547 (ISSN-L)
  • 1557-1246
OpenAlex
W2789431015
Cited by
6
References
36
Citations by year
2025: 1 · 2024: 1 · 2022: 2 · 2021: 1 · 2020: 1
FWCI
0.39
Type
journal article
Language
en
Open access
Locations
  • Journal of Attention Disorders (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • OpenSIUC (Southern Illinois University Carbondale) (submittedVersion)
Funders
Eunice Kennedy Shriver National Institute of Child Health and Human Development (R03HD048752, R15HD065627)
Topics
  • Health Sciences › Medicine › Psychiatry and Mental health › Attention Deficit Hyperactivity Disorder
  • Social Sciences › Psychology › Neuropsychology and Physiological Psychology › Developmental and Educational Neuropsychology
  • Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
Keywords
executive function, attention deficit hyperactivity disorder, reading disorders, sensitivity and specificity, diagnostic screening, parent and teacher ratings, children
MeSH terms
Attention Deficit Disorder with Hyperactivity, Child, Dyslexia, Female, Humans, Learning Disabilities, Male, Sensitivity and Specificity, Executive Function

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review