Concordance of teacher-rated and performance-based measures of executive functioning in preschoolers
The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
The most common method used to evaluate child behavior and functioning is rating scales completed by parents and/or teachers. Given that executive functioning (EF) plays a fundamental role in the developing child's cognitive, behavioral, and social-emotional development, it would be ideal if ratings of EF and performance-based EF measures assess the same construct. However, most studies report a small to negligible association between performance-based measures and ratings of EF. There are few studies investigating this association for preschoolers, and most only include parent ratings. Teachers may be more reliable reporters of EF behaviors due to the higher demand for EF skills in the preschool setting than at home and because teachers may have a better sense of what behaviors are normative. In this study, we reviewed the associations between three EF rating scales completed by teachers on 243 preschool children. Results showed small to moderate correlations with EF measures of inhibition and cognitive flexibility/switching for all three scales, with the strongest associations observed between Child Behavior Rating Scale (CBRS) Behavioral Regulation subscale and child EF measures. Exploratory multivariate path analyses showed that, after controlling for age, sex, and socioeconomic status (SES), Behavioral Regulation significantly predicted performance-based measures of EF and accounted for incrementally more variance in the models. We conclude that in ideal situations, it is best to measure EF using both rating scales and performance-based measures of EF. The CBRS seems to be a sensitive measure of EF in preschoolers and may be a helpful brief screening tool for use with teachers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In a study conducted by Tamm & Peugh (25) with 243 preschool children, the results indicated a small to moderate correlation between teacher ratings and EF test results, and more importantly, the strongest correlation was found between the Child Behavior Rating Scale’s (CBRS) Behavioral Regulation subscale and the EF test results.”
From Executive functioning in children with ADHD: Investigating the cross-method correlations between performance tests and rating scales (2024) · cites this paper as Tamm & Peugh (25)
Citations
Cites
Hierarchical Linear Models: Applications and Data Analysis Methods
Davidian, M. (2003) · Journal of the American Statistical Association · doi:10.1198/jasa.2003.s288 · Cited by 13,723
Centering predictor variables in cross-sectional multilevel models: A new look at an old issue.
Enders, C. K., & Tofighi, D. (2007) · Psychological Methods · doi:10.1037/1082-989x.12.2.121 · Cited by 4,462 · cited 1×
“Centering is an appropriate strategy to ignore multiple levels of nesting without biasing standard errors (Enders & Tofighi, 2007; Peugh, 2014; Raudenbush & Bryk, 2002; Snijders & Bosker, 2012).”
BackgroundExecutive function in preschoolers: A review using an integrative framework.
Garon, N., Bryson, S. E., & Smith, I. M. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.31 · Cited by 2,695 · cited 5×
“…performance-based paradigms to assess EF in preschoolers, in particular for measures of inhibition (e.g., delay of gratification, day/night stroop), working memory (e.g., digit span), and attention/ response shifting (e.g., shape school, dimensional change card sort, A-not-B) (Garon et al., 2008).”
BackgroundInfluentialA Developmental Perspective on Executive Function
Best, J. R., & Miller, P. H. (2010) · Child Development · doi:10.1111/j.1467-8624.2010.01499.x · Cited by 2,692 · cited 2×
“Furthermore, there are rapid developmental changes in EF during this period (Best & Miller, 2010).”
BackgroundA Compendium of Neuropsychological Tests: Administration, Norms, and Commentary
Selnes, O. A. (1991) · Neurology · doi:10.1212/wnl.41.11.1856-a · Cited by 2,252
Assessment and Development of Executive Function (EF) During Childhood
Anderson, P. (2002) · Child Neuropsychology · doi:10.1076/chin.8.2.71.8724 · Cited by 2,218 · cited 2×
“This pattern continues, with 5-year olds performing better than their younger counterparts across a range of EF tasks (Anderson, 2002).”
BackgroundIndividual Differences in Inhibitory Control and Children's Theory of Mind
Carlson, S. M., & Moses, L. J. (2001) · Child Development · doi:10.1111/1467-8624.00333 · Cited by 2,072 · cited 1×
“We controlled for age because there are rapid changes in neurocognitive development, particularly inhibition and delay of gratification, with age (Carlson & Moses, 2001; Diamond & Taylor, 1996; Kochanska, Murray, & Harlan, 2000) and because the measures do not have age-normed scores.”
MethodKochanska, G., Murray, K. T., & Harlan, E. T. (2000) · Developmental Psychology · doi:10.1037/0012-1649.36.2.220 · Cited by 1,993
Friedman, N. P., & Miyake, A. (2017) · Cortex · doi:10.1016/j.cortex.2016.04.023 · Cited by 1,834 · cited 1×
“Furthermore, dissociations within EF provide meaningful information for understanding impaired EF in clinical populations (Friedman & Miyake, 2017).”
BackgroundLinks between behavioral regulation and preschoolers’ literacy, vocabulary, and math skills.
McClelland, M. M., Cameron, C. E., Connor, C. M., Farris, C. L., Jewkes, A. M., & Morrison, F. J. (2007) · Developmental Psychology · doi:10.1037/0012-1649.43.4.947 · Cited by 1,561 · cited 6×
“Previous evidence indicates strong reliability and validity for the CBRS (Bronson, Tivnan, & Seppanen, 1995; Matthews, Ponitz, & Morrison, 2009; McClelland et al., 2007; McClelland & Morrison, 2003).”
BackgroundInfluentialToplak, M. E., West, R. F., & Stanovich, K. E. (2013) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12001 · Cited by 1,381
Diamond, A. (2002) · Principles of Frontal Lobe Function · doi:10.1093/acprof:oso/9780195134971.003.0029 · Cited by 1,248 · cited 3×
“one task conflicts with perception or prior task demands are essential milestones children achieve with development (Diamond, 2002).”
BackgroundPonitz, C. C., McClelland, M. M., Matthews, J. S., & Morrison, F. J. (2009) · Developmental Psychology · doi:10.1037/a0015365 · Cited by 934 · cited 2×
“The HTKS correlates significantly with teacher ratings of self-control and behavioral regulation (McClelland et al., 2007), parent ratings of attentional focusing and inhibitory control (Ponitz, McClelland, Matthews, & Morrison, 2009), and classroom functioning (Ponitz et al., 2009).”
BackgroundWhy IQ is not a covariate in cognitive studies of neurodevelopmental disorders
DENNIS, M., FRANCIS, D. J., CIRINO, P. T., SCHACHAR, R., BARNES, M. A., & FLETCHER, J. M. (2009) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617709090481 · Cited by 875 · cited 2×
“We did not control for IQ given that each of the performance-based measures represents an aspect of cognitive functioning related to IQ and because covarying for IQ produces overcorrected anomalous findings about neurocognitive function (Dennis et al., 2009).”
BackgroundMethodChildhood poverty: specific associations with neurocognitive development
Farah, M. J., Shera, D. M., Savage, J. H., Betancourt, L., Giannetta, J. M., Brodsky, N. L., Malmud, E. K., & Hurt, H. (2006) · Brain Research · doi:10.1016/j.brainres.2006.06.072 · Cited by 835 · cited 2×
“covariates because both are known to be related to EF performance (Farah et al., 2006; Matthews et al., 2009; Raaijmakers et al., 2008; Wiebe et al., 2011).”
BackgroundThe structure of executive function in 3-year-olds.
Wiebe, S. A., Sheffield, T., Nelson, J. M., Clark, C. A. C., Chevalier, N., & Espy, K. A. (2011) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2010.08.008 · Cited by 776 · cited 2×
“covariates because both are known to be related to EF performance (Farah et al., 2006; Matthews et al., 2009; Raaijmakers et al., 2008; Wiebe et al., 2011).”
BackgroundEarly gender differences in self-regulation and academic achievement.
Matthews, J. S., Ponitz, C. C., & Morrison, F. J. (2009) · Journal of Educational Psychology · doi:10.1037/a0014240 · Cited by 687 · cited 5×
“covariates because both are known to be related to EF performance (Farah et al., 2006; Matthews et al., 2009; Raaijmakers et al., 2008; Wiebe et al., 2011).”
BackgroundMethodInfluentialDeterminants of intercultural adjustment among expatriate spouses
Ali, A., Van der Zee, K., & Sanders, G. (2003) · International Journal of Intercultural Relations · doi:10.1016/s0147-1767(03)00054-3 · Cited by 508
Toplak, M. E., Bucciarelli, S. M., Jain, U., & Tannock, R. (2008) · Child Neuropsychology · doi:10.1080/09297040802070929 · Cited by 444
An interactive framework for examining prefrontal cognitive processes
Roberts, R. J., & Pennington, B. F. (1996) · Developmental Neuropsychology · doi:10.1080/87565649609540642 · Cited by 392 · cited 1×
“Working memory has also been identified as important for situations involving attentional control and monitoring (Roberts & Pennington, 1996; Rothbart & Posner, 2005).”
BackgroundMCAULEY, T., CHEN, S., GOOS, L., SCHACHAR, R., & CROSBIE, J. (2010) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617710000093 · Cited by 345
The shape school: Assessing executive function in preschool children
Espy, K. A. (1997) · Developmental Neuropsychology · doi:10.1080/87565649709540690 · Cited by 315 · cited 15×
“The Shape School allows for separate assessment of inhibition and switching and has been shown to be associated with later academic performance (math fluency, passage comprehension, understanding directions) in a prospective longitudinal design (Pritchard & Woodward, 2011).”
BackgroundResultInfluentialThe emergence of learning-related social skills in preschool children
McClelland, M. M., & Morrison, F. J. (2003) · Early Childhood Research Quarterly · doi:10.1016/s0885-2006(03)00026-7 · Cited by 308 · cited 1×
“Previous evidence indicates strong reliability and validity for the CBRS (Bronson, Tivnan, & Seppanen, 1995; Matthews, Ponitz, & Morrison, 2009; McClelland et al., 2007; McClelland & Morrison, 2003).”
BackgroundThorell, L. B., & Nyberg, L. (2008) · Developmental Neuropsychology · doi:10.1080/87565640802101516 · Cited by 299 · cited 5×
“Studies with other EF rating scales such as the Dysexecutive Questionnaire for Children (Emslie, Wilson, Burden, Nimmo-Smith, & Wilson, 2003) and Childhood Executive Function Inventory (CHEXI) (Thorell & Nyberg, 2008) report similarly low concordance with performance-based EF measures (Roy, Allain, Roulin, Fournet, &…”
BackgroundMethodResultInfluentialExecutive Function in Preschool Children: Examination Through Everyday Behavior
Isquith, P. K., Gioia, G. A., & Espy, K. A. (2004) · Developmental Neuropsychology · doi:10.1207/s15326942dn2601_3 · Cited by 287
A latent variable approach to determining the structure of executive function in preschool children.
Miller, M. R., Giesbrecht, G. F., Müller, U., McInerney, R. J., & Kerns, K. A. (2012) · Journal of Cognition and Development · doi:10.1080/15248372.2011.585478 · Cited by 272
Raaijmakers, M. A. J., Smidts, D. P., Sergeant, J. A., Maassen, G. H., Posthumus, J. A., van Engeland, H., & Matthys, W. (2008) · Journal of Abnormal Child Psychology · doi:10.1007/s10802-008-9235-7 · Cited by 268 · cited 2×
“covariates because both are known to be related to EF performance (Farah et al., 2006; Matthews et al., 2009; Raaijmakers et al., 2008; Wiebe et al., 2011).”
BackgroundAssessment of executive function in preschool‐aged children
Isquith, P. K., Crawford, J. S., Espy, K. A., & Gioia, G. A. (2005) · Mental Retardation and Developmental Disabilities Research Reviews · doi:10.1002/mrdd.20075 · Cited by 262 · cited 5×
“Cronbach’s alpha coefficients for each condition exceed 0.71 (Isquith et al., 2005).”
BackgroundInfluentialMeasuring behavioral regulation in four societies.
Wanless, S. B., McClelland, M. M., Acock, A. C., Ponitz, C. C., Son, S.-H., Lan, X., Morrison, F. J., Chen, J.-L., Chen, F.-M., Lee, K., et al. (2011) · Psychological Assessment · doi:10.1037/a0021768 · Cited by 174 · cited 14×
“Succeeding at the HTKS requires attention, working memory, and inhibitory control, but inhibitory control is the main component.”
BackgroundInfluentialvon Suchodoletz, A., Gestsdottir, S., Wanless, S. B., McClelland, M. M., Birgisdottir, F., Gunzenhauser, C., & Ragnarsdottir, H. (2013) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2012.05.003 · Cited by 161 · cited 1×
“Those studies have reported weak to moderate correlations between CBRS ratings and performance on the HTKS (McClelland et al., 2007; Von Suchodoletz et al., 2013; Wanless et al., 2011).”
BackgroundStructure of executive functions in typically developing kindergarteners
Monette, S., Bigras, M., & Lafrenière, M. K. (2015) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2015.07.005 · Cited by 154 · cited 3×
“In exploratory multivariate path analyses, we did however show that the CBRS Behavioral Regulation Scale was more strongly associated with measures of inhibition (Shape School, HTKS) and shifting/cognitive flexibility (Shape School, Trails-P) than BRIEF-P Index Scores or the BASC Executive Functioning Content score.”
BackgroundConstruct Validity of Parent Ratings of Inhibitory Control
Bodnar, L. E., Prahme, M. C., Cutting, L. E., Denckla, M. B., & Mahone, E. M. (2007) · Child Neuropsychology · doi:10.1080/09297040600899867 · Cited by 138
Genes and experience in the development of executive attention and effortful control
Rothbart, M. K., & Posner, M. I. (2005) · New Directions for Child and Adolescent Development · doi:10.1002/cd.142 · Cited by 122 · cited 1×
“Working memory has also been identified as important for situations involving attentional control and monitoring (Roberts & Pennington, 1996; Rothbart & Posner, 2005).”
BackgroundBronson, M. B., Tivnan, T., & Seppanen, P. (1995) · Journal of Applied Developmental Psychology · doi:10.1016/0193-3973(95)90035-7 · Cited by 121
Inhibitory processes in toddlers: a latent-variable approach
Gandolfi, E., Viterbori, P., Traverso, L., & Usai, M. C. (2014) · Frontiers in Psychology · doi:10.3389/fpsyg.2014.00381 · Cited by 120
Executive Function Among Preschool Children: Unitary Versus Distinct Abilities
Lerner, M. D., & Lonigan, C. J. (2014) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-014-9424-3 · Cited by 110 · cited 1×
“The two factors are most often working memory and inhibition (Karalunas, Bierman, & Huang-Pollock, 2016; Lerner & Lonigan, 2014; Miller, Giesbrecht, Muller, McInerney, & Kerns, 2012; Monette, Bigras, & Lafreniere, 2015), although one study found the two factors to be interference suppression and…”
BackgroundThe Behavior Rating Inventory of Executive Function: Commentary
DENCKLA, M. B. (2002) · Child Neuropsychology · doi:10.1076/chin.8.4.304.13512 · Cited by 103 · cited 4×
“Results from these studies (Denckla, 2002) and others (Bodnar, Prahme, Cutting, Denckla, & Mahone, 2007; McAuley, Chen, Goos, Schachar, & Crosbie, 2010; Toplak, Bucciarelli, Jain, & Tannock, 2009) report mixed findings, but generally note low concordance between BRIEF subscale ratings and…”
MethodResultInfluentialWilloughby, M. T., Wirth, R. J., & Blair, C. B. (2011) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2010.04.007 · Cited by 102
Miranda, A., Colomer, C., Mercader, J., Fernández, M. I., & Presentación, M. J. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00545 · Cited by 80 · cited 1×
“We did not replicate the findings of moderate associations between EF ratings and performance-based measures of EF in preschoolers (Miranda et al., 2015; Thorell & Nyberg, 2008).”
ResultAmador, J. A., Forns-Santacana, M., Guàrdia‐Olmos, J., & Peró, M. (2005) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-006-4538-x · Cited by 80
Behavioural Assessment of Dysexecutive Syndrome for Children (BADS-C)
(2013) · British Psychological Society eBooks · doi:10.53841/bpstest.2013.badsc · Cited by 80
Ezpeleta, L., & Granero, R. (2014) · Journal of Neuropsychology · doi:10.1111/jnp.12049 · Cited by 59
Does the BRIEF-P Predict Specific Executive Function Components in Preschoolers?
Garon, N. M., Piccinin, C., & Smith, I. M. (2016) · Applied Neuropsychology: Child · doi:10.1080/21622965.2014.1002923 · Cited by 57
Webster-Stratton, C. H. (1985) · Research on Child and Adolescent Psychopathology · doi:10.1007/bf00910640 · Cited by 56 · cited 1×
“Likewise, preschool EF behaviors may be easier to judge, especially in the more structured classroom context (Webster-Stratton, 1985).”
BackgroundThorell, L. B., Eninger, L., Brocki, K. C., & Bohlin, G. (2010) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803390902806527 · Cited by 52
The Behavior Rating Inventory of Executive Function: Introduction
Donders, J. (2002) · Child Neuropsychology · doi:10.1076/chin.8.4.229.13508 · Cited by 46 · cited 2×
“devoted to investigating the clinical utility of the BRIEF (Donders, 2002), which is the most commonly utilized rating scale of EF.”
BackgroundMethodADHD and the Nature of Self-Control
Halperin, J. M. (1999) · Psychiatric Services · doi:10.1176/ps.50.2.275 · Cited by 38 · cited 1×
“EF is a means of selfregulation that modulates behavior in an adaptive fashion to the changing context at hand (Barkley, 2005).”
BackgroundMahoney, G. J., & Wheeden, C. A. (1999) · PsycTESTS Dataset · doi:10.1037/t57739-000 · Cited by 35
Karalunas, S. L., Bierman, K. L., & Huang‐Pollock, C. L. (2016) · Journal of Attention Disorders · doi:10.1177/1087054715627488 · Cited by 32 · cited 1×
“It should be noted that the HTKS is also a strong indicator of working memory because children must hold in mind multiple complex rules (Karalunas et al., 2016).”
BackgroundPreschool executive control on the Shape School task: Measurement considerations and utility.
Pritchard, V. E., & Woodward, L. J. (2011) · Psychological Assessment · doi:10.1037/a0021095 · Cited by 29 · cited 3×
“In fact, development of these skills in the preschool period is considered foundational for more complex executive skills such as planning and conceptual reasoning that rely on the coordinated integration of several EF component processes (Pritchard & Woodward, 2011).”
BackgroundJarman, R. F., Vavrik, J., & Walton, P. D. (1995) · PubMed · Cited by 24
Roy, A., Allain, P., Roulin, J.-L., Fournet, N., & Le Gall, D. (2015) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2015.1072138 · Cited by 23
Conducting Three-Level Cross-Sectional Analyses
Peugh, J. (2013) · The Journal of Early Adolescence · doi:10.1177/0272431613498646 · Cited by 17 · cited 1×
“Centering is an appropriate strategy to ignore multiple levels of nesting without biasing standard errors (Enders & Tofighi, 2007; Peugh, 2014; Raudenbush & Bryk, 2002; Snijders & Bosker, 2012).”
BackgroundSelf-citationDiamond, A., & Taylor, C. (1996) · Developmental Psychobiology · doi:10.1002/(sici)1098-2302(199605)29:4<315::aid-dev2>3.0.co;2-t · cited 1×
“We controlled for age because there are rapid changes in neurocognitive development, particularly inhibition and delay of gratification, with age (Carlson & Moses, 2001; Diamond & Taylor, 1996; Kochanska, Murray, & Harlan, 2000) and because the measures do not have age-normed scores.”
Method
Research works printed but not linked: 7
- Gioia G. A., 2003, Behavior Rating Inventory of Executive Function-Preschool version professional manual
- Gioia G. A., 2000, Behavior rating inventory of executive function professional manual
- Jarman R. F., 1995, Genetic, Social, and General Psychology Monographs, 121, 153
- Muthén L. K., 1998, Mplus user’s guide
- Reynolds C. R., 2004, Behavior assessment system for children, second edition manual
- Spreen O., 1998, A Compendium of neuropsychological tests
- Zelazo P. D., 2003, Child Development, 68, 137
Cited by
Finders, J. K., McClelland, M. M., Geldhof, G. J., Rothwell, D. W., & Hatfield, B. E. (2020) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2020.07.008 · Cited by 120
Laatsch, L., Dodd, J. N., Brown, T. M., Ciccia, A., Connor, F., Davis, K. C., Doherty, M., Linden, M., Locascio, G., Lundine, J. P., et al. (2019) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2019.1678490 · Cited by 100
“…measures and informant rating scales of executive functions measure distinctly different constructs (Olson, Jacobson, & Van Oot, 2013; Tamm & Peugh, 2019 Apr; Tan, Delgaty, Steward, & Bunner, 2018 Dec; Vriezen & Pigott, 2002) Therefore, results from studies included in this EBR may…”
BackgroundHoward, S. J., Vasseleu, E., Neilsen‐Hewett, C., de Rosnay, M., Chan, A., Johnstone, S. J., Mavilidi, M. F., PAAS, F. G. W. C., & Melhuish, E. C. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.733328 · Cited by 74
“While previous research has established cross-sectional associations between EF and SR (Evers et al., 2016; Tamm and Peugh, 2019), the longitudinal design utilized here established ongoing bi-directional associations between EF and SR across the transition to school period.”
ResultMapping neurodevelopmental diversity in executive function
Mareva, S., Lead Investigators, Astle, D. E., Baker, K., Gathercole, S. E., Holmes, J., Kievit, R., Manly, T., Team of Researchers and PhD Students, Akarca, D., et al. (2024) · Cortex · doi:10.1016/j.cortex.2023.11.021 · Cited by 45
Advancing Educational Research on Children’s Self-Regulation With Observational Measures
Eberhart, J., Koepp, A. E., Howard, S. J., Kok, R., McCoy, D. C., & Baker, S. T. (2022) · Journal of Psychoeducational Assessment · doi:10.1177/07342829221143208 · Cited by 35
Metadata
- Authors
- Tamm, L., & Peugh, J.
- Affiliations
- Cincinnati Children's Hospital Medical Center (US)
- Corresponding author
- Tamm, Leanne
- Venue
- Child Neuropsychology
- Publisher
- Informa UK Limited
- Published
- 2018-06-14
- Volume
- 25
- Issue
- 3
- Pages
- 410-424
- ISSN
- 0929-7049 (ISSN-L)
- 1744-4136
- OpenAlex
- W2808340119
- Cited by
- 35
- References
- 54
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 8 · 2023: 2 · 2022: 6 · 2021: 4 · 2020: 5 · 2019: 3
- FWCI
- 5.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Child Neuropsychology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Brady Education Foundation, Inc
- Topics
- Social Sciences › Psychology › Neuropsychology and Physiological Psychology › Developmental and Educational Neuropsychology
- Social Sciences › Education › Early Childhood Education and Development
- Social Sciences › Psychology › Developmental and Educational Psychology › Child and Animal Learning Development
- Keywords
- executive function, preschool children, teacher ratings, performance-based measures, rating scale validity, behavioral regulation, inhibition, cognitive flexibility, screening tool
- MeSH terms
- School Teachers, Child Behavior, Child, Preschool, Female, Humans, Male, Neuropsychological Tests, Executive Function
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.