CoChat
Sign Up
Cited by 0Cognitive Abilities and TestingPsychometric Methodologies and TestingSocial and Intergroup Psychology

Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review

Open access
Cuppello, Stephen, Zibarras, Lara D., Corr, Philip J.·2025·Trends in Psychology
DOI 10.1007/s43076-025-00504-5
Open this paper in CoChat — free accountOpen PDF

Abstract

Commonly used in employee selection, cognitive ability tests (CATs) show mean score group differences by gender, ethnicity and socioeconomic status (SES). Prior research supports both substantive accounts, such as biological and environmental factors, and methodological and contextual accounts, such as stereotype threat and differential item functioning, related to group differences. This systematic review explores the evidence for the theoretical explanations that purport to explain these differences. Two hundred and twenty-five papers met inclusion criteria which explored potential explanations of group differences. These were grouped into 10 factors: biological, environmental, latent trait / measurement invariance, criterion validity, item bias, behaviour in test, anxiety, attitudes, experience, and stereotype threat. Results showed that whilst biological and environmental factors are related to group differences, so are several factors relevant to test design and administration, most notably stereotype threat, differences in domain experience and exposure and self-confidence. Theoretical and practical implications for the occupational use of CATs as well as limitations and recommendations are discussed.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review of 225 papers identifies biological and environmental factors, plus test-design and administration factors—especially stereotype threat, domain experience/exposure, and self-confidence—as related to mean score group differences on cognitive ability tests (CATs). It concludes that methodological and contextual factors offer promising avenues for fairer occupational CAT use, but the evidence is limited by few high-stakes occupational studies and heavy reliance on student and US samples.

Key findings

  • Across 225 papers grouped into 10 factors, biological (63 papers; 67% impact “yes”) and environmental (15; 93% “yes”) factors were supported, but so were test-design/administration factors: stereotype threat (75; 81% “yes”), experience (18; 72% “yes”), and attitudes/self-confidence (23; 52% “yes”).
  • Stereotype threat had the largest evidence base (75 papers; 51 gender, 27 ethnicity, 7 SES), with supported mechanisms including working-memory/executive-resource depletion and interpretation of anxious arousal; interventions included stating no group differences, role models, self-affirmation, and mindfulness. However, few studies replicated effects in high-stakes occupational settings.
  • Experience/exposure findings showed training, practice tests/items, and video games improved spatial performance more for women and sometimes eliminated the gender gap; practice/training improved Black performance more than White performance. Self-confidence mediated gender–spatial performance, and manipulating confidence increased performance.
  • Item bias/DIF exists but generally had negligible impact on overall group mean differences; criterion validity findings were inconsistent; latent trait/measurement invariance findings were discordant. Only 14% of studies met ecological validity criteria, 71% used student samples, 56% used US samples, and average MMAT score was 43%.

Methods

  • PRISMA-P systematic review, not pre-registered. Searched PsycINFO, PsychArticles, Web of Science, and Business Source Ultimate with broad terms; backward and limited forward citation searching (first 20 papers and first 10 from the last five years per paper).
  • Inclusion: adult humans (mean age 18+), original research, English peer-reviewed; excluded descriptive-only papers, bias-measurement-only papers, clinical/cognitive-decline samples, and tests of knowledge/crystallised intelligence or standardised academic tests such as SATs. Initial 1,124 records; 316 duplicates; 541 abstract-excluded; 140 excluded; 127 fully reviewed; plus 98 from citations = 225 papers.
  • Data extraction and synthesis by the first author, reviewed by coauthors; papers tagged into 10 factors; MMAT used for risk of bias; narrative synthesis; extracted “Impact” (yes/no/maybe) and “Ecological Validity” (yes/no/partially).

Limitations

  • Authors state: excluded child samples and high-stakes standardised education testing; broad scope and heterogeneity sacrificed specificity; gender/sex and ethnicity/race terms plus global inclusion may overlook nuances; screening, extraction, and synthesis were completed by one person; forward citation searching was not expansive.
  • Text-evident: only 14% of studies met ecological validity criteria; 71% used student samples; 56% used US samples; only 8% included any participants outside North America/Europe and 4% exclusively outside those regions; average MMAT was 43%, with common risks (confounders accounted for only twice in nonrandomized studies; 18% representative; RCT blinding 10%, baseline comparability 11%).
  • Age differences could not be reviewed: only two papers measured age and neither found reliable causal evidence; only 44 of 225 papers addressed multiple factors, and these rarely provided consistent interaction evidence.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s43076-025-00504-5

Cites

265 of 265 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

23 of 186 cited sources checked · 17 supported · 1 flagged · accuracy not yet measured

  • Findingcitation

    Several papers detail that oral contraceptive use and type influences CAT performance on verbal and spatial tasks.

    “several papers detail oral contraceptive use and type influences CAT performance on verbal and spatial task (Griksiene & Ruksenas, 2011;Hampson et al., 2022)”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Griksiene & Ruksenas, 2011 source
    • cites Hampson et al., 2022 source

      Study 1 Results report differences across contraceptive-cycle phases on spatial and verbal-fluency tasks, and the paper examines how contraceptive formulations relate to spatial performance. It does not establish that contraceptive type influences verbal performance; the General Discussion also cautions that its observational design cannot conclusively establish causation. The reported tests are not identified as CAT. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Findingcitation

    Research has used MGCFA elsewhere to suggest CATs do not necessarily have measurement invariance between cultures.

    “Research has used MGCFA elsewhere to suggest CATs do not necessarily have measurement invariance between cultures (Cockcroft et al., 2015).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Cockcroft et al., 2015 source
  • Findingcitation

    Androgens have been linked to spatial performance.

    “and androgens to spatial performance (Shute et al., 1983).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Shute et al., 1983 source
  • Findingcitation

    Females with male twins outperform those with female twins on spatial tasks.

    “Research has found that females with male twins outperform those with female twins on spatial tasks”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Heil et al., 2011

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Vuoksimaa et al., 2010 source
  • Interpretationcitation

    Pubescent testosterone levels may partially explain group differences in spatial performance.

    “which may partially explain group differences (Vuoksimaa et al., 2012).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Vuoksimaa et al., 2012 source

      The Introduction says testosterone has been suggested to play a role in sex differences in spatial ability. The Discussion reports that pubertal testosterone predicted later mental-rotation performance only among men; it does not show that pubertal testosterone explains differences between groups. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Classified as probably not a substantial claim. 98
  • May combine two claims. 73
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Other authors have found that CAT performance was less predictive of training performance for Black recruits.

    “other authors have found that CAT performance was less predictive of training performance for Black recruits (Carretta & Doub, 1998;De Meijer et al., 2008)”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Carretta & Doub, 1998 source
    • cites De Meijer et al., 2008 source

      The Main results (§3.2; page not available) state the opposite direction: the cognitive ability test, especially its verbal subtests, was more predictive of training performance for ethnic minority trainees than for ethnic majority trainees. The Discussion (§4; page not available) reports corrected validities of .14 and .04, respectively. The paper describes Dutch ethnic groups, not a separate finding for Black recruits. (LLM, conf=0.88)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Interpretationcitation

    Mean score group differences on CATs have the potential to further entrench workplace inequalities.

    “These differences have the potential to further entrench workplace inequalities (Woods & Patterson, 2024).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Woods & Patterson, 2024 source

  • May combine two claims.

    Premisecitation

    CATs show mean score group differences by gender, ethnicity and socioeconomic status (SES).

    “Commonly used in employee selection, cognitive ability tests (CATs) show mean score group differences by gender, ethnicity and socioeconomic status (SES).”
    • Also stated in: § Introduction · boosted
      “though widely reported mean score group differ-ences threaten to undermine their practical utility (e.g., Anastasi, 1958;Bates et al., 2013;Neisser et al., 1996;Roth et al., 2001).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Anastasi, 1958

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Bates et al., 2013 source

      The available passages are bibliographic records (no section or page) and do not discuss cognitive ability test scores across gender, ethnicity, or SES groups. Searches for those group differences and for employee-selection terminology found no substantive passage addressing the claim. (LLM, conf=0.72)

      Experimental — accuracy not yet measured
    • cites Neisser et al., 1996 source
    • The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.

  • May combine two claims.

    Premisecitation

    Adults from lower SES backgrounds typically score modestly lower on intelligence tests, with childhood SES explaining around 5% of adult intelligence.

    “Adults from lower SES backgrounds typically score modestly lower, with childhood SES found to explain around 5% of adult intelligence (Bates et al., 2013).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Bates et al., 2013 source

      The indexed passages returned for this paper are bibliographic metadata (no section or page), not findings about intelligence scores. Searches for the lower-SES comparison and the claimed 5% figure found no passage in our copy that addresses either point. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Risk of bias was assessed using the Mixed Methods Appraisal Tool (MMAT).

    “Risk of bias was conducted using the Mixed Methods Appraisal Tool (MMAT; Hong et al., 2018), as it provides a single framework for which to appraise the studies included in this review employing varying methodologies.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Hong et al., 2018 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Mixed Methods Appraisal Tool has been previously employed to assess risk of bias in systematic reviews.

    “and has been previously employed to assess risk of bias in systematic reviews (e.g., Gledhill et al., 2018).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Gledhill et al., 2018 source

  • May combine two claims.

    Findingcitation

    Multiple papers found women's performance fluctuated throughout the menstrual cycle with better spatial performance in the menstrual phase and better verbal performance in the luteal phase.

    “Multiple papers found women's performance fluctuated throughout the menstrual cycle with better spatial performance in the menstrual phase and better verbal performance in the luteal phase (e.g., Hampson, 1990aHampson, , 1990b;;McCormick & Teillon, 2001;Šimić & Santini, 2012)”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Hampson, 1990a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites McCormick & Teillon, 2001

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Šimić & Santini, 2012 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Previous research has suggested that some environmental factors may be genetically influenced (e.g., SES).

    “Previous research has suggested that some of these factors may be genetically influenced (e.g., SES; Krapohl & Plomin, 2016)”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Krapohl & Plomin, 2016 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Other studies have found no links between testosterone and spatial, verbal or other gendered domains.

    “although other studies have found no links with spatial, verbal or other gendered domains (e.g., Hirnstein et al., 2014;McKeever et al., 1987;Puts et al., 2010).”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites Hirnstein et al., 2014 source
    • cites McKeever et al., 1987 source
    • cites Puts et al., 2010 source

  • May combine two claims.

    Findingcitation

    Forbes and Schmader (2010) found that training a more positive attitude towards the testing domain increased motivation but not performance.

    “Finally, Forbes and Schmader (2010) found that training a more positive attitude towards the testing domain increased motivation but not performance.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Forbes and Schmader (2010) source

  • May combine two claims.

    Findingown data

    Results showed that whilst biological and environmental factors are related to group differences, so are several factors relevant to test design and administration, most notably stereotype threat, differences in domain experience and exposure, and self-confidence.

    “Results showed that whilst biological and environmental factors are related to group differences, so are several factors relevant to test design and administration, most notably stereotype threat, differences in domain experience and exposure and self-confidence.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.

  • May combine two claims.

    Findingown data

    The proportion of student samples varied significantly by factor, with the most common use in Anxiety (94%) and Attitudes (91%) and the least common use in Criterion Validity (0%) and Environmental Differences (50%).

    “This proportion varied significantly by factor, with the most common use in Anxiety (94%) and Attitudes (91%) and the least common use in Criterion Validity (0%) and Environmental Differences (50%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Attitudes had the highest average MMAT score (48%) and Latent Trait/Measurement invariance had the lowest average (38%).

    “Attitudes had the highest average (48%) and Latent Trait/Measurement invariance had the lowest average (38%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Of the 225 papers, 55% employed exclusively US samples, 8% Canadian, 6% German, 4% British, 20% from other single countries, and 7% used samples from multiple countries.

    “Of the 225 papers, 55% employed exclusively US samples, 8% Canadian, 6% German, 4% British, 20% from other single counties and a further 7% used samples from multiple countries.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Of the 225 papers, 147 were classified as quantitative nonrandomized, 79 as quantitative randomized controlled trials, and one as quantitative descriptive.

    “In terms of methodological quality, 147 studies were classified as quantitative nonrandomized, 79 as quantitative randomized controlled trials and one as quantitative descriptive.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The overall average MMAT score was 43% and did not vary much.

    “The overall average MMAT score was 43% and this did not vary much”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    Papers were grouped into ten factors: biological differences, environmental differences, latent trait differences/measurement invariance, criterion validity differences, item bias, test taking behavioural differences, test anxiety, test attitudes and perceptions, test experience, and stereotype threat.

    “This resulted in ten factors, namely: biological differences, environmental differences, latent trait differences/measurement invariance, criterion validity differences, item bias, test taking behavioural differences, test anxiety, test attitudes and perceptions, test experience and stereotype threat.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Ecological validity varied by factor: Criterion Validity met the criteria most frequently (94%), followed by Latent Trait/Measurement invariance (23%); Biological (0%) and Stereotype Threat (4%) met them least.

    “This again varied by factor with Criterion Validity meeting these criteria most frequently (94%) followed by Latent Trait/ Measurement invariance (23%).The two factors meeting these criteria the least were Biological (0%) and Stereotype Threat (4%).”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    In quantitative randomized controlled trials, only 10% met MMAT condition 2.4 (outcome assessors blinded) and only 11% met condition 2.2 (groups comparable at baseline), as 80% did not provide this information.

    “In quantitative randomized controlled trials, only 10% met condition 2.4 (”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    A similar approach concluded that there was measurement invariance with regard to women and men, suggesting group differences were at the latent trait level.

    “A similar approach to these studies concluded that there was measurement invariance with regard to women and men, suggesting group differences were at the latent trait level (Dolan et al., 2006).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dolan et al., 2006

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    One study found that Spearman's hypothesis was erroneously supported when comparing CAT items with personality test items.

    “with one study finding that the hypothesis was erroneously supported when comparing CAT items with personality test items (Wicherts, 2017).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wicherts, 2017 source

  • Classified as probably not a substantial claim.

    Findingnone given

    Thirteen papers related to latent trait differences or measurement invariance.

    “Thirteen papers related to latent trait differences or measurement invariance”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    A further study used MGCFA on a data set previously used to support Spearman's hypothesis and was unable to find factorial invariance, undermining original conclusions.

    “A further study used multi-group confirmatory factor analysis (MGCFA) on a data set previously used to support the hypothesis (cited above: Te Nijenhuis & Van der Flier, 1997) and was unable to find factorial invariance, undermining original conclusions (Dolan et al., 2004).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dolan et al., 2004

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingnone given

    Overall, 8 papers related to gender, 7 to ethnicity and 3 to SES in the environmental factor.

    “Overall, 8 papers related to gender, 7 to ethnicity and 3 to SES”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Similar monthly effects have been found in men, although not as pronounced.

    “Although not as pronounced, similar monthly effects have been found in men (Courvoisier et al., 2013).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Courvoisier et al., 2013

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Authors have found support that group mean score differences are related to differences in processing speeds in more elementary cognitive tasks and so not test bias.

    “authors have variously found support that group mean score differences are related to differences in processing speeds in more elementary cognitive tasks and so not test bias (Vernon & Jensen, 1984)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Vernon & Jensen, 1984 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Measurement invariance has also been supported by finding high correlations between item difficulties between different cultures, suggesting group differences are in g rather than culturally determined.

    “Measurement invariance has also been supported by finding high correlations between item difficulties between different cultures, suggesting group differences are in g rather than culturally determined (Rushton et al., 2007b).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rushton et al., 2007b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Spearman's hypothesis has been criticised as being inappropriate.

    “However, this theory has been criticised as being inappropriate”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Neuropsychological predictors of CAT performance differed between cultures, which suggests different constructs were being measured.

    “that neuropsychological predictors of CAT performance differed between cultures which suggests different constructs were being measured (Fasfous et al., 2013)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Fasfous et al., 2013 source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    2D:4D ratios correlate negatively to prenatal testosterone and positively to prenatal oestrogen.

    “A further hormonal development account is inferred by 2D:4D ratios, which has been found to correlate negatively to prenatal testosterone and positively to prenatal oestrogen.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Other research suggests there is potential to overcome prior knowledge biases with practice items.

    “although other research suggests there is potential to overcome this with practice items (Tanzer et al., 1995).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tanzer et al., 1995 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Many of the 14 papers in the behaviour in test factor examined differences in test strategy.

    “Many of the 14 papers in this factor examined differences in test strategy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The link between pupil dilation and cognitive effort is perhaps less clear.

    “although this link is perhaps less clear.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Of 16 papers relating to anxiety, most were based on gender.

    “Of 16 papers relating to anxiety, most were based on gender.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Berry et al. (2014) concluded that underpredictions of minority performance was unlikely.

    “however, they concluded that underpredictions of minority performance was unlikely.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Berry et al. (2014)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingcitation

    Sokolowski et al. (2019) found women display more maths anxiety which is related to performance.

    “and Sokolowski et al. (2019) found women display more maths anxiety which is related to performance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sokolowski et al. (2019) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Other authors have found that CAT performance was less predictive of job performance.

    “and less predictive of job performance (Gardner & Deadrick, 2008, 2012).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Gardner & Deadrick, 2008 source
    • cites Gardner & Deadrick, 2012 source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Papers cited in criterion validity mostly assume no bias in the criterion, which is often subjective manager ratings.

    “Papers cited here mostly assume no bias in the criterion, which is often subjective manager ratings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Harrison et al. (2006) found no differences in effort between SES or ethnic groups.

    “Harrison et al. (2006) found no differences in effort between SES or ethnic groups.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Harrison et al. (2006) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    All but one simulation study using data from high-stakes testing.

    “with all but one simulation study using data from high-stakes testing.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Findingcitation

    Some suggest that range restriction for minority applicants explains any differences found.

    “with some suggesting that range restriction for minority applicants explains any differences found (Roth et al., 2014)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Roth et al., 2014

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Despite women typically scoring higher in trait anxiety, they perform similarly to men on overall CATs.

    “Despite women typically scoring higher in trait anxiety, they perform similarly to men on overall CATs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Trait and state anxiety are related to performance.

    “Trait and state anxiety are related to performance (Kumari & Corr, 1998)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Kumari & Corr, 1998 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    Most anxiety research looks at gender, but ethnicity and SES have also been studied.

    “Whilst most of this research looks at gender, ethnicity and SES have also been studied.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Generally, studies either did not identify biased items, or biased items only negligibly impacted overall test scores.

    “Generally, studies either did not identify biased items, or biased items only negligibly impacted overall test scores (e.g., Abad et al., 2004;Jensen & McGurk, 1987;Vanderpool & Catano, 2008).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Abad et al., 2004

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Jensen & McGurk, 1987 source
    • cites Vanderpool & Catano, 2008 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Campbell et al. (2018) found women showed greater pupil dilation in a spatial task.

    “Campbell et al. (2018)found women showed greater pupil dilation in a spatial task”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Campbell et al. (2018) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Some papers' findings were more ambiguous.

    “some papers' findings were more ambiguous (Harville, 1996)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Harville, 1996 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Of the papers that met inclusion criteria, 16 referred to criterion validity or predictive validity.

    “Of the papers that met inclusion criteria, 16 referred to criterion validity or predictive validity”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Berry et al. (2014) suggested that CATs are seldom used in isolation in recruitment decisions.

    “Berry et al. (2014) suggested that CATs are seldom used in isolation in recruitment decision”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Berry et al. (2014)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Twenty-three papers examined potential item bias or DIF.

    “Twenty-three papers examined potential item bias or DIF.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Weighting subtests differently can potentially reduce adverse impact without sacrificing criterion validity.

    “Other notable findings were that weighting subtests differently can potentially reduce adverse impact without sacrificing criterion validity (Wee et al., 2014).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wee et al., 2014

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    In all, 15 studies looked at ethnicity and 5 at gender in criterion validity.

    “In all, 15 studies looked at ethnicity and 5 at gender.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Black/White test score differences were not replicated on learning tasks, potentially suggesting cultural bias.

    “and that Black/White test score differences were not replicated on learning tasks potentially suggesting cultural bias (Grubb & Ollendick, 1986).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Grubb & Ollendick, 1986 source

  • May combine two claims.

    Findingnone given

    Studies related to test behaviour have been conducted with gender (9), ethnicity (5) and SES (1).

    “Studies related to test behaviour have been conducted with gender (9), ethnicity (5) and SES (1).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Gender differences on general CATs are not typically found.

    “despite gender differences on general CATs not typically being found”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Item bias research is roughly evenly split between gender (13 papers) and ethnicity (10).

    “Item bias research is roughly evenly split between gender (13 papers) and ethnicity (10).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Some have found modest support for underprediction of Black training performance.

    “Some have found modest support for underprediction of Black training performance (Houston, 1987;Te Nijenhuis & Van der Flier, 2000).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Houston, 1987 source
    • cites Te Nijenhuis & Van der Flier, 2000 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Other research has found modest gender differences on similar tests.

    “although other research has found modest gender differences on similar tests (Arendasy & Sommer, 2012).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Arendasy & Sommer, 2012 source

  • May combine two claims.

    Findingcitation

    Some papers found no predictive bias but found tests were less valid for Black applicants.

    “found no predictive bias but found tests were less valid for Black applicants (Rotundo & Sackett, 1999)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Rotundo & Sackett, 1999 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The authors of Campbell et al. (2018) suggested pupil dilation was a proxy for cognitive effort.

    “which the authors suggested was a proxy for cognitive effort”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Campbell et al. (2018) source

  • May combine two claims.

    Findingnone given

    In all, 12 papers examined ethnicity and 2 examined gender in latent trait/measurement invariance.

    “In all, 12 papers examined ethnicity and 2 examined gender.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Under stereotype threat, people are more likely to persevere with an ineffective problem-solving strategy.

    “a paper investigating stereotype threat found that under threat, people are more likely to persevere with an ineffective problem-solving strategy (Carr & Steele, 2009)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Carr & Steele, 2009 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    There has been some suggestion of group differences in effort.

    “Beyond strategy use, there has been some suggestion of group differences in effort.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    There is support that differences in motivation and face validity perceptions do not mediate the relationship between ethnicity and performance.

    “and do not (Chan, 1997) mediate relationship between ethnicity and performance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chan, 1997 source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Women typically display higher trait anxiety and report greater test anxiety.

    “Anxiety differences are likely better explained in terms of stereotype threat; however, it is notable that women typically display higher trait anxiety and report greater test anxiety.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Multiple papers found in the search looked specifically at cognitive decline in retirement ages, but did not meet inclusion and exclusion criteria as they were not based on working age participants.

    “There were multiple papers found in the search process that looked specifically at cognitive decline in retirement ages, but these did not meet inclusion and exclusion criteria as they were not based on working age participants.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    Genetic evidence for group differences is currently lacking though promising.

    “Finally, genetic evidence is currently lacking though promising, and so the area warrants further exploration.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    There is a paucity of research exploring the interaction of explanatory factors.

    “Additional, there is a paucity of research exploring the interaction of these explanatory factors.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Studies have shown that Black people have lower motivation and face validity perceptions in testing.

    “Studies have shown that Black people have lower motivation and face validity perceptions in testing (Arvey et al., 1990)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Arvey et al., 1990 source

  • May combine two claims.

    Findingnone given

    Parental and individual education, income, immigration status, and language proficiency have been found to relate to ethnic and SES differences.

    “Parental and individual education, income, immigration status, and language proficiency have been found to relate to ethnic and SES differences, whilst gendered play and contraceptive use and type has been found to be related to gender differences.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    There is some support that exploring differences in state anxiety may be insightful.

    “There is some support that exploring differences in state anxiety may be insightful, and whilst only impacting group differences negligibly, exploring DIF in CATs is still something that should be recommended.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Much of the evidence has not been conducted in high-stakes occupational testing and is overly reliant on US student samples.

    “However, much of the evidence has not been conducted in high-stakes occupational testing and is overly reliant on US student samples.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Self-confidence was found to be an important mediator between gender and spatial performance and is the strongest attitudinal account that may relate to group differences on CATs.

    “Self-confidence was found to be an important mediator between gender and spatial performance and is the strongest attitudinal account that may related to group differences on CATs.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Many CATs have been found to show DIF.

    “Many CATs have been found to show DIF, and test publishers should be mindful of this and endeavour to remove biased items from their tests.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Stereotype threat has a large body of evidence that it impacts performance for women and men, lower SES people and ethnic minorities across many cognitive domains.

    “Finally, stereotype threat has a large body of evidence to support that it impacts performance for women and men, lower SES people and ethnic minorities across many cognitive domains.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Only 8% of studies involved at least some participants outside North America and Europe, and only 4% used participants exclusively outside those regions.

    “No notable trends were observed when comparing US studies with non-US studies, however only 8% of studies involved at least some participants outside of North America and Europe with only 4% using participants exclusively outside of those regions.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    The role of sex hormones and monthly hormonal fluctuations in gender differences in stereotypically gendered cognitive domains is clear.

    “Considering biological factors, the role of sex hormones and monthly hormonal fluctuations in gender differences in stereotypically gendered cognitive domains is clear.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Studies relating to attitudes were roughly split between gender (11) and ethnicity (12).

    “Studies relating to attitudes were roughly split between gender (11) and ethnicity (12).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    56% of studies were conducted in the US, which may limit cross-cultural generalisability.

    “A further critique of this literature comes from the fact that 56% of studies were conducted in the US, which may limit cross-cultural generalisability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    There is little to explain potential overprediction and little to address potential bias in criteria used.

    “However, there is little to explain potential overprediction, and little to address potential bias in criteria used.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Some criterion validation studies provide a sound account that CATs do not underpredict performance of women or minorities and may overpredict it.

    “Some of the criterion validation studies reported provide a sound account that CATs do not underpredict performance of women or minorities and may overpredict it.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    In all, 23 papers examined attitudinal causes for group differences.

    “In all, 23 papers examined attitudinal causes for group differences”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Latent trait and measurement invariance findings are inconsistent, with sizable disagreement between studies.

    “Latent trait and measurement invariance findings are inconsistent and there is sizable disagreement between studies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Maertz et al. (2005) found self-efficacy differences between genders but not ethnicities.

    “Maertz et al. (2005) found self-efficacy differences between genders but not ethnicities.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Maertz et al. (2005) source

  • May combine two claims.

    Findingnone given

    Gendered play and contraceptive use and type have been found to relate to gender differences.

    “Parental and individual education, income, immigration status, and language proficiency have been found to relate to ethnic and SES differences, whilst gendered play and contraceptive use and type has been found to be related to gender differences.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Stenlund et al. (2017) did not find gender differences in motivation.

    “Stenlund et al. (2017) did not find gender differences in motivation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Stenlund et al. (2017)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Many of the factors explored have a body of compelling evidence supporting their contribution to group differences, including both substantive accounts and methodological/contextual accounts.

    “Many of the factors explored have a body of compelling evidence to support their contribution, both those related to substantive accounts and methodological/contextual accounts of group differences.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    There have been challenges in replicating stereotype threat studies in high stakes testing.

    “However, there have been challenges in replicating studies in high stakes testing, which may suggest the nature of the experimental approach could be contributing to the magnitude of effects seen.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Hough et al. (2001) found that culture-free, nonverbal ability tests still showed modest group differences, and attempts to address the confound of ethnicity versus culture by building tests with Black cultural components found little success in overcoming differences.

    “They also found evidence suggesting culture-free, nonverbal ability tests still showed modest group differences and that attempts to address the confound of ethnicity vs culture by building tests with Black cultural components found little success in overcoming differences.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Hough et al., 2001 source

  • May combine two claims.

    Premisecitation

    Overall, ethnic differences in mean intelligence scores are the largest, SES differences are modest, and gender differences are not typically seen in g but are found on specific cognitive domains.

    “Overall, ethnic differences in mean intelligence scores are the largest, SES are modest, and gender differences are not typically seen in g but are found on specific cognitive domains.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Paper selection, data extraction and data synthesis were completed by the first author and reviewed by the second and third authors.

    “Paper selection, data extraction and data synthesis were completed by the first author.These processes were reviewed by the second and third authors.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Neisser et al. (1996) reported that most CATs are constructed so there are no overall gender differences, but differences exist within some cognitive domains: men typically perform better in spatial and numerical domains, and women typically perform better in verbal domains.

    “Neisser et al. (1996) reported that most CATs are constructed so there are no overall gender differences; however, differences do exist within some cognitive domains, such as in spatial and numerical domains where men typically perform better, and in verbal domains where women typically perform better.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Neisser et al., 1996 source
    • The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Baenninger and Newcombe (1989) found some support that gender differences in spatial ability can be explained by prior experience.

    “Baenninger and Newcombe (1989) found some support that gender differences in spatial ability can be explained by prior experience.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Baenninger & Newcombe, 1989 source
    • The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Schmidt and Hunter (1998) concluded that scores on CATs were the strongest predictor of job performance for candidates without prior experience in the role.

    “In a review of 85 years of research into recruitment selection processes, Schmidt and Hunter (1998) concluded that scores on CATs were the strongest predictor of job performance when looking at candidates without prior experience in the role.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Schmidt & Hunter, 1998 source

  • May combine two claims.

    Premisecitation

    Hough et al. (2001) reported that White people were typically more motivated to do well on tests, more likely to believe tests would influence their future relationship with the company, and had more positive perceptions of the criterion validity of ability tests; women and older people have greater test anxiety.

    “White people were typically more motivated to do well on tests, were more likely to believe tests would influence their future relationship with the company that asked them to take them and had more positive perceptions of the criterion validity of ability tests.Women and older people have greater test anxiety.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hough et al., 2001 source
    • The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.

  • Classified as probably not a substantial claim.

    Premisecitation

    Cognitive ability tests (CATs) are used in employee selection and recruitment because they predict workplace performance.

    “Cognitive ability tests (CATs) are used as part of selection and recruitment due to their utility in predicting workplace performance (Bertua et al., 2005;Sackett et al., 2022;Schmidt & Hunter, 1998)”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Bertua et al., 2005 source
    • cites Sackett et al., 2022 source
    • cites Schmidt & Hunter, 1998 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Hough et al. (2001) suggested that reducing differences in test perceptions would have at best a small effect on test score differences.

    “findings were sometimes inconsistent and typically suggested reducing differences in test perceptions would have, at best, a small effect on test score differences.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hough et al., 2001 source

  • May combine two claims.

    Premisecitation

    Hough et al. (2001) reported that group differences in test performance lowered when motivation scores were partialled, test motivation had a moderating effect on test validity, and validity perceptions positively correlated with test performance.

    “Group differences in test performance lowered when motivation scores were partialled, test motivation had a moderating effect on test validity and validity perceptions positive correlated with test performance.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Hough et al., 2001 source

  • May combine two claims.

    Premisecitation

    Since Hough et al. (2001), a sizable body of evidence has supported the effects of stereotype threat, but it is recognised that stereotype threat does not fully explain group mean score differences, and some research suggests its effects are overstated.

    “Since then, whilst a sizable body of evidence has supported the effects of stereotype threat, it is recognised that it does not fully explain group mean score differences (e.g., Kirnan et al., 2009) and some research suggests effects are overstated (e.g., Finnigan & Corker, 2016).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Kirnan et al., 2009 source
    • cites Finnigan & Corker, 2016 source

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Quantitative synthesis was not considered appropriate given the heterogeneity of studies to be reviewed.

    “Given the heterogeneity of studies to be reviewed, quantitative synthesis was not considered appropriate.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Influential prior research suggested that group differences on CATs are not a result of test bias.

    “Whilst influential research has suggested these differences are not a result of test bias (e.g., Jensen, 1980)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jensen, 1980

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The Mixed Methods Appraisal Tool is reliable and efficient.

    “It is reliable and efficient (Pace et al., 2012)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Pace et al., 2012 source

  • Classified as probably not a substantial claim.

    Premisecitation

    Hough et al. (2001) concluded that cultural differences have no more than a modest impact on test score differences.

    “They concluded that cultural differences have no more than a modest impact on test score differences.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Hough et al., 2001 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Hunter and Hunter (1984) cited three papers showing that Black people's work performance is similar to, or lower than, levels which their cognitive ability would predict, and suggested these differences are likely due to poverty and hardship.

    “Hunter and Hunter (1984) cited three papers showing that Black people's work performance is similar, or lower, than levels which their cognitive ability would predict.They suggested these differences are likely due to poverty and hardship.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hunter & Hunter, 1984 source

  • May combine two claims.

    Premisecitation

    Roth et al. (2001) found around a one standard deviation difference in mean test scores between Black and White samples, with smaller differences between Hispanic and White samples.

    “A meta-analysis by Roth et al. (2001) looking at ethnic differences in occupational testing found support for previous findings that there is around a one standard deviation difference in mean test scores between Black and White samples, with smaller differences found when comparing Hispanic and White samples.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Roth et al., 2001 source

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Increased weighting was given to papers evaluated as having a lower risk of bias based on MMAT evaluation and to accounts with replicated findings across multiple papers.

    “Increased weighting will be given to papers evaluated as having a lower risk of bias based on MMAT evaluation and accounts with replicated findings across multiple papers.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The current review was conducted according to the systematic review protocols described in PRISMA-P.

    “The current review was conducted according to the systematic review protocols described in the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P; Moher et al., 2015).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Moher et al., 2015

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The review was not pre-registered with PROSPERO or similar.

    “The review was not pre-registered with PROSPERO or similar.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Hough et al. (2001) found little research on test coaching in occupational contexts; some studies showed potential in addressing group differences, but effects were modest and inconsistent, and findings were more consistent where test takers were less familiar with tests.

    “Little research had been done in occupational contexts and whilst some studies demonstrated potential in addressing group differences, effects were modest and inconsistent.Findings did seem to be more consistent where test takers were less familiar with tests.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Hough et al., 2001 source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Environmental research was rarely conducted on high-stakes occupational testing.

    “Similarly to biological research, environmental research was rarely conducted on high-stakes occupational testing.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    A further paper argued that replicating gender differences in spatial ability across 40 countries suggested an evolutionary basis.

    “A further paper argued that replicating gender differences in spatial ability across 40 countries suggested an evolutionary basis (Silverman et al., 2007).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Silverman et al., 2007 source

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    If some form or forms of test bias were impacting the scores of women, ethnic minorities and lower SES people, we would expect tests to underpredict their work performance.

    “The key concept is that if some form or forms of test bias were impacting the scores of women, ethnic minorities and lower SES people, we would expect tests to underpredict their work performance.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The lack of clarity does not suggest whether or not differences may be the result of test bias.

    “which then does not suggest whether or not differences may be the result of test bias.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductnone given

    High quality papers with large samples typically found that whilst DIF exists on test items, it does not meaningfully contribute to group mean score differences.

    “however high quality papers with large samples typically found that whilst DIF exists on test items, it does not meaningfully contribute to group mean score differences.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    94% of anxiety research involved student samples.

    “and 94% of research involved student samples”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Many criterion validity studies were conducted on very large, occupational samples.

    “and many studies were conducted on very large, occupational samples.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The relationship between strategy use and performance is best understood through other factors such as stereotype threat.

    “perhaps suggesting the relationship between strategy use and performance is best understood through other factors such as stereotype threat.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductnone given

    Higher scoring MMAT papers found effects as did lower scoring papers.

    “Higher scoring MMAT papers found effects as did lower scoring papers”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Anxiety research typically benefitted from large sample sizes.

    “though research typically benefitted from large sample sizes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Criterion validation studies benefited from mostly being based on high-stakes testing.

    “Criterion validation studies benefited from mostly being based on high-stakes testing.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Differences in criterion validity could support the relevance of methodological and contextual factors.

    “these papers were considered to meet inclusion criteria as differences in criterion validity could support the relevance of methodological and contextual factors.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Sample sizes in latent trait/measurement invariance papers were typically larger.

    “Whilst sample sizes were typically larger”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The lower sample sizes, lower MMAT scores, and high student proportion suggest this body of work is methodologically weaker than others.

    “suggesting this body of work is methodologically weaker than others.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    MMAT scores in anxiety research were amongst the lowest.

    “MMAT scores were amongst the lowest”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Recruiting using specific domains that are not related to job performance may inadvertently introduce adverse impact.

    “Recruiting using specific domains that are not related to job performance may inadvertently introduce adverse impact.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Anxiety differences are likely better explained in terms of stereotype threat.

    “Anxiety differences are likely better explained in terms of stereotype threat; however, it is notable that women typically display higher trait anxiety and report greater test anxiety.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The application of inclusion and exclusion criteria, data extraction and synthesis were all completed by one person.

    “The application of the inclusion and exclusion criteria, as well as data extraction and synthesis were all completed by one person.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    Students have pursued education and so are potentially more likely to be identified with cognitive domains, therefore stereotype threat research conducted on students may find effects larger than on working population samples.

    “Students have pursued education and so are potentially more likely to be identified with cognitive domains, therefore stereotype threat research conducted on students may find the effects are larger than on working population samples.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Men might perform worse than women in verbal tasks if they spend less time as children in verbal play.

    “By extension, it would follow that men might perform worse than women in verbal tasks if they spend less time as children in verbal play.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The isolation of factors and rarity of interaction-focused studies makes it difficult to tease apart potentially confounding accounts.

    “This makes it incredibly difficult to tease apart potentially confounding accounts.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Greater exposure to test content and more elaborated instructions in selection testing could reduce group differences without sacrificing validity.

    “It is possible that effective test strategies are developed through experience in the given domain, and that greater exposure to test content and more elaborated instructions in selection testing could reduce group differences without sacrificing validity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    There are irrefutable gender implications for the use of CATs in selection contexts.

    “Taken together, whilst the current review finds little to suggest ethnic or SES differences have specified biological causes, there are irrefutable gender implications for the use of CATs in selection contexts.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    Sex hormones may influence gender identification, which in turn may increase stereotype threat in specific domains.

    “These may be biological, but it is also possible that sex hormones influence gender identification, which in turn leads to increases in stereotype threat in specific domains.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The nature of the experimental approach could be contributing to the magnitude of stereotype threat effects seen.

    “However, there have been challenges in replicating studies in high stakes testing, which may suggest the nature of the experimental approach could be contributing to the magnitude of effects seen.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Forward citation searching was not as expansive as it could have been, which could have led to insightful research not being identified.

    “Also, forward citation searching was not as expansive as it could have been, which could have led to insightful research not being identified.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    A threat-related account becomes less relevant when considering developmental hormone levels on subsequent cognitive performance.

    “However, a threat-related account becomes less relevant when considering developmental hormone levels on subsequent cognitive performance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Group differences in CAT scores may not be the same as those seen in occupational samples and the extent to which different accounts explain those differences may vary.

    “Group differences in CAT scores may not be the same as those seen in occupational samples and the extent to which different accounts explain those differences may vary.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    There are likely important differences in activities performed in child play and adult leisure time by different ethnic and SES groups, which could contribute to domain experience and group differences.

    “There are also likely important differences in activities performed in child play and adult leisure time by different ethnic and SES groups, which could contribute to domain experience and group differences.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Anxious arousal has been found to be a mechanism for stereotype threat.

    “For example, anxious arousal has been found to be a mechanism for stereotype threat (Ben-Zeev et al., 2005) and yet no papers identified in this review investigating test anxiety examined stereotype threat.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ben-Zeev et al., 2005

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    University students are not representative of the wider working population and so may bias conclusions.

    “As such, they are not representative of the wider working population and so may bias conclusions.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: University students are typically young, educated and more likely to be from higher SES backgrounds.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    Student samples typically used in this research have vastly different motivation for completing tests as part of research, often for course credit where performance is irrelevant, compared with people taking CATs as part of a recruitment process who are motivated by the potential for a new role where performance is important.

    “Shewach et al. (2019)also commented that student samples typically used in this research have vastly different motivation for completing tests as part of research, often for course credit where performance is irrelevant, when compared with people taking CATs as part of a recruitment process who are motivated by the potential for a new role where performance is important.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Shewach et al. (2019) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Where meaningful gender differences on g are found, anxiety may play a part.

    “Meaningful gender differences are seldom found on g, but where they are, there is a possible that anxiety is playing a part.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Stereotype threat effects are typically more pronounced in individuals who are domain-identified.

    “As an example, Shewach et al. (2019) stated that stereotype threat effects are typically more pronounced in individuals who are domain-identified, that is, performance on the domain is important to them.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Shewach et al. (2019) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    Brain activation differences may be related to non-biological factors.

    “Brain activation differences are compelling, but again may be related to non-biological factors (e.g., Jaušovec & Jaušovec, 2012).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jaušovec & Jaušovec, 2012 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Papers on attitudes scored the highest on the MMAT on average.

    “Papers scored the highest on the MMAT on average”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Gaps in the literature, overuse of student samples, a lack of research on high stakes testing and an overreliance on Western samples all limit the ability to understand the causes of group differences in a recruitment context.

    “Gaps in the literature, overuse of student samples, a lack of research on high stakes testing and an overreliance on Western samples all limit our ability to understand the causes of group differences in a recruitment context.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Changes to test design and implementation will not eliminate group mean score differences in CATs, only potentially reduce them.

    “These are very important considerations for occupational testing and, as with the biological differences above, suggest that there is clear evidence that changes to test design and implementation will not eliminate group mean score differences in CATs, only potentially reduce them.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Having one person complete inclusion/exclusion, data extraction and synthesis could potentially have introduced bias.

    “Whilst the other authors reviewed this work, this could potentially have introduced bias.”
    Quote in the paperNot a substantial claim
148 of 148

  • The authors state this as an assumption.

    For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.

    “No papers in this review discuss differences between participants' identified genders and the biological sex they were assigned at birth, and so these will be assumed to be the same.”

  • The authors state this as an assumption.

    A lack of studies investigating a factor for a demographic group indicates a gap in the literature, not that the factor is irrelevant to that group.

    “If for example several compelling studies were found supporting a particular factor's contribution to gender score differences on CATs, the lack of studies investigating ethnicity would not indicate that this factor is not relevant, only that there is a gap in the literature.”

  • The paper relies on this without support.

    The workplace is not experienced in the same way by different groups of people.

    “These studies are clearly very important, but even when looking at objective criteria, the workplace is not experienced in the same way by different groups of people.”

  • The paper relies on this without support.

    University students are typically young, educated and more likely to be from higher SES backgrounds.

    “University students are typically young, educated and more likely to be from higher SES backgrounds.”

  • The paper relies on this without support.

    CATs are highly predictive of work success.

    “Whist CATs are highly predictive of work success, their use will result in differing selection rates for groups that exhibit mean score differences.”

  • The paper relies on this without support.

    Recruitment tests are high stakes, applicants are aware they are being evaluated, and they are aware of the competitive context.

    “Recruitment tests are high stakes, applicants are aware they are being evaluated, and they are aware of the competitive context.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Anxiety-stereotype-threat overlap is both denied and tabulated

    Minor

    The discussion says no paper investigating test anxiety examined stereotype threat, but Table 3 lists two papers addressing anxiety, attitudes, and stereotype threat together.

  • Environmental impact percentages exceed 100%

    Minor

    The environmental row assigns 93%, 7%, and 7% to the three impact categories, totaling 107%. These are presented as the alternative values of a single impact classification.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. Jensen, A. R. (1980). Bias in Mental Testing. Free Press.
  2. * McCormick, C. M., & Teillon, S. M. (2001). Menstrual cycle variation in spatial ability: Relation to salivary cortisol levels. Hormones and Behavior, 39(1), 29–38. https://doi.org/10.1207/s15327043hup1603_3 · DOI
  3. *Penner, A. M., & Willer, R. (2011). Stigma and glucose levels: Testing ego depletion and arousal explanations of stereotype threat effects. Current Research in Social Psychology, 16(3), n3.
  4. Thurstone, L. L. (1938). Primary mental abilities. University of Chicago Press.

Related

Authors
Cuppello, S., Zibarras, L. D., & Corr, P. J.
Affiliations
City St George's, University of London (GB)
Corresponding author
Cuppello, Stephen
ORCID
Venue
Trends in Psychology
Publisher
Springer Science and Business Media LLC
Published
2025-11-11
ISSN
2358-1883 (ISSN-L)
DOI
10.1007/s43076-025-00504-5
OpenAlex
W4416115091
Semantic Scholar
7ea697cd0a92d42df34ce2d970d7a8ed8e5e0e42
Cited by
0
References
258
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
Locations
  • Trends in Psychology (publishedVersion, cc-by)
  • City Research Online (City University London) (acceptedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
Keywords
cognitive ability tests, group differences, stereotype threat, differential item functioning, measurement invariance, self-confidence, test anxiety, test administration, personnel selection, gender differences, ethnicity, socioeconomic status, biological and environmental factors

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review