The DOI matches its Crossref record; 264 of the 264 references it cites were found in a registry or an index; 23 of the 186 sources its claims cite were checked for support.
Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review
Open accessAbstract
Commonly used in employee selection, cognitive ability tests (CATs) show mean score group differences by gender, ethnicity and socioeconomic status (SES). Prior research supports both substantive accounts, such as biological and environmental factors, and methodological and contextual accounts, such as stereotype threat and differential item functioning, related to group differences. This systematic review explores the evidence for the theoretical explanations that purport to explain these differences. Two hundred and twenty-five papers met inclusion criteria which explored potential explanations of group differences. These were grouped into 10 factors: biological, environmental, latent trait / measurement invariance, criterion validity, item bias, behaviour in test, anxiety, attitudes, experience, and stereotype threat. Results showed that whilst biological and environmental factors are related to group differences, so are several factors relevant to test design and administration, most notably stereotype threat, differences in domain experience and exposure and self-confidence. Theoretical and practical implications for the occupational use of CATs as well as limitations and recommendations are discussed.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This systematic review of 225 papers identifies biological and environmental factors, plus test-design and administration factors—especially stereotype threat, domain experience/exposure, and self-confidence—as related to mean score group differences on cognitive ability tests (CATs). It concludes that methodological and contextual factors offer promising avenues for fairer occupational CAT use, but the evidence is limited by few high-stakes occupational studies and heavy reliance on student and US samples.
Key findings
- Across 225 papers grouped into 10 factors, biological (63 papers; 67% impact “yes”) and environmental (15; 93% “yes”) factors were supported, but so were test-design/administration factors: stereotype threat (75; 81% “yes”), experience (18; 72% “yes”), and attitudes/self-confidence (23; 52% “yes”).
- Stereotype threat had the largest evidence base (75 papers; 51 gender, 27 ethnicity, 7 SES), with supported mechanisms including working-memory/executive-resource depletion and interpretation of anxious arousal; interventions included stating no group differences, role models, self-affirmation, and mindfulness. However, few studies replicated effects in high-stakes occupational settings.
- Experience/exposure findings showed training, practice tests/items, and video games improved spatial performance more for women and sometimes eliminated the gender gap; practice/training improved Black performance more than White performance. Self-confidence mediated gender–spatial performance, and manipulating confidence increased performance.
- Item bias/DIF exists but generally had negligible impact on overall group mean differences; criterion validity findings were inconsistent; latent trait/measurement invariance findings were discordant. Only 14% of studies met ecological validity criteria, 71% used student samples, 56% used US samples, and average MMAT score was 43%.
Methods
- PRISMA-P systematic review, not pre-registered. Searched PsycINFO, PsychArticles, Web of Science, and Business Source Ultimate with broad terms; backward and limited forward citation searching (first 20 papers and first 10 from the last five years per paper).
- Inclusion: adult humans (mean age 18+), original research, English peer-reviewed; excluded descriptive-only papers, bias-measurement-only papers, clinical/cognitive-decline samples, and tests of knowledge/crystallised intelligence or standardised academic tests such as SATs. Initial 1,124 records; 316 duplicates; 541 abstract-excluded; 140 excluded; 127 fully reviewed; plus 98 from citations = 225 papers.
- Data extraction and synthesis by the first author, reviewed by coauthors; papers tagged into 10 factors; MMAT used for risk of bias; narrative synthesis; extracted “Impact” (yes/no/maybe) and “Ecological Validity” (yes/no/partially).
Limitations
- Authors state: excluded child samples and high-stakes standardised education testing; broad scope and heterogeneity sacrificed specificity; gender/sex and ethnicity/race terms plus global inclusion may overlook nuances; screening, extraction, and synthesis were completed by one person; forward citation searching was not expansive.
- Text-evident: only 14% of studies met ecological validity criteria; 71% used student samples; 56% used US samples; only 8% included any participants outside North America/Europe and 4% exclusively outside those regions; average MMAT was 43%, with common risks (confounders accounted for only twice in nonrandomized studies; 18% representative; RCT blinding 10%, baseline comparability 11%).
- Age differences could not be reviewed: only two papers measured age and neither found reliable causal evidence; only 44 of 225 papers addressed multiple factors, and these rarely provided consistent interaction evidence.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
79 No full text of the sources · 62 No basis given · 45 Own interpretation · 37 Sources not found · 28 Own method · 26 Own result · 11 Not checked yet · 10 Supported · 6 Partly supported · 3 Supported so far · 2 Couldn’t confirm · 1 Not supported · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“several papers detail oral contraceptive use and type influences CAT performance on verbal and spatial task (Griksiene & Ruksenas, 2011;Hampson et al., 2022)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not checked — no full text of the sourcecites Griksiene & Ruksenas, 2011 source
- Partly supportedcites Hampson et al., 2022 sourceExperimental — accuracy not yet measured
Study 1 Results report differences across contraceptive-cycle phases on spatial and verbal-fluency tasks, and the paper examines how contraceptive formulations relate to spatial performance. It does not establish that contraceptive type influences verbal performance; the General Discussion also cautions that its observational design cannot conclusively establish causation. The reported tests are not identified as CAT. (LLM, conf=0.85)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“Research has used MGCFA elsewhere to suggest CATs do not necessarily have measurement invariance between cultures (Cockcroft et al., 2015).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
- Supportedcites Cockcroft et al., 2015 source
The Results (Multi-Group Confirmatory Factor Analyses; page not specified in our copy) state that the authors used MGCFA and concluded that the WAIS-III lacks measurement invariance across the British and South African samples. This supports the claim that a cognitive ability test need not be measurement-invariant across cultural groups. (LLM, conf=0.93)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“and androgens to spatial performance (Shute et al., 1983).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Supportedcites Shute et al., 1983 source
The Results passage states that both experiments supported a relationship between androgen levels and spatial abilities. The claim says only that androgens have been linked to spatial performance, without asserting a particular direction or causal effect. (LLM, conf=0.95)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Research has found that females with male twins outperform those with female twins on spatial tasks”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Heil et al., 2011
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Vuoksimaa et al., 2010 source
The Discussion reports that females with male co-twins outperformed females with female co-twins. The Results specify that this was measured on a mental rotation task, a spatial task. (LLM, conf=0.93)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“which may partially explain group differences (Vuoksimaa et al., 2012).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Partly supportedcites Vuoksimaa et al., 2012 sourceExperimental — accuracy not yet measured
The Introduction says testosterone has been suggested to play a role in sex differences in spatial ability. The Discussion reports that pubertal testosterone predicted later mental-rotation performance only among men; it does not show that pubertal testosterone explains differences between groups. (LLM, conf=0.82)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Lastly, a longitudinal study found pubescent testosterone levels were related to adult spatial performance in men”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Supportedcites Vuoksimaa et al., 2012 source
The Discussion concludes that pubertal testosterone was related to spatial cognition 7–9 years later in early adulthood. It specifies that the association was negative for adult mental rotation ability among men, supporting the citing sentence’s broader wording. (LLM, conf=0.94)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“supporting a prenatal masculinization hypothesis through hormonal transfer (Heil et al., 2011;Vuoksimaa et al., 2010)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Heil et al., 2011
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Vuoksimaa et al., 2010 source
The Results report that females with male co-twins performed significantly better on the mental rotation test than females with female co-twins. The Discussion explicitly says this finding is consistent with prenatal masculinization through intrauterine testosterone exposure, while treating that mechanism as a hypothesis rather than a demonstrated cause. (LLM, conf=0.94)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“though this has not been supported in all papers (Toivainen et al., 2018).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Supportedcites Toivainen et al., 2018 source
The Introduction says evidence for prenatal testosterone’s role in spatial ability is mixed and identifies studies with null results. The Discussion also reports that this study did not support the hypothesis across its spatial tasks. These passages support the claim that the hypothesis has not been supported in all papers; page numbers are not available in the supplied copy. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“or gender (Ispas et al., 2010)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Supportedcites Ispas et al., 2010 source
The Results (p. 227) report no evidence of gender-based differential prediction of job performance from GAMA in either employee sample, and the Discussion reiterates that finding. This paper supports the citation as an example of a study that did not find gender-based underprediction; it does not, by itself, establish how many papers reach that result. (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Multiple studies found using human figures as item content reduced the gender gap in mental rotation performance (e.g., Alexander & Evardone, 2008;Doyle & Voyer, 2018;Jansen & Lehmann, 2013)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not checked yetcites Alexander & Evardone, 2008 source
Checked before — its earlier verdict is held back until a new check runs.
- Not checked — no full text of the sourcecites Doyle & Voyer, 2018 source
- Supportedcites Jansen & Lehmann, 2013 source
The Mental rotation results report a smaller gender difference for human figures than for cube figures (d = 0.82 versus 1.78). The Cognitive processing speed discussion explicitly says that embodied human-figure stimuli reduced the gender gap and relates this finding to an earlier study. (LLM, conf=0.94)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not checked yet
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A further comment, which is true for much psychology literature (e.g., Hanel & Vione, 2016), is that most of this research depends on student samples, most commonly psychology student samples.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Partly supportedcites Hanel & Vione, 2016 sourceExperimental — accuracy not yet measured
The Introduction says student samples are extremely common in psychological studies, supporting the general point about psychology research. The Discussion (Implications and limitations) mentions psychology students but says the authors had no data on students’ subjects; it does not establish that psychology students are the most common student sample. (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Self-confidence has received support in mediating the relationship between gender and performance on spatial tests (Arrighi & Hausmann, 2022;Cooke-Simpson & Voyer, 2007;Picucci et al., 2011).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Supportedcites Arrighi & Hausmann, 2022 source
The Discussion (page not provided) states that self-confidence partially mediated the association between sex/gender and performance in both mental-rotation tasks studied. This supports the claim that self-confidence has received support as a mediator of gender differences in spatial-test performance. (LLM, conf=0.93)
- Not checked — no full text of the sourcecites Cooke-Simpson & Voyer, 2007 source
- Not checked — no full text of the sourcecites Picucci et al., 2011 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Also, overreliance on student samples meant only 18% could be seen as representative of target populations.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Several used brain scans during testing and found brain activation differences between women and men (e.g., Gur et al., 2000;Hugdahl et al., 2006;Jaušovec & Jaušovec, 2012;Jordan et al., 2002).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“However, only two papers (Frisby & Osterlind, 2006;Reeve et al., 2008) measured age.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Seventy one percent of studies were based on majority or exclusively student samples.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only 14% of studies met the criteria for ecological validity.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the 225 papers, 181 only addressed one of the ten factors.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In both papers, reliable evidence supporting causal factors for age differences were not found.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the quantitative nonrandomized studies, criteria 3.4 (”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Sixty-three papers examined biological factors as an explanation for group differences in CAT scores.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“with some evidence suggesting female/male differences in spatial abilities were not found for women in the menstrual phase (Moody, 1997).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not checked — no full text of the sourcecites Moody, 1997 source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Parental education level (McKay et al., 2003), individual education level (Dolan et al., 2006) and income (Díaz et al., 2012), which vary by ethnicity and SES, were found to be related to cognitive ability.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not checked — no full text of the sourcecites Dolan et al., 2006 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Testosterone levels have been linked to spatial performance in both genders (e.g., Burkitt et al., 2007;Silverman et al., 1999;Yang et al., 2007)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and findings are less clear”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Oestrogen has been linked to verbal performance (Rosenberg & Park, 2002)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not checked — no full text of the sourcecites Rosenberg & Park, 2002 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“Additionally, multiple papers have used twin samples to support the role of environmental factors in group differences (Johnson & Bouchard, 2007;Rushton et al., 2007a).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not matchedcites Johnson & Bouchard, 2007
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Rushton et al., 2007a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“Finally, preference for gendered spatial toys as a child was related to spatial performance as an adult (Moè et al., 2018)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not matchedcites Moè et al., 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“a further study finding ethnic differences were more pronounced on items a twin comparison suggested were more heritable (Rushton et al., 2007a(Rushton et al., , 2007b)).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Rushton et al., 2007a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Rushton et al., 2007b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Two papers supported a genetic explanation for gender differences”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and no papers included SES.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“administering testosterone resulting in increased spatial performance in women (Aleman et al., 2004)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Aleman et al., 2004
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Out of the 63 papers in this factor, 61 related to gender”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“However, studies did not universally support these difference (e.g., Halari et al., 2006;Unterrainer et al., 2000)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Halari et al., 2006
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites Unterrainer et al., 2000 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Several papers have linked prenatal testosterone to numerical and spatial performance (e.g., Collaer et al., 2007;Kempel et al., 2005)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not checked — no full text of the sourcecites Collaer et al., 2007 source
- Not matchedcites Kempel et al., 2005
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“and differences in lateral differentiation between genders (Levy & Reid, 1978).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not checked — no full text of the sourcecites Levy & Reid, 1978 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Parental education level (McKay et al., 2003), individual education level (Dolan et al., 2006) and income (Díaz et al., 2012), which vary by ethnicity and SES, were found to be related to cognitive ability.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not matchedcites Díaz et al., 2012
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“as has spatial play as a child (Frenken et al., 2016).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not checked — no full text of the sourcecites Frenken et al., 2016 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“with one finding modest support for a genetic basis of gender differences (Johnson & Bouchard, 2007)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Johnson & Bouchard, 2007
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“gonadotropins to verbal and spatial performance (Gordon & Lee, 1986)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Gordon & Lee, 1986
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“Research also provided some support for anatomical differences (Koscik et al., 2009)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Koscik et al., 2009
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“with significant discordance in findings.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“one paper found activation differences were inconsistent with cognitive domains that displayed gender differences (Bell et al., 2006).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Bell et al., 2006
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“Griksiene and Ruksenas (2011) found verbal and spatial performance differences between women using different types of hormonal birth control”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not checked — no full text of the sourcecites Griksiene and Ruksenas (2011) source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“This effect has been widely supported when looking at ethnic differences (e.g., Nyborg & Jensen, 2000;Rushton & Skuy, 2000;Rushton et al., 2004;Te Nijenhuis & Van der Flier, 1997).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only 2 papers related to ethnicity”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“with women using oral contraceptives performing worse on spatial tasks than other women and men (Griksiene et al., 2018).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not checked — no full text of the sourcecites Griksiene et al., 2018 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A lot of research attention has been given to the role of sex hormones in gender differences in ability, although findings are often inconsistent.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Indeed, ethnic differences were significantly reduced when controlling for years of education, language proficiency and genera-tional immigration status (Hausdorf & Robie, 2018).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not checked — no full text of the sourcecites Hausdorf & Robie, 2018 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Parental education level (McKay et al., 2003), individual education level (Dolan et al., 2006) and income (Díaz et al., 2012), which vary by ethnicity and SES, were found to be related to cognitive ability.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not checked — no full text of the sourcecites McKay et al., 2003 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim could be matched to a paper in Crossref, so there is nothing to check it against.
In the paper
“However, findings are not ubiquitous with some studies not finding a link between cycle time and performance (e.g., Epting & Overman, 1998;Gordon & Lee, 1993).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Epting & Overman, 1998
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Gordon & Lee, 1993
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“Ethnic differences on cognitive domains were more pronounced on verbal tasks”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not checked — no full text of the sourcecites Razani et al., 2007 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Support for several environmental explanations of group differences were shown in 15 papers.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“administering luteinising hormone releasing hormone resulting in performance changes on several specific cognitive domains (Gordon et al., 1986).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Gordon et al., 1986
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“findings were frequently replicated across multiple studies.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Lefkowitz and Battista (1995) found that supervisors gave higher performance ratings to same-ethnicity staff”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not checked — no full text of the sourcecites Lefkowitz and Battista (1995) source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Regarding ethnicity, Thames et al. (2015) found Black test takers experienced greater anxiety related to negative performance evaluation than White test takers”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
What the sources say
- Not matchedcites Thames et al. (2015)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“as has regulation of cognition, defined as the way people plan, implement strategies, selfevaluate and error check (Nguyen et al., 2003).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Nguyen et al., 2003 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“though not consistently (Voyer & Jansen, 2016).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not matchedcites Voyer & Jansen, 2016
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Prior knowledge can lead to ethnic biases in items (Fagan & Holland, 2002, 2007)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“though again not consistently (Walsh et al., 1999).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not checked — no full text of the sourcecites Walsh et al., 1999 source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“or found that differential validity was removed by controlling for education (Reeb, 1976).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not checked — no full text of the sourcecites Reeb, 1976 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“and that by preventing more male-favoured holistic strategies, gender differences can be modestly reduced (Hirnstein et al., 2009).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Hirnstein et al., 2009 source
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Many papers do not find this by ethnicity (Sackett et al., 2023)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not matchedcites Sackett et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“and several papers have found that CATs overpredict performance for women and ethnic minorities (e.g., Roberts & Skinner, 1996).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not checked — no full text of the sourcecites Roberts & Skinner, 1996 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“women report being more anxious in highstakes testing (Reeve et al., 2008;Stenlund et al., 2017).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
What the sources say
- Not checked — no full text of the sourcecites Reeve et al., 2008 source
- Not matchedcites Stenlund et al., 2017
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“There has also been some support that using realistic stimuli reduces gender bias (Fisher et al., 2018)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not matchedcites Fisher et al., 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“The use of ineffective test strategies and test preparation have been found to mediate the relationship between ethnicity and performance (Ellis & Ryan, 2003)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Ellis & Ryan, 2003 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“which in turn was related with lower performance”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
What the sources say
- Not matchedcites Thames et al. (2015)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“Some research showed no ethnic differences on culture-free tests (Domino & Morales, 2000)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not checked — no full text of the sourcecites Domino & Morales, 2000 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Several papers have found women use different, often piecemeal, strategies in spatial tasks (Cherney & Collaer, 2005;Heil & Jansen-Osmann, 2008;Picucci et al., 2011)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Additionally, Gabriel et al. (2011) found stress impacted spatial performance differently for women and men”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
What the sources say
- Not matchedcites Gabriel et al. (2011)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“In a study exploring a range of testing behaviours, large differences were found between Black and White people in high-stakes testing, even when ability was controlled for (Frisby & Osterlind, 2006).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Frisby & Osterlind, 2006 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“and when controlling for this, the criterion validity of tests reduced.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not checked — no full text of the sourcecites Lefkowitz and Battista (1995) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Separately, in a study in the USA and Austria, differences in test behaviour were related to motivational differences in completing the experimental task (Tanzer et al., 1995)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Tanzer et al., 1995 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is not clear as to whether CATs predict job performance equally for different groups”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Considering gender, Stenlund et al. (2017) found that women and men employ different testtaking strategies.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not matchedcites Stenlund et al. (2017)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“Similarly, studies found that including gender stereotyped item content created bias (Rahe & Jansen, 2022;Rahe et al., 2020)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not matchedcites Rahe & Jansen, 2022
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites Rahe et al., 2020 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“though providing social context does not seem to reduce ethnic bias (DeShon et al., 1998).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not checked — no full text of the sourcecites DeShon et al., 1998 source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Efforts to reduce experience of test anxiety in an occupational context could be worth exploring to protect against gender differences in CATs, though evidence is less than convincing.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“What is less clear is the mechanisms by which they influence performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Meaningful gender differences are seldom found on g, but where they are, there is a possible that anxiety is playing a part.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
In the paper
“that confidence was related to performance”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not matchedcites Estes and Felker (2012)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, no papers were found that sought to explain age difference in CAT scores.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“State anxiety does appear to be related to test performance”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
In the paper
“There is some support that face valid tests reduce group differences in performance (Grand et al., 2010).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not checked — no full text of the sourcecites Grand et al., 2010 source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“and manipulating confidence increased performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not matchedcites Estes and Felker (2012)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“Ployhart and Ehrhart (2002) found that reducing ethnic differences in test motivation could reduce performance differences by 5-30%.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not checked — no full text of the sourcecites Ployhart and Ehrhart (2002) source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Taken together, whilst the current review finds little to suggest ethnic or SES differences have specified biological causes, there are irrefutable gender implications for the use of CATs in selection contexts.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Experiential difference findings were almost exclusively related to gender differences on spatial tests from papers exploring why women are underrepresented in STEM.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“though support for theories was often mixed and ambiguous.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, anxious arousal has been found to be a mechanism for stereotype threat (Ben-Zeev et al., 2005) and yet no papers identified in this review investigating test anxiety examined stereotype threat.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The majority of studies reviewed were not conducted in these settings.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“though there is support that these differences both do (Chan et al., 1997)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not checked — no full text of the sourcecites Chan et al., 1997 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Differences in test strategy seems to be well supported with groups performing worse on specific domains often employing less effective strategies.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This being said, from the evidence explored here, item bias is highly unlikely to contribute in any meaningful way to group mean score differences in occupational testing.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Perhaps most significantly, finding that stereotype threat, test experience and self-confidence moderate the relationship between certain group identities and scores on CATs has implications for the understanding of group differences and the ways we measure ability.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Threat also was not considered in the one paper finding that manipulating self-confidence influenced test scores (Estes & Felker, 2012).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not matchedcites Estes & Felker, 2012
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Regarding motivational differences, support was less clear cut.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Those papers that did consider multiple factors rarely aimed to consider how factors interact to explain group differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There is some support that exploring differences in state anxiety may be insightful, and whilst only impacting group differences negligibly, exploring DIF in CATs is still something that should be recommended.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Indeed only 14% of studies reviewed met the criteria for ecological validity.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Additionally, a majority of the papers reviewed looked at factors in isolation.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Black people's beliefs that CATs are measures of knowledge rather than ability was related to lower test scores (Palumbo & Steele-Johnson, 2014).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not matchedcites Palumbo & Steele-Johnson, 2014
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“Estes and Felker (2012) found women were less confident in a spatial task”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not matchedcites Estes and Felker (2012)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“In a study looking not at strategy but test experience, Cherney et al. (2003) found exposure to test items or providing detailed instructions reduced the gender gap in spatial performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
What the sources say
- Not checked — no full text of the sourcecites Cherney et al. (2003) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The impact of environmental factors on group mean score differences is unambiguous.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Whilst this review found biological and environmental factors to be important, these artifacts of the testing process need to be considered when exploring group differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Other differences in test behaviour such as differences in effort did not appear especially relevant.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No notable trends were observed when comparing US studies with non-US studies, however only 8% of studies involved at least some participants outside of North America and Europe with only 4% using participants exclusively outside of those regions.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“From the reviewed research, it is clear that occupational testing has received much less research attention than educational testing, presumably as private occupational test publishers are less motivated than public education departments to explore differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Moè et al. (2009) found that a belief that ability can be improved was related to higher spatial performance in women.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not matchedcites Moè et al. (2009)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“opening up the possibility that group differences may be partially explained by biases in tests or testing processes.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Introduction
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“DIF, whereby two individuals possessing identical levels of a latent trait but belonging to different groups have different chances of getting a given question correct, was generally found to have a weak impact on overall test scores where it existed (Hough et al., 2001).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Hough et al., 2001 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Socioeconomic status (SES) and educational opportunities are likely to confound the relationship between ethnicity and cognitive ability.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“subsequent research has found significant limitations in methods used (e.g., Dolan et al., 2004;Wicherts, 2017)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Introduction
What the sources say
- Not matchedcites Dolan et al., 2004
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not checked — no full text of the sourcecites Wicherts, 2017 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“A more recent meta-analysis by Sackett and colleagues (2022) reported a slightly smaller but still large effect size when looking at Black-White CAT score differences in occupational samples (d =.79).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Cognitive Ability and Mean Score Group Differences
What the sources say
- Not checked — no full text of the sourcecites Sackett et al., 2022 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Substantive accounts, such as genetic group differences (e.g., Warne, 2021) or socio-cultural differences (e.g., Hausdorf & Robie, 2018), are based on biological or environmental factors that are unlikely to be mitigated through changes to testing procedures.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Introduction
What the sources say
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“However, methodological and contextual accounts, such as stereotype threat (e.g., Steele & Aronson, 1995) or differential item functioning (DIF; e.g., Abad et al., 2004), suggest potential avenues through which changes to the administration of CATs could minimise mean score group differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Introduction
What the sources say
- Not checked — no full text of the sourcecites Steele & Aronson, 1995 source
- Not matchedcites Abad et al., 2004
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggesting a link between hormones and cognitive abilities.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These papers were included as findings of either measurement invariance or group difference on latent traits, support that group differences in mean test scores are due to differences in ability, not test bias.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which may limit generalisability.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which could suggest a cause for gender differences in these domains.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Effects were observed in papers scoring higher on the MMAT”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“findings were consistent across papers scoring higher and lower on the MMAT”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“suggesting environmental factors contribute to differences (Razani et al., 2007).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Not checked — no full text of the sourcecites Razani et al., 2007 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“However, no studies relating to this factor were conducted on high-stakes testing”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“providing strong support for biological accounts contributing to CAT score differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and strong support was found that various environmental factors contribute to differences for each group.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Samples sizes were typically larger”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Whilst sample sizes in these papers were often small”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“so findings were less likely to be due to methodological limitations.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggesting research quality was not a factor influencing findings.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Sample sizes were on the lower end”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggesting this body of research is methodologically weaker than others.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite this, taken as a whole, criterion validation studies do not offer much to this paper's research question.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“and 86% of samples were primarily students”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“MMAT scores were on the lower end”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggesting methodological limitations in this area”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There were no compelling accounts of variations in test behaviour that contribute to group differences, beyond those that may be explained though other accounts such as stereotype threat.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“as were average MMAT scores”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Given such a conflicting body of literature, latent trait and measurement invariance research currently does not offer much to address the research question for this paper.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“the average MMAT score for this factor was the lowest for all factors”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which has interesting implications for generalising findings from experimental studies to high-stakes testing.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggesting differences in anxiety may contribute to group score differences on CATs.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“There are likely to be significant differences in terms of factors such as motivation and anxiety when taking a test for an experiment compared with taking a test for a job (e.g., Barry et al., 2010) and text anxiety is found to negatively impact test scores in occupational testing (McCarthy & Goffin, 2005).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not checked — no full text of the sourcecites Barry et al., 2010 source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests that group differences are not based on test biases.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This review also excluded studies that were solely based on performance on highstakes standardised education testing, due to assumed confounding factors such as differences in parental and school support.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Thomsen et al. (2000) found support that some brain activation differences between women and men when taking tests was actually a result in different test strategy adoption, which could suggest some biological accounts for group differences are actually behavioural ones.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not matchedcites Thomsen et al. (2000)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
In the paper
“This is especially important when considering that organisations lack highly valid selection methods that don't show group differences (Sackett et al., 2022).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not checked — no full text of the sourcecites Sackett et al., 2022 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Research regarding stereotype threat has found that individuals under threat are more likely to withdraw or put in less effort as a means of self-preservation (e.g., Spencer et al., 2016), and so this could explain why motivational differences were found in some studies.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
What the sources say
- Not checked — no full text of the sourcecites Spencer et al., 2016 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thomsen et al. (2000) found support that some brain activation differences between women and men when taking tests was actually a result in different test strategy adoption, which could suggest some biological accounts for group differences are actually behavioural ones.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In order to focus on studies relevant for occupational testing, we did not include studies primarily involving child samples.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Indeed, the overuse of student samples is concerning as student and nonstudent samples typically produce differing results (Shen et al., 2011).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not checked — no full text of the sourcecites Shen et al., 2011 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Without significantly more research in this area, it is not possible to reliably construct a conceptual model to represent the interactions between factors, which would be valuable in understanding how to potentially mitigate group differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Research regarding stereotype threat has found that individuals under threat are more likely to withdraw or put in less effort as a means of self-preservation (e.g., Spencer et al., 2016), and so this could explain why motivational differences were found in some studies.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As such, it is not possible to infer whether these criterion validation studies effectively support measurement invariance, especially when results are often inconsistent.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The workplace is not experienced in the same way by different groups of people.
In the paper
“There are likely to be significant differences in terms of factors such as motivation and anxiety when taking a test for an experiment compared with taking a test for a job (e.g., Barry et al., 2010) and text anxiety is found to negatively impact test scores in occupational testing (McCarthy & Goffin, 2005).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not checked — no full text of the sourcecites McCarthy & Goffin, 2005 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Whist CATs are highly predictive of work success, their use will result in differing selection rates for groups that exhibit mean score differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: CATs are highly predictive of work success.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Concerns around demand characteristics aside, gener-alising findings from student samples who have little or no employment experience to occupational testing is problematic.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“however the mechanism seems to be better explained through stereotype threat.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
In the paper
“Spencer et al. (2016) describe anxiety as a supported mechanism for stereotype threat and so it logically follows that stereotype threat may impact test performance differently on high-stakes testing when compared with laboratory studies.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not checked — no full text of the sourcecites Spencer et al. (2016) source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 98
- It may state two separate claims as one. 73
- The quote only partly states this claim. 1
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“other authors have found that CAT performance was less predictive of training performance for Black recruits (Carretta & Doub, 1998;De Meijer et al., 2008)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not checked — no full text of the sourcecites Carretta & Doub, 1998 source
- Disputedcites De Meijer et al., 2008 sourceExperimental — accuracy not yet measured
The Main results (§3.2; page not available) state the opposite direction: the cognitive ability test, especially its verbal subtests, was more predictive of training performance for ethnic minority trainees than for ethnic majority trainees. The Discussion (§4; page not available) reports corrected validities of .14 and .04, respectively. The paper describes Dutch ethnic groups, not a separate finding for Black recruits. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
Every source the paper cites for this claim was checked and supports it.
In the paper
“These differences have the potential to further entrench workplace inequalities (Woods & Patterson, 2024).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Introduction
What the sources say
- Supportedcites Woods & Patterson, 2024 source
In “Selection Design: Compounding Issues of Socio-Economic Inequality” (page not provided), the authors say that heavy reliance on cognitive testing, given mean score differences, results in less diversity and maintains socio-economic inequality. The framing section also identifies adverse impact as a way such testing can maintain and exacerbate social inequality. (LLM, conf=0.93)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Commonly used in employee selection, cognitive ability tests (CATs) show mean score group differences by gender, ethnicity and socioeconomic status (SES).”
- Also stated in: § Introduction · boosted
“though widely reported mean score group differ-ences threaten to undermine their practical utility (e.g., Anastasi, 1958;Bates et al., 2013;Neisser et al., 1996;Roth et al., 2001).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Abstract
What the sources say
- Not matchedcites Anastasi, 1958
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Could not verifycites Bates et al., 2013 sourceExperimental — accuracy not yet measured
The available passages are bibliographic records (no section or page) and do not discuss cognitive ability test scores across gender, ethnicity, or SES groups. Searches for those group differences and for employee-selection terminology found no substantive passage addressing the claim. (LLM, conf=0.72)
- Not checked — no full text of the sourcecites Neisser et al., 1996 source
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.
- Also stated in: § Introduction · boosted
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Adults from lower SES backgrounds typically score modestly lower, with childhood SES found to explain around 5% of adult intelligence (Bates et al., 2013).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Cognitive Ability and Mean Score Group Differences
What the sources say
- Could not verifycites Bates et al., 2013 sourceExperimental — accuracy not yet measured
The indexed passages returned for this paper are bibliographic metadata (no section or page), not findings about intelligence scores. Searches for the lower-SES comparison and the claimed 5% figure found no passage in our copy that addresses either point. (LLM, conf=0.76)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“Risk of bias was conducted using the Mixed Methods Appraisal Tool (MMAT; Hong et al., 2018), as it provides a single framework for which to appraise the studies included in this review employing varying methodologies.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Inclusion criteria
What the sources say
- Supportedcites Hong et al., 2018 source
Section 2 (p. 3) identifies the MMAT as a tool for appraising the methodological quality of empirical studies across five study-design categories. The Conclusion (p. 5) says it provides criteria for different designs within a single tool, supporting the cited reason for using it to assess studies with varying methodologies. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“and has been previously employed to assess risk of bias in systematic reviews (e.g., Gledhill et al., 2018).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Inclusion criteria
What the sources say
- Supportedcites Gledhill et al., 2018 source
In “Assessing risk of bias,” Gledhill et al. state that they used the MMAT to appraise studies included in their systematic review. “Establishing rigour” explicitly links the MMAT criteria to their risk-of-bias ratings. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Multiple papers found women's performance fluctuated throughout the menstrual cycle with better spatial performance in the menstrual phase and better verbal performance in the luteal phase (e.g., Hampson, 1990aHampson, , 1990b;;McCormick & Teillon, 2001;Šimić & Santini, 2012)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Hampson, 1990a
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites McCormick & Teillon, 2001
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Šimić & Santini, 2012 source
The passage labeled Abstract reports that many studies found better verbal-task performance in the midluteal phase than in the menstrual phase, and that healthy young women seemed to perform spatial tasks better in the menstrual phase. It also notes studies with no spatial difference. This supports the claim about findings across multiple papers, although this paper's own results favored midluteal performance on its spatial task. (LLM, conf=0.87)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
Every source the paper cites for this claim was checked and supports it.
In the paper
“Previous research has suggested that some of these factors may be genetically influenced (e.g., SES; Krapohl & Plomin, 2016)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
What the sources say
- Supportedcites Krapohl & Plomin, 2016 source
The Abstract (page not provided) reports that polygenic scores accounted for approximately 2.5% of variation in family SES. The Discussion (page not provided) also describes how exposure to environments can vary with genotype. Together, these support the hedged claim that an environmental factor such as SES may be genetically influenced. (LLM, conf=0.91)
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“although other studies have found no links with spatial, verbal or other gendered domains (e.g., Hirnstein et al., 2014;McKeever et al., 1987;Puts et al., 2010).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Supportedcites Hirnstein et al., 2014 source
The Results section reports that testosterone levels were not significantly associated with any cognitive task score in men or women. The tested tasks included spatial and verbal measures; the Discussion likewise states that testosterone levels were generally unrelated to cognitive performance. Page numbers are not available in the supplied copy. (LLM, conf=0.92)
- Supportedcites McKeever et al., 1987 source
The paper’s opening summary states that it found no relationship between testosterone and either spatial or verbal measures. Its discussion likewise reports no correlation with spatial ability or spatial–verbal discrepancy scores in males or females. The finding concerns the ability measures tested, not every possible gendered domain. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites Puts et al., 2010 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Finally, Forbes and Schmader (2010) found that training a more positive attitude towards the testing domain increased motivation but not performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Supportedcites Forbes and Schmader (2010) source
The General Discussion reports that retraining women to have more positive math attitudes increased their math motivation in three experiments. In “Implication for Performance” (page unavailable), the authors report null effects of attitude retraining on math-test performance in Studies 1 and 2. The finding concerns women’s motivation in the math domain, rather than testing domains generally. (LLM, conf=0.92)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results showed that whilst biological and environmental factors are related to group differences, so are several factors relevant to test design and administration, most notably stereotype threat, differences in domain experience and exposure and self-confidence.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Abstract
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This proportion varied significantly by factor, with the most common use in Anxiety (94%) and Attitudes (91%) and the least common use in Criterion Validity (0%) and Environmental Differences (50%).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Attitudes had the highest average (48%) and Latent Trait/Measurement invariance had the lowest average (38%).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the 225 papers, 55% employed exclusively US samples, 8% Canadian, 6% German, 4% British, 20% from other single counties and a further 7% used samples from multiple countries.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In terms of methodological quality, 147 studies were classified as quantitative nonrandomized, 79 as quantitative randomized controlled trials and one as quantitative descriptive.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The overall average MMAT score was 43% and this did not vary much”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This resulted in ten factors, namely: biological differences, environmental differences, latent trait differences/measurement invariance, criterion validity differences, item bias, test taking behavioural differences, test anxiety, test attitudes and perceptions, test experience and stereotype threat.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Inclusion criteria
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This again varied by factor with Criterion Validity meeting these criteria most frequently (94%) followed by Latent Trait/ Measurement invariance (23%).The two factors meeting these criteria the least were Biological (0%) and Stereotype Threat (4%).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In quantitative randomized controlled trials, only 10% met condition 2.4 (”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one. The quote only partly states this claim.
In the paper
“A similar approach to these studies concluded that there was measurement invariance with regard to women and men, suggesting group differences were at the latent trait level (Dolan et al., 2006).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
- Not matchedcites Dolan et al., 2006
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
In the paper
“with one study finding that the hypothesis was erroneously supported when comparing CAT items with personality test items (Wicherts, 2017).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
- Not checked — no full text of the sourcecites Wicherts, 2017 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Thirteen papers related to latent trait differences or measurement invariance”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“A further study used multi-group confirmatory factor analysis (MGCFA) on a data set previously used to support the hypothesis (cited above: Te Nijenhuis & Van der Flier, 1997) and was unable to find factorial invariance, undermining original conclusions (Dolan et al., 2004).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
- Not matchedcites Dolan et al., 2004
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Overall, 8 papers related to gender, 7 to ethnicity and 3 to SES”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Although not as pronounced, similar monthly effects have been found in men (Courvoisier et al., 2013).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not matchedcites Courvoisier et al., 2013
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
In the paper
“authors have variously found support that group mean score differences are related to differences in processing speeds in more elementary cognitive tasks and so not test bias (Vernon & Jensen, 1984)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
- Not checked — no full text of the sourcecites Vernon & Jensen, 1984 source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Measurement invariance has also been supported by finding high correlations between item difficulties between different cultures, suggesting group differences are in g rather than culturally determined (Rushton et al., 2007b).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
- Not matchedcites Rushton et al., 2007b
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“However, this theory has been criticised as being inappropriate”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“that neuropsychological predictors of CAT performance differed between cultures which suggests different constructs were being measured (Fasfous et al., 2013)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
- Not checked — no full text of the sourcecites Fasfous et al., 2013 source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A further hormonal development account is inferred by 2D:4D ratios, which has been found to correlate negatively to prenatal testosterone and positively to prenatal oestrogen.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“although other research suggests there is potential to overcome this with practice items (Tanzer et al., 1995).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not checked — no full text of the sourcecites Tanzer et al., 1995 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Many of the 14 papers in this factor examined differences in test strategy.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“although this link is perhaps less clear.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Of 16 papers relating to anxiety, most were based on gender.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“however, they concluded that underpredictions of minority performance was unlikely.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not matchedcites Berry et al. (2014)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“and Sokolowski et al. (2019) found women display more maths anxiety which is related to performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
What the sources say
- Not checked — no full text of the sourcecites Sokolowski et al. (2019) source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“and less predictive of job performance (Gardner & Deadrick, 2008, 2012).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Papers cited here mostly assume no bias in the criterion, which is often subjective manager ratings.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Harrison et al. (2006) found no differences in effort between SES or ethnic groups.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Harrison et al. (2006) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“with all but one simulation study using data from high-stakes testing.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“with some suggesting that range restriction for minority applicants explains any differences found (Roth et al., 2014)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not matchedcites Roth et al., 2014
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedconditional- Asserted: fails — The authors may not put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite women typically scoring higher in trait anxiety, they perform similarly to men on overall CATs.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Trait and state anxiety are related to performance (Kumari & Corr, 1998)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
What the sources say
- Not checked — no full text of the sourcecites Kumari & Corr, 1998 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Whilst most of this research looks at gender, ethnicity and SES have also been studied.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Generally, studies either did not identify biased items, or biased items only negligibly impacted overall test scores (e.g., Abad et al., 2004;Jensen & McGurk, 1987;Vanderpool & Catano, 2008).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Campbell et al. (2018)found women showed greater pupil dilation in a spatial task”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Campbell et al. (2018) source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“some papers' findings were more ambiguous (Harville, 1996)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not checked — no full text of the sourcecites Harville, 1996 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Of the papers that met inclusion criteria, 16 referred to criterion validity or predictive validity”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Berry et al. (2014) suggested that CATs are seldom used in isolation in recruitment decision”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not matchedcites Berry et al. (2014)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Twenty-three papers examined potential item bias or DIF.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Other notable findings were that weighting subtests differently can potentially reduce adverse impact without sacrificing criterion validity (Wee et al., 2014).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not matchedcites Wee et al., 2014
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In all, 15 studies looked at ethnicity and 5 at gender.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“and that Black/White test score differences were not replicated on learning tasks potentially suggesting cultural bias (Grubb & Ollendick, 1986).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
What the sources say
- Not checked — no full text of the sourcecites Grubb & Ollendick, 1986 source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Studies related to test behaviour have been conducted with gender (9), ethnicity (5) and SES (1).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“despite gender differences on general CATs not typically being found”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Item bias research is roughly evenly split between gender (13 papers) and ethnicity (10).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Some have found modest support for underprediction of Black training performance (Houston, 1987;Te Nijenhuis & Van der Flier, 2000).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“although other research has found modest gender differences on similar tests (Arendasy & Sommer, 2012).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
What the sources say
- Not checked — no full text of the sourcecites Arendasy & Sommer, 2012 source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“found no predictive bias but found tests were less valid for Black applicants (Rotundo & Sackett, 1999)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
What the sources say
- Not checked — no full text of the sourcecites Rotundo & Sackett, 1999 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“which the authors suggested was a proxy for cognitive effort”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Campbell et al. (2018) source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In all, 12 papers examined ethnicity and 2 examined gender.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“a paper investigating stereotype threat found that under threat, people are more likely to persevere with an ineffective problem-solving strategy (Carr & Steele, 2009)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
What the sources say
- Not checked — no full text of the sourcecites Carr & Steele, 2009 source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Beyond strategy use, there has been some suggestion of group differences in effort.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“and do not (Chan, 1997) mediate relationship between ethnicity and performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not checked — no full text of the sourcecites Chan, 1997 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Anxiety differences are likely better explained in terms of stereotype threat; however, it is notable that women typically display higher trait anxiety and report greater test anxiety.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There were multiple papers found in the search process that looked specifically at cognitive decline in retirement ages, but these did not meet inclusion and exclusion criteria as they were not based on working age participants.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, genetic evidence is currently lacking though promising, and so the area warrants further exploration.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Additional, there is a paucity of research exploring the interaction of these explanatory factors.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Studies have shown that Black people have lower motivation and face validity perceptions in testing (Arvey et al., 1990)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not checked — no full text of the sourcecites Arvey et al., 1990 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Parental and individual education, income, immigration status, and language proficiency have been found to relate to ethnic and SES differences, whilst gendered play and contraceptive use and type has been found to be related to gender differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There is some support that exploring differences in state anxiety may be insightful, and whilst only impacting group differences negligibly, exploring DIF in CATs is still something that should be recommended.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Conclusion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, much of the evidence has not been conducted in high-stakes occupational testing and is overly reliant on US student samples.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Self-confidence was found to be an important mediator between gender and spatial performance and is the strongest attitudinal account that may related to group differences on CATs.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Many CATs have been found to show DIF, and test publishers should be mindful of this and endeavour to remove biased items from their tests.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, stereotype threat has a large body of evidence to support that it impacts performance for women and men, lower SES people and ethnic minorities across many cognitive domains.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No notable trends were observed when comparing US studies with non-US studies, however only 8% of studies involved at least some participants outside of North America and Europe with only 4% using participants exclusively outside of those regions.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Considering biological factors, the role of sex hormones and monthly hormonal fluctuations in gender differences in stereotypically gendered cognitive domains is clear.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Studies relating to attitudes were roughly split between gender (11) and ethnicity (12).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A further critique of this literature comes from the fact that 56% of studies were conducted in the US, which may limit cross-cultural generalisability.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, there is little to explain potential overprediction, and little to address potential bias in criteria used.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Some of the criterion validation studies reported provide a sound account that CATs do not underpredict performance of women or minorities and may overpredict it.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In all, 23 papers examined attitudinal causes for group differences”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Latent trait and measurement invariance findings are inconsistent and there is sizable disagreement between studies.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Maertz et al. (2005) found self-efficacy differences between genders but not ethnicities.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not checked — no full text of the sourcecites Maertz et al. (2005) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Parental and individual education, income, immigration status, and language proficiency have been found to relate to ethnic and SES differences, whilst gendered play and contraceptive use and type has been found to be related to gender differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
In the paper
“Stenlund et al. (2017) did not find gender differences in motivation.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
What the sources say
- Not matchedcites Stenlund et al. (2017)
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Many of the factors explored have a body of compelling evidence to support their contribution, both those related to substantive accounts and methodological/contextual accounts of group differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, there have been challenges in replicating studies in high stakes testing, which may suggest the nature of the experimental approach could be contributing to the magnitude of effects seen.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“They also found evidence suggesting culture-free, nonverbal ability tests still showed modest group differences and that attempts to address the confound of ethnicity vs culture by building tests with Black cultural components found little success in overcoming differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Hough et al., 2001 source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Overall, ethnic differences in mean intelligence scores are the largest, SES are modest, and gender differences are not typically seen in g but are found on specific cognitive domains.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Cognitive Ability and Mean Score Group Differences
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Paper selection, data extraction and data synthesis were completed by the first author.These processes were reviewed by the second and third authors.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Data Extraction and Synthesis
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Neisser et al. (1996) reported that most CATs are constructed so there are no overall gender differences; however, differences do exist within some cognitive domains, such as in spatial and numerical domains where men typically perform better, and in verbal domains where women typically perform better.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Cognitive Ability and Mean Score Group Differences
What the sources say
- Not checked — no full text of the sourcecites Neisser et al., 1996 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.
In the paper
“Baenninger and Newcombe (1989) found some support that gender differences in spatial ability can be explained by prior experience.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Baenninger & Newcombe, 1989 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.
In the paper
“In a review of 85 years of research into recruitment selection processes, Schmidt and Hunter (1998) concluded that scores on CATs were the strongest predictor of job performance when looking at candidates without prior experience in the role.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Cognitive Ability and Mean Score Group Differences
What the sources say
- Not checked — no full text of the sourcecites Schmidt & Hunter, 1998 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“White people were typically more motivated to do well on tests, were more likely to believe tests would influence their future relationship with the company that asked them to take them and had more positive perceptions of the criterion validity of ability tests.Women and older people have greater test anxiety.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Hough et al., 2001 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Cognitive ability tests (CATs) are used as part of selection and recruitment due to their utility in predicting workplace performance (Bertua et al., 2005;Sackett et al., 2022;Schmidt & Hunter, 1998)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Introduction
What the sources say
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“findings were sometimes inconsistent and typically suggested reducing differences in test perceptions would have, at best, a small effect on test score differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Hough et al., 2001 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Group differences in test performance lowered when motivation scores were partialled, test motivation had a moderating effect on test validity and validity perceptions positive correlated with test performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Hough et al., 2001 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Since then, whilst a sizable body of evidence has supported the effects of stereotype threat, it is recognised that it does not fully explain group mean score differences (e.g., Kirnan et al., 2009) and some research suggests effects are overstated (e.g., Finnigan & Corker, 2016).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Given the heterogeneity of studies to be reviewed, quantitative synthesis was not considered appropriate.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Inclusion criteria
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Whilst influential research has suggested these differences are not a result of test bias (e.g., Jensen, 1980)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Introduction
What the sources say
- Not matchedcites Jensen, 1980
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
In the paper
“It is reliable and efficient (Pace et al., 2012)”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Inclusion criteria
What the sources say
- Not checked — no full text of the sourcecites Pace et al., 2012 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“They concluded that cultural differences have no more than a modest impact on test score differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Hough et al., 2001 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Hunter and Hunter (1984) cited three papers showing that Black people's work performance is similar, or lower, than levels which their cognitive ability would predict.They suggested these differences are likely due to poverty and hardship.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Hunter & Hunter, 1984 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“A meta-analysis by Roth et al. (2001) looking at ethnic differences in occupational testing found support for previous findings that there is around a one standard deviation difference in mean test scores between Black and White samples, with smaller differences found when comparing Hispanic and White samples.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Cognitive Ability and Mean Score Group Differences
What the sources say
- Not checked — no full text of the sourcecites Roth et al., 2001 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Increased weighting will be given to papers evaluated as having a lower risk of bias based on MMAT evaluation and accounts with replicated findings across multiple papers.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Inclusion criteria
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“The current review was conducted according to the systematic review protocols described in the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P; Moher et al., 2015).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Method
What the sources say
- Not matchedcites Moher et al., 2015
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The review was not pre-registered with PROSPERO or similar.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Method
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Little research had been done in occupational contexts and whilst some studies demonstrated potential in addressing group differences, effects were modest and inconsistent.Findings did seem to be more consistent where test takers were less familiar with tests.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Factors Relating to Mean Score Demographic Group Differences
What the sources say
- Not checked — no full text of the sourcecites Hough et al., 2001 source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly to biological research, environmental research was rarely conducted on high-stakes occupational testing.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Environmental
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“A further paper argued that replicating gender differences in spatial ability across 40 countries suggested an evolutionary basis (Silverman et al., 2007).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Biological
What the sources say
- Not checked — no full text of the sourcecites Silverman et al., 2007 source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The key concept is that if some form or forms of test bias were impacting the scores of women, ethnic minorities and lower SES people, we would expect tests to underpredict their work performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which then does not suggest whether or not differences may be the result of test bias.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“however high quality papers with large samples typically found that whilst DIF exists on test items, it does not meaningfully contribute to group mean score differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Item Bias
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“and 94% of research involved student samples”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“and many studies were conducted on very large, occupational samples.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“perhaps suggesting the relationship between strategy use and performance is best understood through other factors such as stereotype threat.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Higher scoring MMAT papers found effects as did lower scoring papers”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“though research typically benefitted from large sample sizes.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Criterion validation studies benefited from mostly being based on high-stakes testing.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“these papers were considered to meet inclusion criteria as differences in criterion validity could support the relevance of methodological and contextual factors.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Criterion Validity
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Whilst sample sizes were typically larger”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Latent Trait/Measurement Invariance
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“suggesting this body of work is methodologically weaker than others.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Behaviour in Test
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“MMAT scores were amongst the lowest”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Anxiety
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Recruiting using specific domains that are not related to job performance may inadvertently introduce adverse impact.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Anxiety differences are likely better explained in terms of stereotype threat; however, it is notable that women typically display higher trait anxiety and report greater test anxiety.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The application of the inclusion and exclusion criteria, as well as data extraction and synthesis were all completed by one person.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Limitations
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Students have pursued education and so are potentially more likely to be identified with cognitive domains, therefore stereotype threat research conducted on students may find the effects are larger than on working population samples.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By extension, it would follow that men might perform worse than women in verbal tasks if they spend less time as children in verbal play.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This makes it incredibly difficult to tease apart potentially confounding accounts.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is possible that effective test strategies are developed through experience in the given domain, and that greater exposure to test content and more elaborated instructions in selection testing could reduce group differences without sacrificing validity.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Taken together, whilst the current review finds little to suggest ethnic or SES differences have specified biological causes, there are irrefutable gender implications for the use of CATs in selection contexts.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These may be biological, but it is also possible that sex hormones influence gender identification, which in turn leads to increases in stereotype threat in specific domains.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, there have been challenges in replicating studies in high stakes testing, which may suggest the nature of the experimental approach could be contributing to the magnitude of effects seen.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Also, forward citation searching was not as expansive as it could have been, which could have led to insightful research not being identified.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Limitations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, a threat-related account becomes less relevant when considering developmental hormone levels on subsequent cognitive performance.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Group differences in CAT scores may not be the same as those seen in occupational samples and the extent to which different accounts explain those differences may vary.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are also likely important differences in activities performed in child play and adult leisure time by different ethnic and SES groups, which could contribute to domain experience and group differences.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“For example, anxious arousal has been found to be a mechanism for stereotype threat (Ben-Zeev et al., 2005) and yet no papers identified in this review investigating test anxiety examined stereotype threat.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not matchedcites Ben-Zeev et al., 2005
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Not matched
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As such, they are not representative of the wider working population and so may bias conclusions.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: University students are typically young, educated and more likely to be from higher SES backgrounds.
In the paper
“Shewach et al. (2019)also commented that student samples typically used in this research have vastly different motivation for completing tests as part of research, often for course credit where performance is irrelevant, when compared with people taking CATs as part of a recruitment process who are motivated by the potential for a new role where performance is important.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not checked — no full text of the sourcecites Shewach et al. (2019) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Meaningful gender differences are seldom found on g, but where they are, there is a possible that anxiety is playing a part.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“As an example, Shewach et al. (2019) stated that stereotype threat effects are typically more pronounced in individuals who are domain-identified, that is, performance on the domain is important to them.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
What the sources say
- Not checked — no full text of the sourcecites Shewach et al. (2019) source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Brain activation differences are compelling, but again may be related to non-biological factors (e.g., Jaušovec & Jaušovec, 2012).”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
What the sources say
- Not checked — no full text of the sourcecites Jaušovec & Jaušovec, 2012 source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Papers scored the highest on the MMAT on average”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Attitudes
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Gaps in the literature, overuse of student samples, a lack of research on high stakes testing and an overreliance on Western samples all limit our ability to understand the causes of group differences in a recruitment context.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These are very important considerations for occupational testing and, as with the biological differences above, suggest that there is clear evidence that changes to test design and implementation will not eliminate group mean score differences in CATs, only potentially reduce them.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Discussion
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Whilst the other authors reviewed this work, this could potentially have introduced bias.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Limitations
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The authors state this as an assumption.
For this review, participants' identified gender and the biological sex they were assigned at birth are assumed to be the same.
“No papers in this review discuss differences between participants' identified genders and the biological sex they were assigned at birth, and so these will be assumed to be the same.”
The authors state this as an assumption.
A lack of studies investigating a factor for a demographic group indicates a gap in the literature, not that the factor is irrelevant to that group.
“If for example several compelling studies were found supporting a particular factor's contribution to gender score differences on CATs, the lack of studies investigating ethnicity would not indicate that this factor is not relevant, only that there is a gap in the literature.”
The paper relies on this without support.
The workplace is not experienced in the same way by different groups of people.
“These studies are clearly very important, but even when looking at objective criteria, the workplace is not experienced in the same way by different groups of people.”
The paper relies on this without support.
University students are typically young, educated and more likely to be from higher SES backgrounds.
“University students are typically young, educated and more likely to be from higher SES backgrounds.”
The paper relies on this without support.
CATs are highly predictive of work success.
“Whist CATs are highly predictive of work success, their use will result in differing selection rates for groups that exhibit mean score differences.”
The paper relies on this without support.
Recruitment tests are high stakes, applicants are aware they are being evaluated, and they are aware of the competitive context.
“Recruitment tests are high stakes, applicants are aware they are being evaluated, and they are aware of the competitive context.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Implications
“no papers identified in this review investigating test anxiety examined stereotype threat.”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Multiple Factor Research
“2 Anxiety, Attitudes, Stereotype Threat”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Conclusion
“Environmental 15 93% 7% 7%”
§ Factors Related to Mean Score Group Differences on Cognitive Ability Tests: A Systematic Review > Inclusion criteria
“a variable labelled 'Impact' was extracted stating whether or not the article found that the factor may have more than a negligible impact on group test scores (yes, no, maybe).”
Citations
Cites
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Stereotype threat and the intellectual test performance of African Americans.
Steele, C. M., & Aronson, J. (1995) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.69.5.797 · Cited by 9,020
"General Intelligence," Objectively Determined and Measured
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412107 · Cited by 6,207
The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers
Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.-P., Griffiths, F., Nicolau, B., O’Cathain, A., et al. (2018) · Education for Information · doi:10.3233/efi-180221 · Cited by 4,737
Schmidt, F. L., & Hunter, J. E. (1998) · Psychological Bulletin · doi:10.1037//0033-2909.124.2.262 · Cited by 4,180
Human Cognitive Abilities: A Survey of Factor-Analytic Studies
Carroll, J. B. (1993) · doi:10.1017/cbo9780511571312 · Cited by 4,121
Stereotype Threat and Women's Math Performance
Spencer, S. J., Steele, C. M., & Quinn, D. M. (1999) · Journal of Experimental Social Psychology · doi:10.1006/jesp.1998.1373 · Cited by 3,435
Intelligence: Knowns and unknowns.
Neisser, U., Boodoo, G., Bouchard, T. J., Boykin, A. W., Brody, N., Ceci, S. J., Halpern, D. F., Loehlin, J. C., Perloff, R., Sternberg, R. J., et al. (1996) · American Psychologist · doi:10.1037/0003-066x.51.2.77 · Cited by 2,913
Validity and utility of alternative predictors of job performance.
Hunter, J. E., & Hunter, R. F. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.96.1.72 · Cited by 2,107
Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., Rodgers, M., Britten, N., Roen, K., & Duffy, S. (2006) · Lancaster University · doi:10.13140/2.1.1018.4643 · Cited by 2,070
Research works printed but not linked: 4
- Jensen, A. R. (1980). Bias in Mental Testing. Free Press.
- * McCormick, C. M., & Teillon, S. M. (2001). Menstrual cycle variation in spatial ability: Relation to salivary cortisol levels. Hormones and Behavior, 39(1), 29–38. https://doi.org/10.1207/s15327043hup1603_3 · DOI
- *Penner, A. M., & Willer, R. (2011). Stigma and glucose levels: Testing ego depletion and arousal explanations of stereotype threat effects. Current Research in Social Psychology, 16(3), n3.
- Thurstone, L. L. (1938). Primary mental abilities. University of Chicago Press.
Metadata
- Authors
- Cuppello, S., Zibarras, L. D., & Corr, P. J.
- Affiliations
- City St George's, University of London (GB)
- Corresponding author
- Cuppello, Stephen
- Venue
- Trends in Psychology
- Published
- 2025-11-11
- ISSN
- 2358-1883 (ISSN-L)
- OpenAlex
- W4416115091
- Semantic Scholar
- 7ea697cd0a92d42df34ce2d970d7a8ed8e5e0e42
- Cited by
- 0
- References
- 258
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Trends in Psychology (publishedVersion, cc-by)
- City Research Online (City University London) (acceptedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Sociology and Political Science › Social and Intergroup Psychology
- Keywords
- cognitive ability tests, group differences, stereotype threat, differential item functioning, measurement invariance, self-confidence, test anxiety, test administration, personnel selection, gender differences, ethnicity, socioeconomic status, biological and environmental factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
