Cross-national harmonization of neurocognitive assessment across five sites in a global study.

May combine two claims.

Despite extensive prospective harmonization, the Netherlands site deviated from other sites on four tasks and the India site deviated on one task.

“Nevertheless, despite our extensive prospective harmonization steps, task score performance deviated from the other sites in the Netherlands site (on four tasks) and the India site (on one task).”

• AssumptionRests on an assumption: The neurocognitive tasks measure the same cognitive constructs across all five sites despite translation and cultural differences.

May combine two claims.

After controlling for sociodemographic characteristics, the Netherlands was the only site with significantly higher ToL performance compared to the others, which did not differ from one another.

“After controlling for sociodemographic characteristics, as in the N-back, the Netherlands was the only site with significantly higher performance on the ToL compared to the others, which did not differ from one another.”

May combine two claims.

No significant associations of RA task scores with any demographic variable or interactions with site were detected at alpha 0.05.

“No significant associations of RA task scores with any demographic variable or interactions of these associations with the site were detected at an alpha value of 0.05 (Table 3).”

May combine two claims.

For the ITC variance differences, the USA site had the highest variability and South Africa the lowest.

“For both, the USA site presented the highest variability, whereas South Africa presented the lowest (see Table S6 for more detail).”

May combine two claims.

In the N-back, the USA also had higher accuracy than India, while other sites did not differ from one another.

“The USA also had higher accuracy than India, while the other sites did not differ from one another (Table 2).”

May combine two claims.

No other demographic variable nor significant site interactions were detected for ITC.

“no other demographic variable nor significant site interactions were detected.”

May combine two claims.

For the N-back difficulty slope, the USA slope was significantly less steeply declining than Brazil, India, and South Africa, while other sites did not differ.

“Furthermore, the USA slope was found to be significantly less steeply declining than Brazil, India, and South Africa, while the other sites did not differ from one another (Table 2, Figure S2).”

May combine two claims.

India had a lower RA log α value (−0.65 vs −0.33 overall) and nearly a full standard deviation (SD=19) lower average percentage of risky choices (32% in India vs 47% overall).

“Specifically, India was lower in its log α value (−0.65 in India vs. −0.33 overall) and nearly a full standard deviation (SD = 19) lower in its average percentage of risky choices across trials (32% in India vs. 47% overall).”

May combine two claims.

In the ToL, between-site differences emerged for difficult loads 3 and 5 (p=0.023 and 0.003), total percentage correct (partial η2=0.070, p=0.004), and difficulty slope (F=5.58, p<0.001).

“Between site differences emerged for difficult loads 3 and 5 (p-values = 0.023 and 0.003, respectively), for total percentage of correct responses (partial η2 = 0.070, p-value = 0.004), and for the difficulty slope (F-value = 5.58, p-value < 0.001).”

May combine two claims.

In the ToL, there were no associations with education, but SES was associated with the majority of task variables.

“There were no associations with education, but socioeconomic status (WAMI) was associated with the majority of variables from this task (Table 3).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The average age of the sample was 30 years, ranging from 26.9 years in India to 32.9 years in the Netherlands.

“The average age of the sample was 30 years, ranging across sites from 26.9 years in India to 32.9 years in the Netherlands.”

May combine two claims.

In the ToL, higher SES was associated with higher total percentage correct (β=18.781, p=0.004) and a higher difficulty slope (β=11.503, p<0.001), indicating a less steep decline in accuracy.

“The total percentage of correct responses (β = 18.781, p-value = 0.004) and the difficulty slope were higher (β = 11.503, p-value < 0.001), indicating a less-steep decline in accuracy for respondents with higher SES.”

May combine two claims.

In the N-back, significant site differences were observed in the more difficult conditions N=2 and N=3 (p<0.001 and 0.004) and in total percentage correct (partial η2=0.114, p<0.001).

“In the more difficult conditions (N = 2 and N = 3) significant differences were observed (p-values = <0.001 and 0.004) as well as in the total percentage of correct responses (partial η2 = 0.114, p-value < 0.001)”

May combine two claims.

For the ITC, site variance differed for log k (F=3.82, p=0.005) and percentage of delayed choices (F=2.46, p=0.046).

“the ITC task showed differences for the log k value (F=3.82, p=0.005) and the percentage of delayed choices (F=2.46, p=0.046).”

May combine two claims.

Both main outcomes of the RA task, log α and percentage of risky choices, were significantly different between sites with medium effect sizes (log α partial η2=.09, p=0.001; percentage risky choices partial η2=0.11, p<0.001) after adjusting for demographic variables.

“Both main outcomes of the RA task, the log α value and the percentage of risky choices, were significantly different between sites with medium effect sizes (log α partial η2 = .09, p = 0.001; percentage of risky choices partial η2 = 0.11, p < 0.001) after adjusting for demographic variables (Table 2, Figure S6).”

May combine two claims.

Several associations were found between task performance and sociodemographic variables across sites, and mainly in the ToL, the association with age and SES differed by site, reflected by an interaction effect.

“Several associations were found between task performance and sociodemographic variables across sites and, mainly in the ToL, the association with age and SES differed by site, reflected by an interaction effect.”

May combine two claims.

Years of education were highest in Brazil (17.1 years) and lowest in South Africa (14.9 years).

“Across the sites, the number of years of education was highest in Brazil (17.1 years) and lowest in South Africa (14.9 years).”

May combine two claims.

For the ToL difficulty slope, the Netherlands had a less steep decline (−4.15) than Brazil, India, and South Africa, but not the USA; the USA also had a less steep decline than Brazil and India.

“Regarding the difficulty slope, the Netherlands presented a less-steep decline in performance as difficulty increased (−4.15) which was significantly different from Brazil, India, and South Africa sites, but not the USA site; the USA site also had a significantly less-steep decline than the Brazil and India sites (Table 2).”

May combine two claims.

Significant site differences were found for SST go RT (partial η2=0.068, p=0.007), with the Netherlands and USA exhibiting the fastest times.

“However, we found significant site differences for go RT (partial η2 = 0.068, p-value = 0.007), with the Netherlands and the USA exhibiting the fastest times.”

May combine two claims.

There was no overall significant effect of age on the ToL difficulty slope (β=−0.047, p=0.109), but there was an age-by-site interaction for the difficulty slope (p=0.002).

“While there was no overall significant effect of age on the difficulty slope (β = −0.047, p-value = 0.109), there was an age-by-site interaction for the difficulty slope (p-value = 0.002) (Table 3).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Among the 221 participants, 16 had a past lifetime diagnosis of MDD (7.2%) and 11 had a past lifetime diagnosis of an anxiety disorder (5.0%), totaling 10.0% of the sample.

“Among the 221 participants, 16 had a past lifetime diagnosis of MDD (7.2%), and 11 had a past lifetime diagnosis of an anxiety disorder (5.0%), totaling 10.0% of the sample.”

May combine two claims.

Post-hoc comparisons showed the N-back total percentage correct site effect was mostly driven by the Netherlands having higher accuracy (87%) than Brazil (78%, p<0.001), India (76%, p<0.001), South Africa (80%, p=0.003), and the USA (84%, p=0.031).

“Post-hoc comparisons showed that the site effect on the total percentage of correct responses was mostly driven by the Netherlands site which had higher accuracy (87%) than Brazil (78%, p-value < 0.001), India (76%, p-value < 0.001), South Africa (80%, p-value = 0.003), and the USA (84%, p-value = 0.031).”

May combine two claims.

The same N-back pattern was observed for the difficulty slope, with the USA site also presenting better performance than the other sites.

“The same pattern was observed for the difficulty slope, with the USA site also presenting better performance than the other sites (Table 2).”

May combine two claims.

Re-running ToL analyses without the Netherlands led to small effect size differences among the remaining four sites (all partial η2<0.014) in all conditions and total percent correct, with only the difficulty slope still differing across sites.

“Re-running the analyses without the Netherlands data, and making comparisons across the remaining 4 sites, led to an overall small effect size difference (all partial η2 < 0.014) in all conditions and for total percent correct with only the difficulty slope still presenting differences across sites (Table 2).”

May combine two claims.

Dropping the India site led to overall small effect sizes for RA (partial η2=0.05 for log value and 0.06 for percentage risky choices) but still significant (p=0.039 for log α, p=0.011 for percentage risky choices).

“Dropping the India site and making comparisons across the other four sites led to overall small effect sizes (partial η2 = 0.05 for log value and = 0.06 for the percentage of risky choices) but still significant (p = 0.039 for log α value, p = 0.011 for percentage of risky choices) (Table 2).”

May combine two claims.

Apart from a significant association between go-RT and age (older subjects were slower to respond), there were no associations with demographics or significant interactions by site for the SST.

“Apart from a significant association between go-RT and age (i.e., older subjects were slower to respond), there were no associations with any demographics or significant interactions by site (Table 3).”

May combine two claims.

For the N-back difficulty slope, the Netherlands had a significantly less steep declining slope (−10.9) than Brazil (−17.3, p<0.001), India (−18.2, p<0.001), and South Africa (−16.0, p=0.005), but not the USA (−12.3, p=0.448).

“Similar site-specific effects were found for the difficulty slope such that the Netherlands had a significantly less steep declining slope (−10.9) than Brazil (−17.3, p < 0.001), India (−18.2, p < 0.001), and South Africa (−16.0, p = 0.005), but not the USA (−12.3, p = 0.448).”

May combine two claims.

Both ToL total percentage correct and difficulty slope had an interaction by site (p=0.036 and 0.018, respectively).

“Both variables also presented an interaction by site (p-values = 0.036 and 0.018, respectively)”

May combine two claims.

ToL total percentage correct was negatively associated with age (β=−0.215, p=0.001) and had an age-by-site interaction (p=0.008).

“Overall task performance (total % correct) was negatively associated with age (β = −0.215, p-value = 0.001) and presented an interaction effect by site (p-value = 0.008)”

May combine two claims.

In the ToL, males performed more accurately than females for total percentage correct (β=2.68 percentage points higher in males, p=0.010) and showed less decline in accuracy as difficulty increased (β=1.04, p=0.031).

“When observing the association between sociodemographic and ToL performance across all sites, males performed more accurately than females for the total percentage of correct responses (β = 2.68 percentage points higher in males, p-value = 0.010) and showed less decline in accuracy as difficulty increased (β = 1.04, p-value = 0.031).”

May combine two claims.

The ToL difference was present for total percentage correct (also levels 3 and 5) and the difficulty slope, with the USA also presenting better performance on the difficulty slope.

“The difference was present for the total percentage of correct responses (also levels 3 and 5), and the difficulty slope (in this last variable, the USA site also presented better performance when compared to the other sites).”

May combine two claims.

Post-hoc ITC comparisons found differences were driven by the Netherlands having a significantly lower log k value (−5.47 vs −4.55 overall) and a higher percentage of choices of the larger delayed reward (54% vs 43% overall) than all other sites.

“Post-hoc comparisons found that site differences were driven by the Netherlands site having a significantly lower log k value (−5.47 Netherlands vs. −4.55 overall) and a higher percentage of choices of the larger, delayed reward (54% Netherlands vs. 43% overall) than all other sites.”

May combine two claims.

Post-hoc inspection indicated the India site significantly differed from the other sites and Brazil had more risk aversion than the Netherlands for both RA measures.

“Post-hoc inspection indicated that the India site significantly differed from the other sites and that the Brazil site had more risk aversion than the Netherlands site for both measures.”

May combine two claims.

In the N-back, age (β=−0.242, p=0.015) and education (β=0.729, p=0.037) were associated with total percentage correct, and SES was associated with the difficulty slope (β=18.032, p=0.015).

“Associations between the task variables and sociodemographic covariates revealed that age (β = −0.242, p-value = 0.015) and education (β = 0.729, p-value = 0.037) were associated with the total percentage of correct responses, whereas SES was associated with the difficulty slope (β = 18.032, p-value = 0.015).”

May combine two claims.

Consistent with previous research, the study found a significant decline in N-back task performance at higher difficulty levels and total score as one ages.

“Consistent with previous research, we found a significant decline in task performance at higher difficulty levels (and the total score), as one ages (Dobbs & Rule, 1989; Bopp & Verhaeghen, 2020).”

• cites Dobbs & Rule, 1989Not checked yet

  Not checked in the free run.

• cites Bopp & Verhaeghen, 2020Not checked yet

  Not checked in the free run.

May combine two claims.

After adjusting for covariates, task performance significantly varied across sites, and these differences were attenuated by dropping the Netherlands data.

“After adjusting for covariates, task performance significantly varied across sites, which was attenuated by dropping the Netherlands data from the analysis.”

May combine two claims.

Both ITC task measures were significantly associated with SES as measured by WAMI, indicating higher SES was associated with lower discount rates (β=−3.047, p=0.016 for log k; β=38.642, p=0.016 for % delayed choices).

“Apart from a significant association of both ITC task measures with socioeconomic status as measured by the WAMI, indicating that higher SES was associated with lower discount rates (β = −3.047, p-value = 0.016 for log k value and β = 38.642, p-value = 0.016 for % of delayed choices)”

• Also stated in: § DISCUSSION · plain

  “Across sites, socioeconomic status was associated with temporal discounting, such that higher-SES individuals had lower discount rates.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The proportion of females ranged from 38.2% to 64.6% across sites, with no statistical differences between sites.

“The proportion of females ranged from 38.2% to 64.6% (Table 1), and there were no statistical differences between sites.”

May combine two claims.

Post-hoc analyses showed Brazil had a strong association between age and a more steeply declining ToL difficulty slope (decrease in accuracy between difficulty levels became steeper by −0.3209 percent per difficulty level and year of age, p<0.001), while the age-slope association in India, Netherlands, South Africa, and USA was not different from zero (p=0.677, 0.262, 0.255, 0.389).

“Post-hoc analyses showed that the Brazil site showed a strong association between age and more steeply declining slope for higher task difficulty (the decrease in accuracy between difficulty levels becomes steeper by −0.3209 percent per difficulty level and year of age, p-value < 0.001), while the association of age with the difficulty slope in all other sites was not different from zero (India: p”

May combine two claims.

Across all sites, significant associations were found between ToL performance and sex, age, and SES, with younger and higher-SES males performing better.

“Across all sites, significant associations were found between performance on the ToL and sex, age, and SES. Specifically, younger and higher-SES males performed better relative to other participants.”

May combine two claims.

The sample included healthy adults from all five sites, with the largest number in Brazil (n=51) and the smallest in India (n=34).

“The sample included healthy adults from all five sites, with the largest number in Brazil (n=51) and the lowest in India (n=34; Table 1).”

May combine two claims.

Post-hoc ToL comparisons showed the Netherlands had a higher total percentage correct (92%) than Brazil (86%), India (87%), and South Africa (88%), but not the USA (89%).

“Post-hoc comparisons between sites revealed that the Netherlands site had a higher total percentage of correct responses (92%) than Brazil (86%), India (87%), and South Africa (88%), but not the USA (89%).”

May combine two claims.

Post-hoc tests for the ToL age-by-site interaction indicated Brazil showed a higher decline in total performance with age than India (p=0.046), the Netherlands (p<0.001), and South Africa (p=0.004), but not the USA (p=0.128).

“with post-hoc tests indicating that the Brazil site showed a higher decline in total performance than India (p = 0.046), the Netherlands (p < 0.001), South Africa (p = 0.004), but not USA (p = 0.128), as participants got older (Figure 2A).”

May combine two claims.

More participants met exclusion criteria on the N-Back task, with data available for 82.4% of the sample.

“More participants met our exclusion criteria on the N-Back task (data available for 82.4%)”

May combine two claims.

Age was negatively associated with outcomes on the cold tasks (N-back, ToL and SST) but did not affect performance on the choice tasks.

“age was negatively associated with outcomes in the “cold” tasks (N-back, ToL and SST), while it did not affect the performance on the choices’ tasks.”

May combine two claims.

Site interactions with age and SES were found for several ToL measures, including total percentage correct and difficulty slope, indicating Brazil had the strongest association between older age and decline in performance.

“Of note, site interactions with age and SES were found for several measures of this task, including the total percentage of correct responses and difficulty slope, indicating that the Brazil site had the strongest association between older age and decline in performance (Figure 2).”

May combine two claims.

Participants in India were significantly more risk-averse compared to participants at all other sites, who did not differ from each other.

“Our first key finding was that participants in the India site were significantly more risk-averse compared to participants at all other sites, who did not differ from each other.”

May combine two claims.

Variability in both ITC measures differed across sites, being greatest in the USA and lowest in South Africa.

“we observed variability differences across sites in both ITC measures (with the most variability in the USA and the lowest in South Africa)”

May combine two claims.

After adjusting for demographic covariates, only one site differed on each task: the Netherlands on the N-back, ToL, SST and ITC tasks, and India on the RA task.

“the fact that we encountered differences in only one site for each task evaluated in the study after adjusting for demographic covariates (Netherlands was different for N-back, ToL, SST, and ITC tasks, whereas India was different in the RA task)”

• Also stated in: § CONCLUSION · plain

  “only the Netherlands (in variables of the N-back, ToL, SST and ITC task) and India (in both variables measures for the RA task) were discrepant after adjusting for demographic covariates.”

Classified as probably not a substantial claim.

Because currency differed across sites, a purchasing power parity (PPP) conversion formula was calculated to ensure monetary rewards were appropriately rewarding across currencies and countries.

“As the currency of monetary reward differed across the sites, a suitable conversion formula based on purchasing power parities (PPP) was calculated to ensure the monetary rewards were appropriately rewarding across currencies and countries”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The stringent (prospective) harmonization approach is considered ideal and involves using identical methods and instruments across studies or sites.

“The stringent (prospective) approach is considered ideal and consists of identical methods and instruments across studies/sites.”

• cites Fortier et al., 2011Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A neurocognitive working group with researchers from each site met monthly by conference call to ensure consistency of the harmonization process and to troubleshoot issues.

“A neurocognitive working group composed of researchers involved in neurocognitive assessments at each site met monthly over conference calls to ensure consistency of the harmonization process and to troubleshoot any issues.”

Classified as probably not a substantial claim. May combine two claims.

Independent regional language experts at each site performed forward and backward translation of task instructions, and a native English speaker from the USA site compared the original and back-translated English versions for consistency.

“Independent regional language experts at each site carried out forward and backward translation of task instructions. For consistency, each site’s original and back-translated English versions were compared by a native English speaker from the USA site.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Quality checks were performed at regular intervals to ensure data quality at each site.

“Quality checks were performed at regular intervals to ensure data quality at each site.”

Classified as probably not a substantial claim.

Prospective harmonization methods were used to ensure that task implementation and processing of derived measures were procedurally similar across sites.

“Prospective harmonization methods were employed to ensure procedural similarity of task implementation and processing of derived task measures across sites.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study's design and analysis were not pre-registered.

“This paper’s design and analysis were not pre-registered.”

May combine two claims.

Quality control was performed to ensure participants were not responding randomly in the ITC and RA tasks by verifying that responses were a function of task variables, and data from participants with a Tjur coefficient below 0.20 were discarded.

“We performed quality control to ensure that participants were not responding randomly in the ITC and RA tasks by verifying that their responses were a function of task variables that should be relevant to the choice (as in Pehlivanova et al., 2018).”

• Also stated in: § Neurocognitive evaluation · plain

  “We assessed the goodness-of-fit of these regression models using the coefficient of discrimination (Tjur, 2009) and discarded data from participants who had a Tjur coefficient value < 0.20.”

• cites Pehlivanova et al., 2018Not checked yet

  Not checked in the free run.

• cites Tjur, 2009Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

To ensure uniformity in data collection, tasks were piloted with two volunteers at each site in their respective languages, and screenshots and videos of the tasks were checked during these pilot sessions to standardize stimuli presentation.

“Subsequently, to ensure uniformity in data collection, the tasks were piloted with two volunteers at each site in their respective languages. To guarantee the standardization of stimuli presentation across sites, screenshots and videos of the tasks were checked during these pilot sessions.”

May combine two claims.

The STAN study found comparable task performance between India and USA sites, indicating cross-national reliability of the STAN for that sample.

“The performance across tasks was comparable between the two sites, indicating cross-national reliability of the STAN, at least with respect to this sample.”

• cites Cherkil et al., 2012Not checked yet

  Not checked in the free run.

May combine two claims.

The study's sample of 221 participants across five sites provides at least 80% power to detect a partial eta-squared of 0.0523 or greater at alpha .05, so it has adequate power for at least medium effect sizes.

“The sample size of 221 participants across 5 sites provides at least 80% power to detect differences with a partial eta squared of 0.0523 or greater at the .05 level. Hence, we have adequate power to detect at least medium effect sizes.”

Classified as probably not a substantial claim.

Examiners at all sites were required to familiarize themselves with the SOP and the tasks to ensure uniform examiner training.

“Examiners at all sites were required to familiarize themselves with the SOP and the tasks to ensure uniformity in the training of the examiners.”

Classified as probably not a substantial claim.

A standard operating procedures (SOP) manual was created as a reference guide for all sites to ensure uniformity in task administration.

“A manual of standard operating procedures (SOP) was created to serve as a reference guide for all sites to ensure uniformity in task administration.”

May combine two claims.

In the CNTB study, significant differences between ethnic groups were found for most tests, mainly explained by educational attainment, which accounted for 15% of the total variance.

“Significant differences between ethnic groups were found for most tests, mainly attributable to differences in educational attainment, explaining 15% of the total variance.”

• cites Nielsen et al., 2018Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Prospective data harmonization may help researchers combine individual participant data from various sites more effectively in the era of replication studies.

“In the era of replication studies, prospective data harmonization may help researchers combine individual participant data from various sites more effectively (Griffith et al., 2013).”

• cites Griffith et al., 2013Not checked yet

  Not checked in the free run.

May combine two claims.

Both the STAN and CNTB studies highlight the effect of language and educational attainment on neurocognitive performance.

“Both studies highlight the effect of language and educational attainment on neurocognitive performance.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

During the COVID-19 pandemic, some testing conditions were modified for safety, but the core procedures remained the same.

“During the COVID-19 pandemic, some modifications to the testing conditions were made to ensure all safety protocols were followed, but the core procedures remained the same.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Cognitive performance, especially fluid intelligence, is expected to decrease with age.

“cognitive performance, especially skills related to fluid intelligence, is expected to decrease with age (Cattell, 1963; Baltes et al., 1999)”

• cites Cattell, 1963Not checked yet

  Not checked in the free run.

• cites Baltes et al., 1999Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In models testing site and sociodemographic effects, each sociodemographic association was controlled for the remaining sociodemographic variables.

“Importantly, these models testing sites and sociodemographic variables were mutually controlled (i.e., the association with age was controlled by sex, years of education, and SES; the association with sex was controlled by age, years of education, and SES, and so on).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Previous studies found better ToL performance by males, including on higher difficulty items.

“These findings are similar to those of previous studies that found better performance by males, including on items with higher difficulty levels (Sousa et al., 2001; Mataix-Cols & Bartrés-Faz, 2002).”

• cites Sousa et al., 2001Not checked yet

  Not checked in the free run.

• cites Mataix-Cols & Bartrés-Faz, 2002Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Some cross-national studies have shown differential memory performance in Western versus Eastern samples, driven by differential focus on salient objects versus context.

“Although some cross-national studies have shown differential memory performance in Western (Americans, Canadians, or Western Europeans) versus Eastern (Chinese, Japanese, or Koreans) samples, driven by a differential focus on salient objects vs. context (Park and Huang 2010; Leger and Gutchess 2021)”

• cites Park and Huang 2010Not checked yet

  Not checked in the free run.

• cites Leger and Gutchess 2021Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Evaluators at all sites completed online ethics training and were certified.

“Evaluators from all sites completed online training on ethics in research and were certified after completing this course.”

Classified as probably not a substantial claim.

Fluid intelligence is known to correlate with planning ability.

“As fluid intelligence is known to correlate with planning ability (D’Antuano et al. 2016)”

• cites D’Antuano et al. 2016Not checked yet

  Not checked in the free run.

May combine two claims.

The analyses did not correct for multiple comparisons and used an alpha level of 0.05 to maximize sensitivity to site differences.

“Given that we wanted our analyses to be as sensitive as possible to site differences, we opted not to correct for multiple comparisons; therefore, the alpha level adopted in this study was 0.05.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Written informed consent was obtained from all study participants.

“Also, written informed consent was obtained from all study participants.”

May combine two claims.

More difficult N-back levels exert greater load on working memory and produce more errors.

“with more difficult levels exerting greater load on working memory, and more errors.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Numerous previous studies find humans tend to slow with age in processing speed and motor reactions.

“Numerous previous studies that have examined processing speed and motor reactions over the lifespan find that humans tend to slow with age (Salthouse, 1996, Nilsson et al., 2014; Papp et al., 2014).”

• cites Salthouse, 1996Not checked yet

  Not checked in the free run.

• cites Nilsson et al., 2014Not checked yet

  Not checked in the free run.

• cites Papp et al., 2014Not checked yet

  Not checked in the free run.

May combine two claims.

This is the first study to use multiple prospectively harmonized neurocognitive tasks to examine cognition across these five countries, and the first to do so in strictly evaluated healthy samples from Brazil, India, the Netherlands, South Africa and the USA.

“To our knowledge, this is the first study of its kind to use multiple prospectively harmonized neurocognitive tasks to examine similarities and differences in cognition across these five countries, distributed on different continents.”

• Also stated in: § CONCLUSION · hedged

  “To our knowledge, this is the first study to harmonize these five neurocognitive tasks across sites and is certainly the first to do so in samples of healthy participants strictly evaluated (in terms of psychiatric symptoms) from Brazil, India, the Netherlands, South Africa, and the USA.”

May combine two claims.

The age range of 18-50 and recruitment restricted to participants with IQ>80 and no current or past psychiatric disorder preclude generalizing the findings to older adults or to participants with borderline IQ.

“the age range of study participants (18–50) and the fact that we only recruited participants with IQ>80 without a current or past history of psychiatric disorders precludes us from generalizing our findings to older adults or participants with borderline IQ.”

May combine two claims.

Because the task sequence was not counterbalanced, fatigue or other order effects could have affected the results and hampered generalization of the findings.

“all participants conducted the evaluation by performing the tasks in the same order: since we did not counterbalance the tasks sequence, it is possible that fatigue or other order effects in our sample could affect the results hampering the generalization of our findings.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A tendency toward risk aversion has been previously reported among Indian participants, particularly in rural, low-resource settings.

“A tendency toward risk aversion has been previously reported among Indian participants, particularly in rural, low-resource settings (e.g., Binswanger, 1980; Senapati, 2020)”

• cites Binswanger, 1980Not checked yet

  Not checked in the free run.

• cites Senapati, 2020Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Previous studies evaluating ToL performance across the lifespan in predominantly Western populations noted performance remains stable from late adolescence through the fourth decade of life.

“although previous studies evaluating ToL performance across the lifespan in predominantly Western populations (Kaller et al., 2016; Unterrainer et al., 2019) have noted that performance remains stable from late adolescence through the fourth decade of life.”

• cites Kaller et al., 2016Not checked yet

  Not checked in the free run.

• cites Unterrainer et al., 2019Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Even with identical instructions and standardization, cross-site differences in results could arise from external factors such as sample selection, intelligence, motivation, attention, evaluator expertise, or random factors.

“Even if we could have the exact same instructions and standardization of the tasks in different sites, different results could arrive from other external factors, like sample selection, different levels of intelligence, motivation or attention during the test, evaluators’ expertise, or other random factors related to the evaluation.”

Classified as probably not a substantial claim. May combine two claims.

Harmonizing task instructions, administration procedures and testing environment aims to give participants at all sites similar testing conditions and equivalent understanding of the instructions.

“The rationale behind the harmonization of the task instructions, administration procedures, and testing environment is to ensure that participants at all sites have similar testing conditions and have received and understood the task instructions the same way.”

May combine two claims.

The study's results associating SES with ToL performance confirm and add to existing literature that has identified an association between SES and cognitive performance.

“Finally, our results associating SES with ToL performance also confirm and add to the existing literature that has previously identified an association between SES and cognitive performance (Wu et al., 2016; Gonzalez et al., 2020; Taylor et al., 2021).”

• cites Wu et al., 2016Not checked yet

  Not checked in the free run.

• cites Gonzalez et al., 2020Not checked yet

  Not checked in the free run.

• cites Taylor et al., 2021Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Lack of generalized trust in the social environment may lead to higher discounting rates.

“Lack of generalized trust in the social environment may lead to higher discounting rates (Jachimowicz et al., 2017).”

• cites Jachimowicz et al., 2017Not checked yet

  Not checked in the free run.

May combine two claims.

The site differences in exclusion proportions do not represent a lack of task harmonization but likely reflect other subject-specific factors that differ between sites.

“Given the strict procedural standards and quality control cutoffs across sites, these differences do not represent a lack of task harmonization but instead the likely presence of other subject-specific factors that differ between sites.”

Classified as probably not a substantial claim.

Not correcting for multiple comparisons increases the chance of false positives, that is, of concluding a variable differs across sites when it does not.

“in order to increase sensitivity for finding differences across sites, we opt not to use any method for correcting for multiple comparisons. This increases the chances of finding false positives, i.e., to consider that a specific variable is different across sites when, in reality, it is not.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Cultural influences may still play a role in interpreting task performance across sites, through task familiarity, patterns of abilities, commitment to the evaluation, and cultural values.

“Nevertheless, cultural influences may still play a role in terms of task familiarity, patterns of abilities, commitment to the evaluation, and cultural values, in interpreting task performance across sites.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Ensuring that tasks measure a meaningful latent construct equivalently across groups (measurement invariance) is a crucial aspect of cross-national cognitive research, beyond comparing means and standard deviations.

“A crucial aspect for the field of cross-national investigations in cognitive measures, beyond the average and standard deviation comparison, is to ensure that the tasks have a meaningful latent construct that is being measured equivalently across all groups (measurement invariance).”

May combine two claims.

The cross-site results suggest the Indian risk-aversion tendency may not be related to SES, since SES was controlled and no demographic effects were observed for RA measures.

“but our cross-site results suggest that this tendency may not be related to SES, once this covariate was controlled in the analysis and no demographic effects were observed to relate with the RA measures.”

May combine two claims.

One important factor influencing delay discounting is trust that delayed outcomes will materialize; uncertainty about delayed rewards can make immediate payoffs preferable.

“One important factor that influences delay discounting is trust that delayed outcomes will materialize; when there is uncertainty about whether or when delayed rewards will arrive, it can be preferable to take immediate payoffs instead (McGuire & Kable, 2013; Kidd et al., 2013).”

• cites McGuire & Kable, 2013Not checked yet

  Not checked in the free run.

• cites Kidd et al., 2013Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Stereotype threat alters neurocognitive performance.

“the stereotype threat has been indicated to alter neurocognitive performance (Moritz et al., 2018)”

• cites Moritz et al., 2018Not checked yet

  Not checked in the free run.

May combine two claims.

The SSRT showed no site effects or variability differences, which may indicate consistency across countries in inhibition response.

“The fact that we did not observe site effects in the SSRT, nor in its variability, may indicate consistency across countries in this task that evaluates the inhibition response.”

Classified as probably not a substantial claim.

It is difficult to guarantee that measured covariates are reliable and reflect the same construct across sites.

“even measuring possible confounding variables that could play a role in cognitive performance, it is remarkably difficult to guarantee that those covariates are reliable and reflect the same construct across sites.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Years of education is not a direct measure of academic achievement, because repeating years of school is not captured and a measure of time does not evaluate schooling quality.

“the sociodemographic variable “years of education” is not a direct measure of academic achievement, as a participant could have repeated years of school and only a measure of time does not evaluate the quality of schooling.”