The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Psychometrically derived 60-question benchmarks: Substantial efficiencies and the possibility of human-AI comparisons
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Survey on Evaluation of Large Language Models
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3641289 · Cited by 4,254
HellaSwag: Can a Machine Really Finish Your Sentence?
Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1472 · Cited by 830
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
Mirzadeh, I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S., & Farajtabar, M. (2024) · arXiv · doi:10.48550/arxiv.2410.05229 · Cited by 714
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575
The debate over understanding in AI’s large language models
Mitchell, M., & Krakauer, D. C. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2215907120 · Cited by 340
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-Tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2025) · IEEE Transactions on Audio, Speech and Language Processing · doi:10.1109/taslpro.2025.3606231 · Cited by 136
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Luo, Y., Yang, Z., Meng, F., Li, Y., Zhou, J., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2308.08747 · Cited by 135
Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations
Miller, E. (2024) · arXiv · doi:10.48550/arxiv.2411.00640 · Cited by 9
tinyBenchmarks: evaluating LLMs with fewer examples
Polo, F. M., Weber, L., Choshen, L., Sun, Y., Xu, G., & Yurochkin, M. (2024) · arXiv · doi:10.48550/arxiv.2402.14992 · Cited by 8
Research works printed but not linked: 35
- Carretta, 2022, Correction for range restriction: Lessons from 20 research scenarios, Military Psychology, 34, 551, 10.1080/08995605.2021.2022067 · DOI
- Chalmers, 2012, Mirt: A multidimensional item response theory package for the R environment, Journal of Statistical Software, 48, 1, 10.18637/jss.v048.i06 · DOI
- Clark, 1995, Constructing validity: Basic issues in objective scale development, Psychological Assessment, 7, 309, 10.1037/1040-3590.7.3.309 · DOI · Paper page
- Clause, 1998, Parallel test form development: A procedure for alternate predictors and an example, Personnel Psychology, 51, 193, 10.1111/j.1744-6570.1998.tb00722.x · DOI
- Cronbach, 1951, Coefficient alpha and the internal structure of tests, Psychometrika, 16, 297, 10.1007/BF02310555 · DOI · Paper page
- Cureton, 1966, Corrected item-test correlations, Psychometrika, 31, 93, 10.1007/BF02289461 · DOI
- Detterman, 1989, Correlations of mental tests with each other and with cognitive variables are highest for low IQ groups, Intelligence, 13, 349, 10.1016/S0160-2896(89)80007-8 · DOI
- Flora, 2020, Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates, Advances in Methods and Practices in Psychological Science, 3, 484, 10.1177/2515245920951747 · DOI
- Frisbie, 1988, Reliability of scores from teacher-made tests, Educational Measurement: Issues and Practice, 7, 25, 10.1111/j.1745-3992.1988.tb00422.x · DOI
- Gignac, 2024, Rethinking the Dunning-Kruger effect: Negligible influence on a limited segment of the population, Intelligence, 104, 10.1016/j.intell.2024.101830 · DOI
- Gignac, 2024, Defining intelligence: Bridging the gap between human and artificial perspectives, Intelligence, 104, 10.1016/j.intell.2024.101832 · DOI
- Haley, 2006, Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank, Journal of Clinical Epidemiology, 59, 1174, 10.1016/j.jclinepi.2006.02.010 · DOI
- Harrison, 2017, Applying modern psychometric techniques to melodic discrimination testing: Item response theory, computerised adaptive testing, and automatic item generation, Scientific Reports, 7, 3618, 10.1038/s41598-017-03586-z · DOI
- Ilić, 2024, Evidence of interrelated cognitive-like capabilities in large language models: Indications of artificial general intelligence or achievement?, Intelligence, 106, 10.1016/j.intell.2024.101858 · DOI
- Jones, 2007, A history and overview of psychometrics, Vol. 26, 1
- Kristof, 1969, Estimation of true score and error variance for tests under various equivalence assumptions, Psychometrika, 34, 489, 10.1007/BF02290603 · DOI
- Levy, 1967, The correction for spurious correlation in the evaluation of short-form tests, Journal of Clinical Psychology, 23, 84, 10.1002/1097-4679(196701)23:1<84::AID-JCLP2270230123>3.0.CO;2-2 · DOI
- Levy, 1968, Short-form tests: A methodological review, Psychological Bulletin, 69, 410, 10.1037/h0025736 · DOI
- Lorenzo-Seva, 2011, The Hull method for selecting the number of common factors, Multivariate Behavioral Research, 46, 340, 10.1080/00273171.2011.564527 · DOI
- Marianti, 2023, Comparing item-total correlation and item-theta correlation in test item selection: A simulation and empirical study, Jurnal Penelitian dan Evaluasi Pendidikan, 27, 133, 10.21831/pep.v27i2.61477 · DOI
- McCloskey, 1989, Catastrophic interference in connectionist networks: The sequential learning problem, Psychology of Learning and Motivation, 24, 109, 10.1016/S0079-7421(08)60536-8 · DOI · Paper page
- McDonald, 1974, Difficulty factors in binary data, British Journal of Mathematical and Statistical Psychology, 27, 82, 10.1111/j.2044-8317.1974.tb00530.x · DOI
- McNeish, 2018, Thanks coefficient alpha, we’ll take it from here, Psychological Methods, 23, 412, 10.1037/met0000144 · DOI · Paper page
- Messick, 1995, Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning, American Psychologist, 50, 741, 10.1037/0003-066X.50.9.741 · DOI · Paper page
- Myszkowski, 2018, A snapshot of g? Binary and polytomous item-response theory investigations of the last series of the Standard Progressive Matrices (SPM-LS), Intelligence, 68, 109, 10.1016/j.intell.2018.03.010 · DOI
- Pfeiffer, 2011, A new method for estimating the variance overlap between the short and the long form of a psychological test, Educational and Psychological Measurement, 71, 380, 10.1177/0013164410386782 · DOI
- Raven, 2000, The Raven’s progressive matrices: Change and stability over culture and time, Cognitive Psychology, 41, 1, 10.1006/cogp.1999.0735 · DOI · Paper page
- Ruchensky, 2017, Examining the reliability and validity of an abbreviated Psychopathic Personality Inventory—Revised (PPI-R) in four samples, Psychological Assessment, 29, 238, 10.1037/pas0000335 · DOI
- Sakaguchi, 2021, Winogrande: An adversarial winograd schema challenge at scale, Communications of the ACM, 64, 99, 10.1145/3474381 · DOI · Paper page
- Smith, 2000, On the sins of short-form development, Psychological Assessment, 12, 102, 10.1037/1040-3590.12.1.102 · DOI
- Spiliotopoulou, 2009, Reliability reconsidered: Cronbach's alpha and paediatric assessment in occupational therapy, Australian Occupational Therapy Journal, 56, 150, 10.1111/j.1440-1630.2009.00785.x · DOI
- Swailes, 2002, The “Belbin” team role inventory: Reinterpreting reliability estimates, Journal of Managerial Psychology, 17, 529, 10.1108/02683940210439432 · DOI · Paper page
- Vaci, 2014, Restricting range restricts conclusions, Frontiers in Psychology, 5, 569, 10.3389/fpsyg.2014.00569 · DOI
- Vandenberg, 1978, Mental rotations, a group test of three-dimensional spatial visualization, Perceptual and Motor Skills, 47, 599, 10.2466/pms.1978.47.2.599 · DOI · Paper page
- Walker, 2024, General figure and face-specific closure ability: Predictors of trait-autism?, Current Psychology, 43, 1179, 10.1007/s12144-023-04351-4 · DOI
Metadata
- Authors
- Gignac, G. E., & Ilić, D.
- Venue
- Intelligence
- Publisher
- Elsevier BV
- Published
- 2025-05
- Volume
- 110
- Pages
- 101922
- ISSN
- 0160-2896
- Cited by
- 4
- References
- 9
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Psychometrically derived 60-question benchmarks: Substantial efficiencies and the possibility of human-AI comparisons. https://app.cochat.ai/papers/10.1016/j.intell.2025.101922
- The paper
- Gignac, G. E., & Ilić, D. (2025). Psychometrically derived 60-question benchmarks: Substantial efficiencies and the possibility of human-AI comparisons. Intelligence. https://doi.org/10.1016/j.intell.2025.101922
