Objective measurement of AI literacy: Development and validation of the AI competency objective scale (AICOS)
The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Markus et al. (2025) also note that many AI literacy tools rely on subjective self-assessment and may not capture actual competence.”
From AI Literacy and Ethical Capability in Business Education: A Structured Narrative Review and Competency Framework (2026) · cites this paper as Markus et al. (2025)
In the citing paper
“Their AI Competency Objective Scale responds to that gap with an objective test that includes generative AI literacy as a distinct dimension.”
From AI Literacy and Ethical Capability in Business Education: A Structured Narrative Review and Competency Framework (2026) · cites this paper as Markus et al. (2025)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Kumar, Y., Koul, A., Singla, R., & Ijaz, M. F. (2023) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03612-z · Cited by 1,040 · cited 1×
“In the workplace, AI is optimizing processes in healthcare, such as diagnostics, medical imaging, and surgery (Benzakour et al., 2023; Kumar et al., 2023; Larrazabal et al., 2020), and in education, where it is improving performance assessments and enabling personalized learning (Chiu & Silva,…”
RetractedA First Course in Factor Analysis
Comrey, A. L., & Lee, H. B. (2013) · doi:10.4324/9781315827506 · Cited by 4,546
Identifying careless responses in survey data.
Meade, A. W., & Craig, S. B. (2012) · Psychological Methods · doi:10.1037/a0028085 · Cited by 3,865 · cited 1×
“The selection and construction of the control questions were based on the work of Barone et al. (2015) and Meade and Craig (2012).”
Conceptualizing AI literacy: An exploratory review
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100041 · Cited by 2,205 · cited 2×
“Based on Ng et al. (2021), Carolus, Koch, et al. (2023) developed the Meta AI Literacy Scale (MAILS), a robust and widely cited subjective assessment tool.”
Heuristics versus statistics in discriminant validity testing: a comparison of four procedures
Franke, G., & Sarstedt, M. (2019) · Internet Research · doi:10.1108/intr-12-2017-0515 · Cited by 1,951
Artificial intelligence: A powerful paradigm for scientific research
Xu, Y., Liu, X., Cao, X., Huang, C., Liu, E., Qian, S., Liu, X., Wu, Y., Dong, F., Qiu, C.-W., et al. (2021) · The Innovation · doi:10.1016/j.xinn.2021.100179 · Cited by 1,726 · cited 1×
“AI is also used in finance and science for applications such as fraud prevention (Khare & Srivastava, 2023) and innovative hypothesis generation (Sourati & Evans, 2023; Xu et al., 2021).”
Is More Ever Too Much? The Number of Indicators per Factor in Confirmatory Factor Analysis
Marsh, H. W., Hau, K.-T., Balla, J. R., & Grayson, D. (1998) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3302_1 · Cited by 1,307
Larrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1919012117 · Cited by 685 · cited 1×
“…workplace, AI is optimizing processes in healthcare, such as diagnostics, medical imaging, and surgery (Benzakour et al., 2023; Kumar et al., 2023; Larrazabal et al., 2020), and in education, where it is improving performance assessments and enabling personalized learning (Chiu & Silva, 2024;…”
Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., & Wienrich, C. (2023) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2023.100014 · Cited by 488
Self-citationCarolus, A., Koch, M., Straka, S., Latoschik, M. E., & Wienrich, C. (2023) · arXiv · doi:10.48550/arxiv.2302.09319 · Cited by 411
Self-citationNg, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13411 · Cited by 390
Creation and Evaluation of a Pretertiary Artificial Intelligence (AI) Curriculum
Chiu, T. K. F., Meng, H., Chai, C.-S., King, I., Wong, S., & Yam, Y. (2022) · IEEE Transactions on Education · doi:10.1109/te.2021.3085878 · Cited by 380
Generative AI Literacy: Twelve Defining Competencies
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025) · Digital Government: Research and Practice · doi:10.1145/3685680 · Cited by 333 · cited 13×
“The present study developed the AI Competency Objective Scale (AICOS), an advanced instrument for measuring AI literacy that integrates current AI literacy and measurement models and takes into account the latest developments in the field of artificial intelligence by incorporating Generative AI Literacy (Annapureddy…”
InfluentialHornberger, M., Bewersdorff, A., & Nerdel, C. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100165 · Cited by 276 · cited 7×
“In previous studies, individuals with technical backgrounds, such as engineering and computer science, demonstrated higher AI literacy compared to those with other educational backgrounds, including social sciences (Hornberger et al., 2023; Mansoor et al., 2024).”
InfluentialLaupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2023.100338 · Cited by 269 · cited 2×
“Many existing AI literacy scales rely on self-reporting, capturing only individuals' subjective perceptions of their AI knowledge and skills (Carolus, Koch, et al., 2023; Laupichler et al., 2023; Lee & Park, 2024; Ng et al., 2024; Pinski & Benlian, 2023; Wang & Chuang, 2024; Zhang et al., 2024).”
Explicating AI Literacy of Employees at Digital Workplaces
Cetindamar, D., Kitto, K., Wu, M., Zhang, Y., Abedin, B., & Knight, S. (2024) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2021.3138503 · Cited by 239 · cited 1×
“…the increasing integration of AI literacy content in curricula and training, many individuals still lack essential competencies (Antonenko & Abramowitz, 2023; Carolus, Augustin, et al., 2023; Cetindamar et al., 2022; Markus et al., 2024; Mertala & Fagerlund, 2024; Nader et al., 2024; Yim, 2024).”
Laupichler, M. C., Aster, A., Meyerheim, M., Raupach, T., & Mergen, M. (2024) · BMC Medical Education · doi:10.1186/s12909-024-05400-7 · Cited by 170
Kong, S.-C., Cheung, M.-Y. W., & Tsang, O. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100214 · Cited by 167
Tempelaar, D., Rienties, B., & Nguyen, Q. (2020) · PLOS ONE · doi:10.1371/journal.pone.0233977 · Cited by 137
Carolus, A., Augustin, Y., Markus, A., & Wienrich, C. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100114 · Cited by 128 · cited 1×
“…the increasing integration of AI literacy content in curricula and training, many individuals still lack essential competencies (Antonenko & Abramowitz, 2023; Carolus, Augustin, et al., 2023; Cetindamar et al., 2022; Markus et al., 2024; Mertala & Fagerlund, 2024; Nader et al., 2024; Yim, 2024).”
Self-citationKhanjani, Z., Watson, G., & Janeja, V. P. (2023) · Frontiers in Big Data · doi:10.3389/fdata.2022.1001063 · Cited by 121
Verma, A., Lamsal, K., & Verma, P. (2022) · Industry and Higher Education · doi:10.1177/0950422221990990 · Cited by 115
Development and validation of ChatGPT literacy scale
Lee, S., & Park, G. (2024) · Current Psychology · doi:10.1007/s12144-024-05723-0 · Cited by 100 · cited 1×
“Many existing AI literacy scales rely on self-reporting, capturing only individuals' subjective perceptions of their AI knowledge and skills (Carolus, Koch, et al., 2023; Laupichler et al., 2023; Lee & Park, 2024; Ng et al., 2024; Pinski & Benlian, 2023; Wang & Chuang, 2024; Zhang et al., 2024).”
AI Literacy - Towards Measuring Human Competency in Artificial Intelligence
Pinski, M., & Benlian, A. (2023) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2023.021 · Cited by 94
ChatGPT and the digitisation of writing
Zhao, X., Cox, A., & Cai, L. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-02904-x · Cited by 93
Artificial intelligence literacy among university students—a comparative transnational survey
Mansoor, H. M. H., Bawazir, A., Alsabri, M. A., Alharbi, A., & Okela, A. H. (2024) · Frontiers in Communication · doi:10.3389/fcomm.2024.1478476 · Cited by 84 · cited 2×
“In previous studies, individuals with technical backgrounds, such as engineering and computer science, demonstrated higher AI literacy compared to those with other educational backgrounds, including social sciences (Hornberger et al., 2023; Mansoor et al., 2024).”
Zhang, H., Perry, A., & Lee, I. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-024-00398-x · Cited by 75
Meta AI literacy scale: Further validation and development of a short version
Koch, M. J., Carolus, A., Wienrich, C., & Latoschik, M. E. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e39686 · Cited by 44 · cited 1×
“To this end, subjective AI literacy was measured using the short version of the Meta AI Literacy Scale by Koch et al. (2024) (10 items, α = .77).”
Self-citation
Research works printed but not linked: 59
- Ajzen, 2011, Knowledge and the prediction of behavior: The role of information accuracy in the theory of planned behavior, Basic and Applied Social Psychology, 33, 101, 10.1080/01973533.2011.568834 · DOI · Paper page
- Alcalá-Quintana, 2013, Fitting model-based psychometric functions to simultaneity and temporal-order judgment data: MATLAB and R routines, Behavior Research Methods, 45, 972, 10.3758/s13428-013-0325-2 · DOI
- Ames, 2015, An NCME instructional module on item‐fit statistics for item response theory models, Educational Measurement: Issues and Practice, 34, 39, 10.1111/emip.12067 · DOI
- Antonenko, 2023, In-service teachers'(mis) conceptions of artificial intelligence in K-12 science education, Journal of Research on Technology in Education, 55, 64, 10.1080/15391523.2022.2119450 · DOI · Paper page
- Asio, 2024, Artificial intelligence (AI) literacy and academic performance of tertiary level students: A preliminary analysis, Social Sciences, Humanities and Education Journal (SHE Journal), 5, 309
- Barone, 2015, When deal depth doesn't matter: How handedness consistency influences consumer response to horizontal versus vertical price comparisons, Marketing Letters, 26, 213, 10.1007/s11002-013-9276-8 · DOI
- Beauducel, 2006, On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA, Structural Equation Modeling, 13, 186, 10.1207/s15328007sem1302_2 · DOI · Paper page
- Benzakour, 2023, Artificial intelligence in spine surgery, International Orthopaedics, 47, 457, 10.1007/s00264-022-05517-8 · DOI
- Borges, 2017, Measuring the quality of life in hypertension according to item response theory, Revista de Saúde Pública, 51, 45, 10.1590/s1518-8787.2017051006845 · DOI
- Brady, 2005, Assessment of learning with multiple-choice questions, Nurse Education in Practice, 5, 238, 10.1016/j.nepr.2004.12.005 · DOI
- Carlson, 2009, Objective and subjective knowledge relationships: A quantitative analysis of consumer research findings, Journal of Consumer Research, 35, 864, 10.1086/593688 · DOI · Paper page
- Chalmers, 2012, mirt: A multidimensional item response theory package for the R environment, Journal of Statistical Software, 48, 1, 10.18637/jss.v048.i06 · DOI · Paper page
- Chen, 1997, Local dependence indexes for item pairs using item response theory, Journal of Educational and Behavioral Statistics, 22, 265, 10.3102/10769986022003265 · doi:10.3102/10769986022003265 · Paper page
- Chiu, 2024, Developing and validating measures for AI literacy tests: From self-reported to objective measures, Computers and Education: Artificial Intelligence, 7
- Cohen, 1992, A power primer, Psychological Bulletin, 112, 155, 10.1037/0033-2909.112.1.155 · doi:10.1037/0033-2909.112.1.155 · Paper page
- Cronbach, 1955, Construct validity in psychological tests, Psychological Bulletin, 52, 281, 10.1037/h0040957 · DOI · Paper page
- Ding, 2024, Development of an AI literacy assessment for non-technical individuals: What do teachers know?, Contemporary Educational Technology, 16, ep512, 10.30935/cedtech/14619 · DOI
- Feuerstahler, 2022, Metric stability in item response models, Multivariate Behavioral Research, 57, 94, 10.1080/00273171.2020.1809980 · DOI
- Hair, 2014, PLS-SEM: Indeed a silver bullet, Journal of Marketing Theory and Practice, 19, 139, 10.2753/MTP1069-6679190202 · DOI · Paper page
- Hammond, 1998, Multiple‐choice examinations: Adopting an evidence‐based approach to exam technique, Anaesthesia, 53, 1105, 10.1046/j.1365-2044.1998.00583.x · doi:10.1046/j.1365-2044.1998.00583.x
- Hayo, 2014, The German public and its trust in the ECB: The role of knowledge and information search, Journal of International Money and Finance, 47, 286, 10.1016/j.jimonfin.2014.07.003 · doi:10.1016/j.jimonfin.2014.07.003
- Henseler, 2015, A new criterion for assessing discriminant validity in variance-based structural equation modeling, Journal of the Academy of Marketing Science, 43, 115, 10.1007/s11747-014-0403-8 · DOI · Paper page
- Jin, 2024, GLAT: The generative AI literacy assessment test, arXiv.org
- Kanniainen, 2022, Online research and comprehension performance profiles among sixth‐grade students, including those with reading difficulties and/or attention and executive function difficulties, Reading Research Quarterly, 57, 1213, 10.1002/rrq.463 · DOI
- Kaur, 2023, Exploring AI literacy among older adults, Assistive Technology: Shaping a Sustainable and Inclusive World, 306, 9
- Khare, 2023, AI-Powered fraud prevention: A comprehensive analysis of machine learning applications in online transactions, Journal of Emerging Technologies and Innovative Research, 10, 2349
- Kharvi, 2024, Understanding the impact of AI-Generated deepfakes on public opinion, political discourse, and personal security in social media, IEEE Security and Privacy, 22, 115, 10.1109/MSEC.2024.3405963 · DOI
- Kılıç, 2021, Comparison of confirmatory factor analysis estimation methods on mixed-format data, International Journal of Assessment Tools in Education, 8, 21, 10.21449/ijate.782351 · DOI
- Klerck, 2007, The effect of knowledge types on consumer‐perceived risk and adoption of genetically modified foods, Psychology and Marketing, 24, 171, 10.1002/mar.20157 · DOI · Paper page
- Kose, 2012, Comparison of unidimensional and multidimensional models based on item response theory in terms of both variables of test length and sample size, Procedia - Social and Behavioral Sciences, 46, 135, 10.1016/j.sbspro.2012.05.082 · DOI
- Lewis, 2003, Measuring transactive memory systems in the field: Scale development and validation, Journal of Applied Psychology, 88, 587, 10.1037/0021-9010.88.4.587 · DOI · Paper page
- Linacre, 2017, Teaching rasch measurement, Rasch Measurement Transactions, 31, 1630
- Lund, 2023, Information literacy, data literacy, privacy literacy, and ChatGPT: Technology literacies align with perspectives on emerging technology adoption within communities, Human Technology, 19, 163, 10.14254/1795-6889.2023.19-2.2 · DOI
- Ma, 2023, Are digital natives overconfident in their privacy literacy? Discrepancy between self-assessed and actual privacy literacy, and their impacts on privacy protection behavior, Frontiers in Psychology, 14, 10.3389/fpsyg.2023.1224168 · DOI
- MacCallum, 1999, Sample size in factor analysis, Psychological Methods, 4, 84, 10.1037/1082-989X.4.1.84 · DOI · Paper page
- Manika, 2021, Drawing on subjective knowledge and information receptivity to examine an environmental sustainability policy: Insights from the UK's bag charge policy, European Management Review, 18, 249, 10.1111/emre.12453 · DOI
- Markus, 2024, Effects of AI understanding-training on AI literacy, usage, self-determined interactions, and anthropomorphization with voice assistants, Computers and Education Open, 6, 10.1016/j.caeo.2024.100176 · DOI · Paper page
- Maydeu-Olivares, 2013, Goodness-of-fit assessment of item response theory models, Measurement: Interdisciplinary Research and Perspectives, 11, 71
- Maydeu-Olivares, 2006, Limited information goodness-of-fit testing in multidimensional contingency tables, Psychometrika, 71, 713, 10.1007/s11336-005-1295-9 · DOI · Paper page
- Mertala, 2024, Finnish 5th and 6th graders' misconceptions about artificial intelligence, International Journal of Child-Computer Interaction, 39, 10.1016/j.ijcci.2023.100630 · DOI · Paper page
- Messick, 1995, Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning, American Psychologist, 50, 741, 10.1037/0003-066X.50.9.741 · DOI · Paper page
- Moore, 2008, The trouble with overconfidence, Psychological Review, 115, 502, 10.1037/0033-295X.115.2.502 · DOI · Paper page
- Moorman, 2004, Subjective knowledge, search locations, and consumer choice, Journal of Consumer Research, 31, 673, 10.1086/425102 · DOI · Paper page
- Nader, 2024, Public understanding of artificial intelligence through entertainment media, AI & Society, 39, 713, 10.1007/s00146-022-01427-w · DOI · Paper page
- Pinski, 2024, Building metaknowledge in AI literacy–the effect of gamified vs. text-based learning on AI literacy metaknowledge, 10.24251/HICSS.2024.619 · DOI
- Pratama, 2023, Revolutionizing education: Harnessing the power of artificial intelligence for personalized learning, Klasikal: Journal of education, language teaching and science, 5, 350, 10.52208/klasikal.v5i2.877 · DOI · Paper page
- Raju, 1995, Differential effects of subjective knowledge, objective knowledge, and usage experience on decision making: An exploratory investigation, Journal of Consumer Psychology, 4, 153, 10.1207/s15327663jcp0402_04 · DOI · Paper page
- Raju, 2007, Standardized conditional SEM: A case for conditional reliability, Applied Psychological Measurement, 31, 169, 10.1177/0146621606291569 · DOI
- Reeve, 2005, Applying item response theory modeling for evaluating questionnaire item and scale properties, Assessing quality of life in clinical trials: methods of practice, 2, 55, 10.1093/oso/9780198527695.003.0005 · DOI · Paper page
- Reise, 2009, Item response theory and clinical measurement, Annual Review of Clinical Psychology, 5, 27, 10.1146/annurev.clinpsy.032408.153553 · DOI · Paper page
- Robinson, 2018, Using multi‐item psychometric scales for research and practice in human resource management, Human Resource Management, 57, 739, 10.1002/hrm.21852 · DOI · Paper page
- Schermelleh-Engel, 2003, Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures, Methods of psychological research online, 8, 23
- Sourati, 2023, Accelerating science with human-aware artificial intelligence, Nature Human Behaviour, 7, 1682, 10.1038/s41562-023-01648-z · DOI · Paper page
- Sternberg, 1995, A prototype view of expert teaching, Educational Researcher, 24, 9, 10.3102/0013189X024006009 · doi:10.3102/0013189x024006009 · Paper page
- Toker Gokce, 2024, Investigating the level of artificial intelligence literacy of university students using decision trees, Education and Information Technologies, 1
- Wang, 2024, Artificial intelligence self-efficacy: Scale development and validation, Education and Information Technologies, 29, 4785, 10.1007/s10639-023-12015-w · DOI · Paper page
- Waters, 2018, Examining the interrelations among objective and subjective health literacy and numeracy and their associations with health knowledge, Journal of General Internal Medicine, 33, 1945, 10.1007/s11606-018-4624-2 · DOI
- Yen, 1984, Effects of local item dependence on the fit and equating performance of the three-parameter logistic model, Applied Psychological Measurement, 8, 125, 10.1177/014662168400800201 · DOI · Paper page
- Yim, 2024, A critical review of teaching and learning artificial intelligence (AI) literacy: Developing an intelligence-based AI literacy framework for primary school education, Computers and Education: Artificial Intelligence
Cited by
The AI literacy heptagon: A structured approach to AI literacy in higher education
Hackl, V., Müller, A. E., & Sailer, M. (2026) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2026.100540 · Cited by 41
“Existing instruments such as SNAIL (Laupichler et al., 2023) emphasize technical, critical appraisal and application dimensions; more recent questionnaires such as AILQ (Ng et al., 2024) or AICOS (Markus et al., 2025) provide broader perspectives.”
Yueh, R. (2026) · International Journal of Business & Management Studies · doi:10.56734/ijbms.v7n7a1 · Cited by 1
Metadata
- Authors
- Markus, A., Carolus, A., & Wienrich, C.
- ORCID
- Markus, André
- Venue
- Computers and Education: Artificial Intelligence
- Publisher
- Elsevier BV
- Published
- 2025-12
- Volume
- 9
- Pages
- 100485
- ISSN
- 2666-920X
- Cited by
- 30
- References
- 28
- Language
- en
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Funders
- Federal Republic of Germany Ministry for Labour and Social Affairs (DKI.00.00030.21)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.