Developing a holistic AI literacy assessment matrix – Bridging generic, domain-specific, and ethical competencies
Open accessThe DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Abstract
Motivated by a holistic understanding of AI literacy, this work presents an interdisciplinary effort to make AI literacy measurable in a comprehensive way, considering generic and domain-specific AI literacy as well as AI ethics. While many AI literacy assessment tools have been developed in the last 2-3 years, mostly in the form of self-assessment scales and less frequently as knowledge-based assessments, previous approaches only accounted for one specific area of a comprehensive understanding of AI competence, namely cognitive aspects within generic AI literacy. Considering the demand for AI literacy development for different professional domains and reflecting on the concept of competence in a way that goes beyond mere cognitive aspects of conceptual knowledge, there is an urgent need for assessment methods that capture domain-specific AI literacy on each of the three competence dimensions of cognition, behavior, and attitude. In addition, competencies for AI ethics are becoming more apparent, which further calls for a comprehensive assessment of AI literacy for this very matter. This conceptual paper aims to provide a foundation upon which future AI literacy assessment instruments can be built and provides insights into what a framework for item development might look like that addresses both generic and domain-specific aspects of AI literacy as well as AI ethics literacy, and measures more than just knowledge-related aspects based on a holistic approach.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 2×
“Especially with the emergence of large language models (LLMs), the challenges and risks that AI poses to the education system have been widely discussed (e.g., [33,58,80,88]).”
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · cited 1×
“…of domain-specific AI literacy, which considers the domain requirements for the application of AI in that domain and thus goes well beyond the concept of generic AI literacy, requiring that the constructors of domain-specific courses themselves possess both domain and AI knowledge [189].”
Motivation reconsidered: The concept of competence.
White, R. W. (1959) · Psychological Review · doi:10.1037/h0040934 · Cited by 5,681 · cited 1×
“White [179, p. 317 argued: "that it is necessary to make competence a motivational concept, that there is a competence motivation ” .”
Enhancing teaching through constructive alignment
Biggs, J. (1996) · Higher Education · doi:10.1007/bf00138871 · Cited by 4,084
MacKenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011) · MIS Quarterly · doi:10.2307/23044045 · Cited by 3,165 · cited 1×
“Second, AI literacy instruments that were developed according to this matrix as well as existing AI literacy instruments can be used to validate this holistic framework and provide insights into the nature and the relationships among generic, domain-specific, and ethical AI literacy [111].”
Conceptualizing AI literacy: An exploratory review
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100041 · Cited by 2,205 · cited 7×
“…available to all members of a social group, which in terms of AI literacy aims at enabling an AI-literate society, but also which competencies should be available in a complementary way, e.g. for the purposeful use and integration of AI technologies in specific professional domains [22, 127,176].”
InfluentialMikalef, P., & Gupta, M. (2021) · Information & Management · doi:10.1016/j.im.2021.103434 · Cited by 1,700
Artificial Intelligence and Machine Learning in Clinical Medicine, 2023
Haug, C. J., & Drazen, J. M. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2302038 · Cited by 1,215
Cited by
Liu, M., Zhang, L. J., & Zhang, D. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108569 · Cited by 43
AI literacy of library and information science students: A study of Bangladesh, India and Pakistan
Hossain, Z., Biswas, M. S., & Khan, G. (2026) · Journal of Librarianship and Information Science · doi:10.1177/09610006241309323 · Cited by 43
The AI literacy heptagon: A structured approach to AI literacy in higher education
Hackl, V., Müller, A. E., & Sailer, M. (2026) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2026.100540 · Cited by 41
“• Ethical Awareness and Reasoning (EAR): This dimension is complex and it is "necessary to comprise many different and complex aspects under one term" (Knoth et al., 2024a).”
Developing and validating measures for AI literacy tests: From self-reported to objective measures
Chiu, T. K. F., Chen, Y., Yau, K. W., Chai, C.-S., Meng, H., King, I., Wong, S., & Yam, Y. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100282 · Cited by 41
KI meets lehre – die notwendigkeit DER curricularen verankerung von KI-kompetenzen
Lambrecht, G.-M., Lintfert, B., Martschiske, R., & Wiehenbrauk, D. (2025) · Zeitschrift für Hochschulentwicklung · doi:10.21240/zfhe/sh-ki-2/02 · Cited by 5
Metadata
- Authors
- Knoth, N., Decker, M., Laupichler, M. C., Pinski, M., Buchholtz, N., Bata, K., & Schultz, B.
- Venue
- Computers and Education Open
- Publisher
- Elsevier BV
- Published
- 2024-06
- Volume
- 6
- Pages
- 100177
- ISSN
- 2666-5573 (ISSN-L)
- OpenAlex
- W4394689246
- Cited by
- 117
- References
- 63
- FWCI
- 44.77
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Developing a holistic AI literacy assessment matrix – Bridging generic, domain-specific, and ethical competencies. https://app.cochat.ai/papers/10.1016/j.caeo.2024.100177
- The paper
- Knoth, N., Decker, M., Laupichler, M. C., Pinski, M., Buchholtz, N., Bata, K., & Schultz, B. (2024). Developing a holistic AI literacy assessment matrix – Bridging generic, domain-specific, and ethical competencies. Computers and Education Open. https://doi.org/10.1016/j.caeo.2024.100177
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.