The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Measuring user competence in using artificial intelligence: validity and reliability of artificial intelligence literacy scale
Abstract
As artificial intelligence (AI) became a part of daily life, it has become important to determine user competence in using AI technology. Here, we propose the concept of AI literacy and develop a quantitative scale for obtaining accurate data regarding the AI literacy of ordinary users. We first identified the primary core constructs of AI literacy, including awareness, use, evaluation, and ethics. Next, we generated 65 items to capture these four constructs; only 31 items were retained after a three-step content validation process. Then, we conducted a survey, and collected two samples of data. By reducing the number of items using the first sample and performing reliability and validity tests on the second sample, we obtained a 12-item instrument for the quantitative measurement of AI literacy. The results confirmed that the proposed four-construct model is an adequate representation of AI literacy. Further, AI literacy is significantly related to digital literacy, attitude towards robots, and users’ daily usage of AI. This study will not only aid researchers in understanding how user competence in using AI technology affects human–AI interactions but will also help designers develop AI applications that are aligned with the AI literacy levels of the target users.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This attitude scale was used because it is one of the shortest and, thus, one of the most resource-efficient instruments designed to capture attitudes; it had already been validated in the native language of the course participants.”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [22]
In the citing paper
“This could be an indication of discriminant validity because, in theory, AI literacy and attitudes toward AI are assumed to be two different constructs, representing cognitive/skill and affective aspects, respectively [9,16,20-22].”
From Evaluating AI Courses: A Valid and Reliable Instrument for Assessing Artificial-Intelligence Learning through Comparative Self-Assessment (2023) · cites this paper as [22]
In the citing paper
“The reliability coefficients of the sub-dimensions range between 0.72 and 0.76, and the overall reliability (Cronbach’s alpha) of the scale is 0.84, indicating good internal consistency.”
From Integrating artificial intelligence into preservice teacher education: enhancing AI literacy and digital material design competency (2026) · cites this paper as Wang et al. (2022)
In the citing paper
“The Artificial Intelligence Literacy Scale (AILS; Wang et al., 2023) measures AI understanding and competence across four subscales: awareness, usage, evaluation and ethics, using 12 items on a 7- point Likert scale.”
From The role of critical thinking on undergraduates' reliance behaviours on generative AI in problem‐solving (2025) · cites this paper as Wang et al., 2023
In the citing paper
“The validity of the self-report measurement instruments for AI literacy [11,12,14,15] has only been tested regarding their factorial structure.”
From Meta AI literacy scale: Further validation and development of a short version (2024) · cites this paper as [15]
In the citing paper
“It appears that individuals with higher positive attitudes, lower negative attitudes, and more willingness to use technology are also more likely to consider themselves competent [1, 46, 48].”
In the citing paper
“According to Wang et al. [47], the degree of AI literacy can determine individuals' readiness to embrace AI technologies.”
From Modeling teachers’ acceptance of generative artificial intelligence use in higher education: the role of AI literacy, intelligent TPACK, and perceived trust (2024) · cites this paper as [47]
In the citing paper
“AI literacy improves understanding of GenAI's advantages and functions, which encourages a favorable view of its acceptance and use in educational contexts [3,47,48,64].”
From Modeling teachers’ acceptance of generative artificial intelligence use in higher education: the role of AI literacy, intelligent TPACK, and perceived trust (2024) · cites this paper as [47]
In the citing paper
“Wang et al. [58] affirmed a notable correlation between AI literacy and attitudes toward robots’ and users’ regular use of AI.”
From ChatGPT in Learning: Assessing Students’ Use Intentions through the Lens of Perceived Value and the Influence of AI Literacy (2024) · cites this paper as [58]
In the citing paper
“Higher AI literacy enables users to better understand the capabilities and limitations of AI technologies, leading to more effective adoption and usage [22,58].”
From ChatGPT in Learning: Assessing Students’ Use Intentions through the Lens of Perceived Value and the Influence of AI Literacy (2024) · cites this paper as [58]
In the citing paper
“Wang et al. [58] indicated that the level of AI literacy determines whether students embrace AI technologies.”
From ChatGPT in Learning: Assessing Students’ Use Intentions through the Lens of Perceived Value and the Influence of AI Literacy (2024) · cites this paper as [58]
In the citing paper
“Their ability to leverage the tool efficiently contributes to their perception of the AI tool as highly useful [22,58,62].”
From ChatGPT in Learning: Assessing Students’ Use Intentions through the Lens of Perceived Value and the Influence of AI Literacy (2024) · cites this paper as [58]
In the citing paper
“The support for H10, which states that AI literacy positively influences the perceived value of ChatGPT, underscores the significance of students’ understanding of and familiarity with AI in appreciating the tool’s potential and contributions to their academic and personal endeavours.”
From ChatGPT in Learning: Assessing Students’ Use Intentions through the Lens of Perceived Value and the Influence of AI Literacy (2024) · cites this paper as [58]
Citations
Cites
Evaluating Structural Equation Models with Unobservable Variables and Measurement Error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.2307/3151312 · Cited by 109,314
Evaluating structural equation models with unobservable variables and measurement error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800104 · Cited by 109,190
(1994) · Journal of the American Dietetic Association · doi:10.1016/0002-8223(94)92216-0 · Cited by 46,726
Principles and Practice of Structural Equation Modeling
Tabri, N., & Elliott, C. M. (2012) · Canadian Graduate Journal of Sociology and Criminology · doi:10.15353/cgjsc.v1i1.3787 · Cited by 39,094
A new criterion for assessing discriminant validity in variance-based structural equation modeling
Henseler, J., Ringle, C. M., & Sarstedt, M. (2015) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-014-0403-8 · Cited by 38,459
null (2016) · Philosophy Study · doi:10.17265/2159-5313/2016.09.003 · Cited by 28,317
Einax, J. W., Zwanziger, H. W., & Geiß, S. (1997) · Wiley eBooks · doi:10.1002/352760216x.ch5 · Cited by 18,779
Scale Development: Theory and Applications.
Xie, Y.-P., & Devellis, R. (1992) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2075704 · Cited by 14,143
Costello, A. B., & Osborne, J. (2005) · University of Massachusetts Amherst · doi:10.7275/jyj1-4868 · Cited by 10,297
Constructing validity: Basic issues in objective scale development.
Clark, L. A., & Watson, D. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.309 · Cited by 6,567
Research works printed but not linked: 15
- Buckingham D., 1989, Radical Philosophy, 51, 12
- Buckingham D., 2007, Journal of Educational Multimedia and Hypermedia, 16, 323
- Calvani A., 2008, Journal of E-Learning and Knowledge Society, 4, 183
- 10.1177/002224378001700401 · DOI
- Eshet Y., 2004, Journal of Educational Multimedia and Hypermedia, 13, 93
- Finstad K., 2010, Journal of Usability Studies, 5, 104
- Gapski, H. 2007. “Some Reflections on Digital Literacy.” Paper Presented at the Proceedings of the 3rd International Workshop on Digital Literacy.
- Gilster P., 1997, Digital Literacy
- 10.7551/mitpress/8975.001.0001 · DOI · Paper page
- 10.17265/2159-5313/2016.09.003 · DOI · Paper page
- Kim Y., 2013, Journal of Digital Convergence, 11, 9
- Likert R., 1932, Archives of Psychology
- Mayerl J., 2016, Green European, 210
- Tarafdar M., 2019, MIT Sloan Management Review, 60, 37
- Wilson, H. J., and P. R. Daugherty. 2018. “Collaborative Intelligence: Humans and AI Are Joining Forces.” Harvard Business Review .
Cited by
Sorted by citations
Items 1,001–1,003 of 1,003
Mendoza-Zenozain, E. A., & Córdova-Buiza, F. (2026) · Contemporary Educational Technology · doi:10.30935/cedtech/19597
Using emojis to uplift emotion: Effects on mental health chatbots’ performance
Deng, Z., Yan, X., & Fan, G. (2026) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2026.105159
Wang, H., Xu, Z., Huang, Z.-H., Pan, H.-H., & Zheng, G.-B. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1931423
Metadata
- Authors
- Wang, B., Rau, P.-L. P., & Yuan, T.
- Affiliations
- Tsinghua University (CN)
- Corresponding author
- Rau, Pei-Luen Patrick
- Venue
- Behaviour & Information Technology
- Publisher
- Informa UK Limited
- Published
- 2022-05-10
- Volume
- 42
- Issue
- 9
- Pages
- 1324-1337
- ISSN
- 0144-929X (ISSN-L)
- 1362-3001
- OpenAlex
- W4280549083
- Cited by
- 1,036
- References
- 69
- Citations by year
- 2026: 549 · 2025: 323 · 2024: 137 · 2023: 27
- FWCI
- 120.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Behaviour and Information Technology (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- Ministry of Science and Technology of China (2018AAA0101700)
- Topics
- Social Sciences › Education › Digital literacy in education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Gender Studies › Gender and Technology in Education
- Keywords
- AI literacy, AI literacy scale, user competence, AI awareness, AI ethics, AI evaluation, digital literacy, artificial intelligence use, human-AI interaction, scale development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
