CoChat
Sign Up
Cited by 490Digital literacy in educationEthics and Social Impacts of AIArtificial Intelligence in Education

MAILS - Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies

Open access
Carolus, Astrid, Koch, Martin J., Straka, Samantha, Latoschik, Marc Erich, Wienrich, Carolin·2023·Computers in Human Behavior: Artificial Humans
DOI 10.1016/j.chbah.2023.100014
Open this paper in CoChat — free account

Abstract

Valid measurement of AI literacy is important for the selection of personnel, identification of shortages in skill and knowledge, and evaluation of AI literacy interventions. A questionnaire is missing that is deeply grounded in the existing literature on AI literacy, is modularly applicable depending on the goals, and includes further psychological competencies in addition to the typical facets of AIL. This paper presents the development and validation of a questionnaire considering the desiderata described above. We derived items to represent different facets of AI literacy and psychological competencies, such as problem-solving, learning, and emotion regulation in regard to AI. We collected data from 300 German-speaking adults to confirm the factorial structure. The result is the Meta AI Literacy Scale (MAILS) for AI literacy with the facets Use & apply AI, Understand AI, Detect AI, and AI Ethics and the ability to Create AI as a separate construct, and AI Self-efficacy in learning and problem-solving and AI Self-management (i.e., AI persuasion literacy and emotion regulation). This study contributes to the research on AI literacy by providing a measurement instrument relying on profound competency models. Psychological competencies are included particularly important in the context of pervasive change through AI systems.

Verified 2026-11-05 · 1 reference not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

The paper develops and tests MAILS, a modular self-report questionnaire for AI literacy grounded in Ng et al.’s competency model and extended with psychological meta-competencies. In 300 German-speaking adults, confirmatory factor analyses supported AI Literacy facets (Use & Apply AI, Know & Understand AI, Detect AI, AI Ethics), a separate Create AI factor, and two psychological domains: AI Self-efficacy and AI Self-competency.

Key findings

  • The final modified CFA model had good fit despite a significant chi-square: χ²(513) = 886.87, p < .001, CFI = .926, TLI = .920, RMSEA = .057, 95% CI [.051, .063], SRMR = .079.
  • AI Literacy was supported as a higher-order factor with “Use & Apply AI,” “Know & Understand AI,” “Detect AI,” and “AI Ethics”; “Create AI” did not load on AI Literacy and correlated with it at r = .49.
  • Instead of one AI Self-management factor, the study found two domains: “AI Self-efficacy” (AI Learning and AI Problem solving) and “AI Self-competency” (AI Persuasion literacy and AI Emotion regulation).
  • In the extended model including attitudes and willingness to use technology, fit was acceptable: χ²(2035) = 3004.35, p < .001, CFI = .900, TLI = .894, RMSEA = .043, 95% CI [.039, .047], SRMR = .069; AI Literacy correlated with AI Self-competency at .86 and AI Self-efficacy at .93.

Methods

  • Online survey of 300 German-speaking adults recruited via Prolific on 3–4 November 2022; mean age 32.13 years (SD = 11.66, range 18–72), 48.33% female, 50.67% male, 1.00% diverse.
  • Items were generated from AI literacy literature and psychological competency concepts: 72 AI literacy/self-assessment items plus 12 AI self-management items, with 68 items retained for analysis after review.
  • Confirmatory factor analysis was conducted in R with lavaan (version 0.6-12) using robust Satorra-Bentler estimation; the model was modified after poor initial fit by removing low-loading items, moving three Know & Understand AI items to Detect AI, separating Create AI, and splitting AI Self-management into AI Self-efficacy and AI Self-competency.

Limitations

  • The paper states the sample was online and specific to German-speaking individuals living in Germany or Austria, and that no already validated AI literacy instrument was available to validate the questionnaire against.
  • The authors state that because they made changes to the model after testing it, their approach should be considered exploratory, and future research should test the factorial structure in independent samples.
  • The paper states the most important next step is validation with external criteria such as tests, tasks, expert evaluations, group differences, intervention change, or future validated questionnaires; the text also makes evident that the findings rely on self-report and cross-sectional correlational data rather than behavioral outcomes.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.chbah.2023.100014

Cites

49 of 50 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

7 of 35 cited sources checked · 1 supported · 1 flagged · accuracy not yet measured

  • Premisecitation
    § 4 DISCUSSION · p. 9

    Most conceptualisations of AI literacy do not explicitly mention the development of AI.

    “This is in line with most conceptualisations of AI literacy where the development of AI is not explicitly mentioned [13, 23, 29]”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [13] Chai, 2020, An extended theory of planned behavior for the modelling of Chinese secondary school students' intention to learn artificial intelligence, Mathematics source
    • cites [23] Kim, 2021, Why and what to teach: Ai curriculum for elementary school source

      The Existing Guidelines for K-12 AI Education section discusses AI-literacy competencies but does not assess whether most conceptualisations omit AI development. The AI Knowledge section includes programming simple AI applications in this paper’s own framework; that example does not establish what most conceptualisations say. No page numbers are available in our copy. (LLM, conf=0.72)

      Experimental — accuracy not yet measured
    • cites [29] Latikka, 2019, Self-efficacy and acceptance of robots, Computers in Human Behavior source
  • Premisecitation
    § 2.2 A psychological perspective on the measurement of AI literacy · p. 4

    The subjective assessment of one's own competencies is central to the intention to use AI.

    “Outside the theory of the planned behavior-framework, it was also shown that the subjective assessment of one's own competencies is central to the intention to use AI [24, 26].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [24] Kim, 2022, The artificial intelligence literacy scale for middle school students, Journal of the Korea Society of Computer and Information

      Not checked in the free run.

    • cites [26] Kong, 2021, Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds, Computers and Education: Artificial Intelligence source

      Section 2.3, “Empowerment through AI” (page unavailable), defines AI self-efficacy as a person's assessment of their ability and says those with higher self-efficacy are more likely to begin or continue an AI-related task. This supports a connection between perceived competence and engagement, but the paper does not establish that perceived competence is central to an intention to use AI generally. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 2 THEORETICAL BACKGROUND · p. 2

    Wang and colleagues' AI literacy scale does not include the ability to develop AI applications as part of their conceptualisation.

    “For their measurement instrument on AI literacy, Wang and colleagues define AI literacy to have the components awareness , usage , evaluation , and ethics but do not include the ability to develop AI applications as part of their conceptualisation [45].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [45] Sniehotta, 2014, Time to retire the theory of planned behaviour, Health Psychology Review source

      The paper’s Criticism and Concerns about validity sections discuss the theory of planned behaviour, not Wang and colleagues’ AI literacy scale. Searches for the scale’s proposed components and the ability to develop AI applications found no passage addressing either point. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • Findingown data
    § 4 DISCUSSION · p. 9

    The final model found the facets Use & Apply AI, Know & understand AI, Detect AI, and AI Ethics loaded on a superordinate factor called AI Literacy.

    “we found the facets”
    Quote in the paperSubstantial claim
  • Findingown data
    § 3.3 Analysis · p. 9

    Self-assessed competence in the use of new technologies correlated negatively with AI Literacy.

    “It did not correlate with AI Self-efficacy and only negatively with AI Literacy and AI Self-competency.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 16
  • Classified as probably not a substantial claim. 17
  • The quote only partly states this claim. 2

  • May combine two claims.

    Premisecitation
    § 4 DISCUSSION · p. 10

    So far, there is some conceptual thinking to include such aspects, but no measurement tool yet.

    “So far, there is some conceptual thinking to include such aspects [8-10, 13, 47], but no measurement tool yet.”
    Quote in the paperSubstantial claim1 of 5 sources checked
    • cites [8] Bloom, 1956

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [9] Carolus, 2022, Digital interaction literacy model – conceptualizing competencies for literate interactions with voice-based ai systems, Computers and Education: Artificial Intelligence source
    • cites [10] Carolus, 2021, Towards a holistic approach and measurement of humans interacting with speech-based technology

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [13] Chai, 2020, An extended theory of planned behavior for the modelling of Chinese secondary school students' intention to learn artificial intelligence, Mathematics source

      Section 5 (Discussion and Conclusions) says the paper provides a theory-driven, validated survey with psychometric measures for nine factors related to learning AI. Section 3.2 (Instruments) describes its 41-item survey. For the aspects this source measures, the statement that there is 'no measurement tool yet' is contrary to the paper. (LLM, conf=0.86)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [47] Venkatesh, 2012, Consumer acceptance and use of information technology: Extending the unified theory of acceptance and use of technology, MIS Quarterly source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.2 A psychological perspective on the measurement of AI literacy · p. 4

    Learning has been considered as an important part of or addition for AI literacy before.

    “Learning has been considered as an important part of or addition for AI literacy before [8, 10, 13].”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [8] Bloom, 1956

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [10] Carolus, 2021, Towards a holistic approach and measurement of humans interacting with speech-based technology source
    • cites [13] Chai, 2020, An extended theory of planned behavior for the modelling of Chinese secondary school students' intention to learn artificial intelligence, Mathematics source

      The Introduction calls learning AI an important topic, and §3.2 defines AI literacy in terms of students’ perceived knowledge and skills. But §2.1 treats AI literacy as a background factor for students’ intention to learn AI; it does not explicitly describe learning as a component of, or addition to, AI literacy. The indexed passages provide no page numbers. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationcitation
    § 4 DISCUSSION · p. 10

    Create AI is separate and no part of AI Literacy.

    “This fits in with the general picture that Create AI is separate [13, 23, 29] and no part of AI Literacy.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [13] Chai, 2020, An extended theory of planned behavior for the modelling of Chinese secondary school students' intention to learn artificial intelligence, Mathematics source

      The Instruments section (3.2; page unavailable) defines the study’s AI-literacy measure as perceived knowledge and skills in using AI technology, and the Table 1 items concern understanding AI processes rather than creating AI. This supports a distinction in this study’s measure, but the paper does not establish the broader claim that creating AI is categorically separate from, and never part of, AI literacy. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
    • cites [23] Kim, 2021, Why and what to teach: Ai curriculum for elementary school source

      Checked before — its earlier verdict is held back until a new check runs.

    • cites [29] Latikka, 2019, Self-efficacy and acceptance of robots, Computers in Human Behavior source

  • May combine two claims.

    Findingown data
    § 3.3 Analysis · p. 8

    AI Self-efficacy and AI Self-competency showed very high correlations with AI Literacy, and were still highly correlated with each other but to a lesser extent.

    “Interestingly, AI Self-efficacy and AI Self-competency showed very high correlations with AI Literacy, whereas they were still highly correlated with each other, but to a lesser extent.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data
    § 4 DISCUSSION · p. 9

    Items for the evaluation of AI loaded on the factor for Know & understand AI.

    “Interestingly, items for the evaluation of AI did not load on one factor with items on”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation
    § 4.2 Conclusion · p. 11

    Overall, we find the factors Use & Apply AI, Know & understand AI, and AI Ethics with the addition of Detect AI.

    “Overall, we find the factors”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [33] Maslej, 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • The quote only partly states this claim.

    Findingown data
    § 4 DISCUSSION · p. 9

    The factor Evaluate & Create AI was not found.

    “The factor”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 2.2 A psychological perspective on the measurement of AI literacy · p. 4

    Emotion regulation helps to reduce negative and increase positive emotional arousal.

    “Additionally, emotion regulation helps to reduce negative and increase positive emotional arousal [8] and, by that, also increases perceived behavioral control.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] Bloom, 1956

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 1 INTRODUCTION · p. 2

    The MAILS instrument is embedded in the current literature on AI Literacy.

    “is embedded in the current literature on AI Literacy”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 4 DISCUSSION · p. 10

    Individuals with a higher positive attitude also deal with the topic more often and are therefore more competent.

    “Presumably, individuals with a higher positive attitude also deal with the topic more often and are therefore more competent.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4 DISCUSSION · p. 10

    This is in line with the conceptualisation by Long and Magerko, who included the ability to recognize AI in their framework.

    “This is in line with the conceptualisation by Long and Magerko [29], who included the ability to recognize AI in their framework.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] Latikka, 2019, Self-efficacy and acceptance of robots, Computers in Human Behavior source

  • May combine two claims.

    Interpretationown data
    § 4 DISCUSSION · p. 10

    Recognizing AI does not seem to be necessarily tied to knowing and understanding AI, but also is an independent competence.

    “Possibly, recognizing AI does not seem to be necessarily tied to knowing and understanding AI, but also is an independent competence.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4 DISCUSSION · p. 10

    Individuals with higher positive attitudes, lower negative attitudes, and more willingness to use technology are also more likely to consider themselves competent.

    “It appears that individuals with higher positive attitudes, lower negative attitudes, and more willingness to use technology are also more likely to consider themselves competent [1, 46, 48].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [1] Ajzen, 1985, From intentions to actions: A theory of planned behavior, Action control source
    • cites [46] for Statistics, U. I. (n.d.).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [48] Wang, 2022 source
    • Rests on an assumption: Self-assessed AI competencies reflect actual AI competencies.

  • May combine two claims.

    Interpretationown data
    § 4 DISCUSSION · p. 10

    These correlative findings suggest that there are relations between competencies related to AI and attitudes towards AI as well as willingness to use new technology.

    “These correlative findings suggest that there are relations between competencies related to AI and attitudes towards AI as well as the willingness to use new technology.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4 DISCUSSION · p. 10

    Problem solving and learning are competencies primarily aimed at managing information and information processing, while persuasion literacy and emotion regulation also focus on management of information, albeit with greater personal value.

    “Possibly, problem solving and learning are competencies that are primarily aimed at managing information and information processing, while persuasion literacy and emotion regulation also focus on the management of information, albeit with greater personal value (own decisions and emotions).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2 THEORETICAL BACKGROUND · p. 2

    Most conceptualisations of AI literacy parallel Bloom's taxonomy regarding their general configuration of skills.

    “As Ng and colleagues conclude [33], most conceptualisations of AI literacy parallel Bloom's taxonomy [7] regarding their general configuration of skills.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [33] Maslej, 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [7] van Beuningen, 2014

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2 THEORETICAL BACKGROUND · p. 2

    Ng and colleagues' review organizes AI literacy conceptualisations into four concepts: know and understand AI, use and apply AI, evaluate and create AI, and AI Ethics.

    “Ng and colleagues, in their review on AI literacy conceptualisations in education, postulate that these can be organized into four concepts: (1) know and understand AI, (2) use and apply AI, (3) evaluate and create AI, and (4) AI Ethics [33].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] Maslej, 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data
    § MAILS - Meta AI Literacy Scale: Development and Testing of an AI Literacy Questionnaire Based on Well-Founded Competency Models and Psychological Change- and Meta-Competencies · p. 1

    The resulting measurement instrument measures AI literacy with the facets 'Use & apply AI', 'Understand AI', 'Detect AI', and 'AI Ethics', and the ability to 'Create AI' as a separate construct, and 'AI Self-efficacy in learning and problem solving' and 'AI Self-management'.

    “The result is a measurement instrument that measures AI literacy with the facets”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning
    § 2 THEORETICAL BACKGROUND · p. 2

    The present paper fully incorporated all aspects of Bloom's taxonomy, including the component creation/development of AI, which was not previously considered.

    “Compared to the previous literature, we have fully incorporated all aspects of the taxonomy, including the component creation/development of AI , which was not previously considered.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning
    § 2.2 A psychological perspective on the measurement of AI literacy · p. 4

    AI Self-management is important to ensure the prolonged and sustainable use of AI.

    “We argue that AI Self-management is important to ensure the prolonged and sustainable use of AI and, thus, consider it to be an important part of our measurement instrument.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: New developments in AI might unsettle potential users, lead to failed behavior, and reduce perceived behavioral control.

  • May combine two claims.

    Premisecitation
    § 1 INTRODUCTION · p. 2

    There are not many ways to measure AI Literacy, and many scales are bound to specific contexts or are not validated.

    “So far, there are not many ways to measure AI Literacy, and many scales are bound to specific contexts (e.g., only usable in a medical setting) or are not validated [20, 46].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [20] 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [46] for Statistics, U. I. (n.d.).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Interpretationnone given
    § 1 INTRODUCTION · p. 2

    The MAILS instrument considers psychological meta-competencies.

    “considers psychological meta-competencies”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 1 INTRODUCTION · p. 2

    The MAILS instrument is cross-contextual applicable.

    “is cross-contextual applicable”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning
    § 2.2 A psychological perspective on the measurement of AI literacy · p. 4

    An instrument for measuring AI literacy with the aim of predicting and preparing the use of AI in a professional context should primarily focus on the subjective assessment of one's own competencies (i.e., behavioral control or self-efficacy).

    “We argue that an instrument for measuring AI literacy with the aim of predicting and preparing the use of AI in a professional context should, from a psychological point of view, primarily focus on the subjective assessment of one's own competencies (i.e., behavioral control or self-efficacy).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data
    § 3.2 Measures · p. 6

    The study obtained informed consent from participants.

    “Prior to participation, the participants were informed about the general purpose of the study and gave their informed consent.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § MAILS - Meta AI Literacy Scale: Development and Testing of an AI Literacy Questionnaire Based on Well-Founded Competency Models and Psychological Change- and Meta-Competencies · p. 1

    The MAILS instrument includes higher-order psychological competencies that are particularly important in the context of pervasive change through AI systems.

    “In addition, higher-order psychological competencies are included that are particularly important in the context of pervasive change through AI systems.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The introduction of AI systems is often accompanied by general change processes that must be mastered constructively.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning
    § 4.1 Limitations and Future Work · p. 11

    We made changes to the model, so our approach is necessary to consider exploratory.

    “Even though we chose confirmatory factor analysis to test our models, we made changes to the model, thus, it is necessary to consider our approach exploratory.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given
    § 4.1 Limitations and Future Work · p. 10

    Other instruments exist but none have been validated with external criteria so far.

    “Although other instruments exist, non have been validated with external criteria so far.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given
    § 4.1 Limitations and Future Work · p. 10

    Our sample was collected online and is specific to German-speaking individuals who live in Germany or Austria.

    “Our sample was collected online and is specific to German-speaking individuals who live in Germany or Austria.”
    Quote in the paperNot a substantial claim
29 of 29

  • The paper relies on this without support.

    The introduction of AI systems is often accompanied by general change processes that must be mastered constructively.

    “since the introduction of AI systems is often accompanied by general change processes that must be mastered constructively.”

  • The paper relies on this without support.

    Many prospective users of AI who are already working will not benefit from AI education in schools right now.

    “However, many prospective users of AI who are already working will not benefit from AI education in schools right now.”

  • The paper relies on this without support.

    New developments in AI might unsettle potential users, lead to failed behavior, and reduce perceived behavioral control.

    “Innovations might unsettle potential users, lead to failed behavior, and thus reduce perceived behavioral control.”

  • The paper relies on this without support.

    Self-assessed AI competencies reflect actual AI competencies.

    “It appears that individuals with higher positive attitudes, lower negative attitudes, and more willingness to use technology are also more likely to consider themselves competent [1, 46, 48].”

  • The paper relies on this without support.

    The factorial structure identified in this sample is valid in the population.

    “Future research should aim to test the factorial structure we identified in independent samples.”

  • The paper relies on this without support.

    The items based on the existing literature adequately cover AI literacy.

    “We based our developed items on the existing literature on AI literacy and psychological competencies.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The second sample (N = 219) also confirmed the measurement model of the MAILS as it showed a good model fit.

    “The second sample (N = 219) also confirmed the measurement model of the MAILS as it showed a good model fit (χ2(513) = 781.49, p < .001, CFI = .95, RMSEA = .05, 95 %-CI [.05, .06], SRMR = .06).”

    From Meta AI literacy scale: Further validation and development of a short version (2024) · cites this paper as [10]

    Section 3.3 (p. 8) reports good fit for a modified MAILS measurement model, but gives different fit statistics: χ²(513) = 886.87, CFI = .926, RMSEA = .057, and SRMR = .079. The paper describes data from 300 participants (Empirical Study, p. 6), not a second sample of 219; section 4.1 (p. 10) says testing the structure in independent samples remains future work. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • In the first study by Carolus, Koch, and colleagues, a factorial structure for the MAILS was confirmed.

    “In the first study by Carolus, Koch, and colleagues [10], a factorial structure for the MAILS was confirmed.”

    From Meta AI literacy scale: Further validation and development of a short version (2024) · cites this paper as [10]

    The Analysis (pp. 7–8) reports that the initial factorial model fit insufficiently, while a modified model fit well. The Limitations (p. 10) explicitly calls the approach exploratory because the model was changed and says the identified structure needs testing in independent samples. Thus, saying the structure was “confirmed” overstates the finding. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • The MAILS includes 34 items: 18 items focusing on AI literacy, four focusing on the ability to create AI, and 6 each focusing on AI self-efficacy and AI self-competency when dealing with AI.

    “The MAILS includes 34 items, 18 items focusing on AI literacy, four focusing on the ability to create AI, and 6 each focusing on AI self-efficacy and AI self-competency when dealing with AI.”

    From Meta AI literacy scale: Further validation and development of a short version (2024) · cites this paper as [10]

    The Conclusion (p. 11) identifies AI literacy, Create AI, AI Self-efficacy, and AI Self-competency as distinct parts of the instrument, and the questionnaire appendix (p. 14) lists items under these headings. The passages retrieved from our copy do not establish the claimed final counts of 34 items or their 18/4/6/6 breakdown; the 68 items in Measures (p. 6) were retained for subsequent analysis, not stated to be the final scale. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Positive correlations of the MAILS with related constructs as postulated in axiom 1c in German- and English-speaking samples have been shown.

    “Moreover, positive correlations of the MAILS with related constructs as postulated in axiom 1 c in German- and English-speaking samples have been shown [10]:”

    From Meta AI literacy scale: Further validation and development of a short version (2024) · cites this paper as [10]

    The Analysis (p. 9) and Discussion (p. 10) report positive correlations between MAILS competencies and related measures, including positive attitudes toward AI. But Limitations and Future Work (p. 10) says the sample was limited to German-speaking people in Germany or Austria; this paper does not establish the claimed correlations in an English-speaking sample. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • Confirmatory analyses of two independent samples had already confirmed the factorial structure of the MAILS.

    “Beginning with axiom 1a (factorial structure), confirmatory analyses of two independent samples had confirmed the factorial structure of the MAILS already ([10]; Koch et al., sub.).”

    From Meta AI literacy scale: Further validation and development of a short version (2024) · cites this paper as [10]

    The Analysis section (pp. 7–8) reports a confirmatory factor analysis and good fit for a modified MAILS model. But Limitations and Future Work (p. 10) calls the approach exploratory and says testing the factorial structure in independent samples is future work. This source therefore does not support the claim that independent samples had already confirmed the structure. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Technology-competence mean is incompatible with its median and maximum

    Major

    With a median of 4.29 and a maximum of 5.00, the mean cannot exceed 4.645. The reported mean of 4.83 is therefore impossible for this sample.

    Reported 4.83 · recomputed 4.645

  • Geographic limitation excludes participants reported to live elsewhere

    Minor

    The limitations describe the sample as German-speaking individuals living in Germany or Austria, but the sample description places only 77% in Germany and 7% in Austria. The remaining 16% lived elsewhere.

  • Initial AI-literacy item count differs between abstract and methods

    Minor

    The abstract says 60 AI-literacy items were derived, plus 12 psychological-competency items. The methods instead say 72 AI-literacy items were generated, plus 12 additional items, for a total of 84.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. Bandura, 2006, Guide for constructing self-efficacy scales, Self-efficacy beliefs of adolescents, 5, 307
  2. Kim, 2022, The artificial intelligence literacy scale for middle school students, Journal of the Korea Society of Computer and Information, 27, 225
  3. Ng, 2022, An examination on primary students' development in ai literacy through digital story writing, Computers & Education: Artificial Intelligence, 3
  4. Polanco-Levicán, 2022, Digital interaction literacy model – conceptualizing competencies for literate interactions, International Journal of Environmental Research and Public Health, 19, 1

Cited by

Sorted by citations

Related

Authors
Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., & Wienrich, C.
Affiliations
University of Würzburg (DE)
Corresponding author
Koch, Martin J.
ORCID
Venue
Computers in Human Behavior: Artificial Humans
Publisher
Elsevier BV
Published
2023-08
Volume
1
Issue
2
Pages
100014
ISSN
2949-8821 (ISSN-L)
DOI
10.1016/j.chbah.2023.100014
OpenAlex
W4387368391
Cited by
488
References
47
Citations by year
2026: 274 · 2025: 160 · 2024: 47 · 2023: 7
FWCI
88.15
Type
journal article
Language
en
Update notices
Open access
Licence
Locations
  • Computers in Human Behavior Artificial Humans (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
Bundesministerium für Arbeit und Soziales (DKI.00.00030.21)
Topics
  • Physical Sciences › Computer Science › Information Systems › Digital literacy in education
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
Keywords
AI literacy, questionnaire development, scale validation, psychological competence, AI self-efficacy, emotion regulation, AI ethics, problem solving, competency model

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-11-05.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-11-05.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review