CoChat
Sign Up
Cited by 52Digital literacy in educationArtificial Intelligence in EducationGender and Technology in Education

Meta AI literacy scale: Further validation and development of a short version

Open access
Koch, Martin J., Carolus, Astrid, Wienrich, Carolin, Latoschik, Marc E.·2024·Heliyon
DOI 10.1016/j.heliyon.2024.e39686PMC PMC11544068
Open this paper in CoChat — free account

Abstract

The concept of AI literacy, its promotion, and measurement are important topics as they prepare society for the steadily advancing spread of AI technology. The first purpose of the current study is to advance the measurement of AI literacy by collecting evidence regarding the validity of the Meta AI Literacy Scale (MAILS) by Carolus and colleagues published in 2023: a self-assessment instrument for AI literacy and additional psychological competencies conducive for the use of AI. For this purpose, we first formulated the intended measurement purposes of the MAILS. In a second step, we derived empirically testable axioms and subaxioms from the purposes. We tested them in several already published and newly collected data sets. The results are presented in the form of three different empirical studies. We found overall evidence for the validity of the MAILS with some unexpected findings that require further research. We discuss the results for each study individually and also together. Also, avenues for future research are discussed. The study's second purpose is to develop a short version (10 items) of the original instrument (34 items). It was possible to find a selection of ten items that represent the factors of the MAILS and show a good model fit when tested with confirmatory factor analysis. Further research will be needed to validate the short scale. This paper advances the knowledge about the validity and provides a short measure for AI literacy. However, more research will be necessary to further our understanding of the relationships between AI literacy and other constructs.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper provides further validity evidence for the 34-item Meta AI Literacy Scale (MAILS) and develops a 10-item short version. Across four studies, most nomological-network axioms were supported, though the AI self-competency factor showed unexpected correlation patterns and the short version still requires independent validation.

Key findings

  • In Study I (N = 300 German-speaking), prior AI use at work, school/university, and everyday life correlated significantly with nearly all MAILS scales/subscales; the only exception was school/university AI use with emotion regulation. Example correlations: total AIL with work use .35*, school use .29*, everyday use .34*.
  • In Study II (N = 149 English-speaking), positive attitude toward AI correlated with all MAILS scales except persuasion literacy (e.g., total AIL .55*), while negative attitude correlated only with AI self-efficacy (−.17*), Use AI (−.17*), emotion regulation (−.18*), and problem-solving (−.17*). MAILS total AIL correlated strongly with other AI literacy measures, e.g., MAIRS-MS Ability .93* and Pinski & Benlian total AIL .85*.
  • In Study III (N = 120 German-speaking), the AI literacy test theta correlated with total AIL .21*, AI self-competency .24, Know & understand AI .25, and emotion regulation .41*. Prior use of AI and learning about AI correlated with most MAILS scales except AI self-competency and its subscales.
  • The 10-item short version showed good CFA fit; the selected third model had χ²(29) = 154.750, p < .001, RMSEA = .090 (90% CI .076, .104), SRMR = .038, TLI = .936, CFI = .958. Items 10, 13, 20, 28, 30, and 33 were selected across all models.

Methods

  • The validation used a nomological-network approach: intended uses were specified, testable axioms/subaxioms derived, and tested across existing and newly collected data sets. Studies I–III tested axioms with Pearson correlations and confirmatory factor analyses; Study IV developed the short version.
  • Samples: Study I N = 300 German-speaking Prolific participants; Study II N = 149 English-speaking participants; Study III N = 120 German-speaking participants; Study IV combined N = 653 participants from Study I, Study II/full Koch et al. data, and additional university-course samples. Data were collected online via SoSci-Survey and Prolific.
  • Measures included the MAILS on an 11-point Likert scale, other AI literacy self-report scales, the General Attitudes Towards Artificial Intelligence Scale, prior AI use and learning items, and the Hornberger et al. AI literacy test analyzed with a unidimensional 3-PL IRT model. Short-version items were selected using corrected item-scale correlations and robust ML-CFA loadings on subfactors and superordinate factors.

Limitations

  • Stated: no structured translation methods were used for the English and German versions, so equivalence is uncertain; the AI literacy test covers only knowledge, not the full range of AI literacy abilities; axioms 1e and 2c remain untested; practical benefits and predictive validity of the MAILS have not been demonstrated; the short version needs an identical validation process.
  • Stated: Study I’s “use AI” item is vague, and some participants may not have been in school/university or able to use AI at work, reducing accuracy.
  • Evident from the text: some comparison AI literacy scales had low internal consistencies (e.g., Wang et al. Awareness α = .50, Ethics α = .35), and the validity of those other self-report instruments had mostly only been tested regarding factorial structure.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.heliyon.2024.e39686

Cites

34 of 35 references verified1 citation errors · 0 not checkable

Statistics

Claims the paper makes

16 of 27 cited sources checked · 8 supported · 1 flagged · accuracy not yet measured

  • Premisenone given

    The validity of self-report measurement instruments for AI literacy [11,12,14,15] has only been tested regarding their factorial structure.

    “The validity of the self-report measurement instruments for AI literacy [11,12,14,15] has only been tested regarding their factorial structure.”
    Quote in the paperSubstantial claim1 of 4 sources checked
    • cites [11] Karaca, 2021, Medical artificial intelligence readiness scale for medical students (MAIRS-MS) – development, validity and reliability study, BMC Med. Educ. source

      The Results > Validity section reports exploratory factor analysis, but it also describes expert review and student interviews used to assess face and content validity. This contradicts the claim that validity was tested only for factorial structure. No page number is available in our copy. (LLM, conf=0.90)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [12] Laupichler, 2023, Development of the Scale for the assessment of non-experts’ AI literacy – an exploratory factor analysis, Comput. Hum. Behav. Rep source
    • cites [14] Pinski, 2023, AI literacy - towards measuring human competency in artificial intelligence
    • cites [15] Wang, 2023, Measuring user competence in using artificial intelligence: validity and reliability of artificial intelligence literacy scale, Behav. Inf. Technol. source
  • Premisecitation

    In the first study by Carolus, Koch, and colleagues, a factorial structure for the MAILS was confirmed.

    “In the first study by Carolus, Koch, and colleagues [10], a factorial structure for the MAILS was confirmed.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source

      The Analysis (pp. 7–8) reports that the initial factorial model fit insufficiently, while a modified model fit well. The Limitations (p. 10) explicitly calls the approach exploratory because the model was changed and says the identified structure needs testing in independent samples. Thus, saying the structure was “confirmed” overstates the finding. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Premisecitation

    Empirical data in an initial factor-analytical review indicate a slightly different structure of the MAILS items than the conceptualization by Ng et al.

    “However, empirical data in an initial factor-analytical review indicate a slightly different structure of the items as described in the following paragraph [10].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source
  • Premisecitation

    The second sample (N = 219) also confirmed the measurement model of the MAILS as it showed a good model fit.

    “The second sample (N = 219) also confirmed the measurement model of the MAILS as it showed a good model fit (χ2(513) = 781.49, p < .001, CFI = .95, RMSEA = .05, 95 %-CI [.05, .06], SRMR = .06).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source

      Section 3.3 (p. 8) reports good fit for a modified MAILS measurement model, but gives different fit statistics: χ²(513) = 886.87, CFI = .926, RMSEA = .057, and SRMR = .079. The paper describes data from 300 participants (Empirical Study, p. 6), not a second sample of 219; section 4.1 (p. 10) says testing the structure in independent samples remains future work. (LLM, conf=0.90)

      Experimental — accuracy not yet measured
  • Premiseown data

    Structural equation modelling revealed that the ability to create AI was not a subdimension of AI literacy but only a correlate.

    “In contrast to the original concept of Ng et al. [3], structural equation modelling revealed that the ability to create AI was not a subdimension of AI literacy but only a correlate.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source
  • Classified as probably not a substantial claim. 10
  • May combine two claims. 28

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The MAILS includes 34 items: 18 items focusing on AI literacy, four focusing on the ability to create AI, and 6 each focusing on AI self-efficacy and AI self-competency when dealing with AI.

    “The MAILS includes 34 items, 18 items focusing on AI literacy, four focusing on the ability to create AI, and 6 each focusing on AI self-efficacy and AI self-competency when dealing with AI.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source

      The Conclusion (p. 11) identifies AI literacy, Create AI, AI Self-efficacy, and AI Self-competency as distinct parts of the instrument, and the questionnaire appendix (p. 14) lists items under these headings. The passages retrieved from our copy do not establish the claimed final counts of 34 items or their 18/4/6/6 breakdown; the 68 items in Measures (p. 6) were retained for subsequent analysis, not stated to be the final scale. (LLM, conf=0.78)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    The Meta AI Literacy Scale (MAILS) is a self-assessment instrument for measuring AI literacy and additional psychological competencies (problem-solving, learning, self-efficacy, persuasion literacy) that are assumed to support successful interaction with AI.

    “The Meta AI Literacy Scale (MAILS; [10]) is a scale for measuring AI literacy and other psychological competencies (problem-solving, learning, self-efficacy, persuasion literacy) that are assumed to support successful interaction with AI [[17], [18], [19]].”
    Quote in the paperSubstantial claim1 of 4 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source
    • cites [17] Carolus, 2023, Digital interaction literacy model – conceptualizing competencies for literate interactions with voice-based AI systems, Comput. Educ. Artif. Intell. source
    • cites [18] Cetindamar, 2022, Knight, explicating AI literacy of employees at digital workplaces, IEEE TranEng. Manag. source
    • cites [19] Dai, 2020, Promoting students' well-being by developing their readiness for the artificial intelligence age, Sustainability source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The design of the MAILS was originally based on the AI literacy conceptualization by Ng et al. concerning AI literacy (not the supplementary psychological competencies) and the taxonomy of Bloom.

    “The design of the instrument was originally based on the AI literacy conceptualization by Ng et al. [3] concerning AI literacy (not concerning the supplementary psychological competencies) and the taxonomy of Bloom [20].”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [3] Ng, 2022, Conceptualizing AI literacy: an exploratory review, Comput. Educ. Artif. Intell. source
    • cites [20] Bloom, 1956

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisereasoning

    The MAILS includes the ability to create AI as a related construct and measures psychological competencies expected to be helpful or needed for prolonged use of AI.

    “To summarize, unlike other available measures of AI literacy, the MAILS includes the ability to create AI as a related construct and also measures psychological competencies that are expected to be helpful or needed for the prolonged use of AI.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The prior findings regarding negative attitudes toward AI were mixed.

    “The prior findings regarding negative attitudes toward AI were mixed [10].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source

  • May combine two claims.

    Premisecitation

    The ability to create AI was not correlated with control over new technology and negatively correlated with competence when dealing with new technologies.

    “Interestingly, the ability to create AI was not correlated with control over new technology and negatively correlated with competence when dealing with new technologies.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source

      The Analysis (p. 9) says Create AI did not correlate with perceived control over new technology, supporting the first part. It says self-assessed competence in using new technologies correlated positively—not negatively—with Create AI. (LLM, conf=0.90)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    AI literacy, AI self-competency, and AI self-efficacy correlated significantly positive with most of the factors of openness to technology (except AI self-efficacy and competence when dealing with new technology) and positive attitude towards AI.

    “AI literacy, AI self-competency, and AI self-efficacy correlated significantly positive with most of the factors of openness to technology (except AI self-efficacy and competence when dealing with new technology) and positive attitude towards AI.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] Carolus, 2023, Mails - meta AI literacy scale: development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies, Comput. Hum. Behav. Artif. Hum. source

      Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    In the Study III sample, all scales of the MAILS showed good or very good internal consistencies (α < .8) except emotion regulation, which had acceptable internal consistency (α = .77).

    “In this sample, all scales of the MAILS showed good or very good internal consistencies (α < .8) except emotion regulation which Cronbach's alpha was still acceptable (α = .77).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The second and third models also showed good fits.

    “The second (χ2(29) = 163.349, p < .001, RMSEA = .093 (90%-CI: .079, .107), SRMR = .040, TLI = .930, CFI = .955) and third model (χ2(29) = 154.750, p < .001, RMSEA = .090 (90%-CI: .076, .104), SRMR = .038, TLI = .936, CFI = .958) also showed good fits.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The relationships between AI literacy and prior use of AI and learning about AI were not found for AI Self-competency and the subscales of emotion regulation and persuasion literacy.

    “However, these relationships were not found for AI Self-competency and the subscales of emotion regulation and persuasion literacy.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Most correlations were of small effect size, only the correlation of emotion regulation and the test scores were of medium size.

    “Most correlations were of small effect size, only the correlation of emotion regulation and the test scores were of medium size.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The current studies were able to further our understanding of axioms 2a (correlation with the prior use of AI) and 2b (correlation with learning about AI).

    “The current studies were also able to further our understanding of axioms 2a (correlation with the prior use of AI) and 2b (correlation with learning about AI).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The other items differed between the models, but there was still partial overlap.

    “The other items differed between the models, however, there was still partial overlap (Appendix A).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    It was possible to find a selection of ten items that represent the factors of the MAILS and show a good model fit when tested with confirmatory factor analysis.

    “It was possible to find a selection of ten items that represent the factors of the MAILS and show a good model fit when tested with confirmatory factor analysis.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    All scales of the MAILS were significantly correlated with the prior use of AI in Study I except emotion regulation, which was not related to the use of AI at work.

    “Interestingly, all scales of the MAILS were significantly correlated with the prior use of AI in Study I except emotion regulation which was not related to the use of AI at work.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In Study I, the average score for use of AI at work was M = 1.84 (SD = 2.79), for use of AI at school/university was M = 1.21 (SD = 2.31), and for use of AI in everyday life was M = 3.73 (SD = 3.03).

    “The average scores (and standard deviation) were M = 1.84 (SD = 2.79) for the use of AI at work, M = 1.21 (SD = 2.31) for the use of AI at school/university, and M = 3.73 (SD = 3.03) for everyday life respectively.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Positive relationships were found between AI literacy and prior use of AI and learning about AI.

    “Confirming axioms 2a and 2b, we found positive relationships between AI literacy and the prior use of AI and learning about AI respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The scales for prior use of AI and learning about AI showed good internal consistency (αUse = .86, αLearning = .84).

    “Both scales showed a good internal consistency (αUse = .86, αLearning = .84).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In Study I, some correlations with use of AI at school/university are non-significant and most are descriptively smaller than the correlations with the other two items.

    “Some correlations with use of AI at school/university are non-significant and most are descriptively smaller than the correlations with the other two items.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    We could not confirm the axiom in its completeness as we found no relationship between AI Self-competency and a positive attitude towards AI but instead found a correlation between AI self-efficacy with a negative attitude towards AI.

    “Unlike expected, we could not confirm the axiom in its completeness as we found no relationship between AI Self-competency and a positive attitude towards AI but instead found a correlation between AI self-efficacy with a negative attitude towards AI.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationown data

    We interpret these findings as a call for more empirical research on the exact relationship between AI literacy and attitudes towards rather than as a counterargument for the validity of the MAILS.

    “We carefully interpret these findings rather as a call for more empirical research on the exact relationship between AI literacy and attitudes towards than consider them a counterargument for the validity of the MAILS.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    We do not assume the correlation of emotion regulation with the test results to be an argument against the validity of the scale.

    “However, we do not assume this to be an argument against the validity of the scale.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    So far, we know that the MAILS measures AI literacy and can detect interindividual and intraindividual differences, but not yet whether these have practical relevance.

    “So far, we know that the MAILS measures AI literacy and can detect interindividual and intraindividual differences, but not yet whether these have practical relevance.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Knowledge is only one limited part of AI literacy, which also includes different abilities and competencies.
    • Rests on an assumption: For the abilities and competencies of AI literacy, so far no objective test exists.

  • May combine two claims.

    Interpretationown data

    The AI self-competence scale showed deviating correlation patterns, which indicates that the skills assessed in this scale should be viewed separately from the rest of the AI competence.

    “Contrary to expectations, the AI self-competence scale showed deviating correlation patterns, which indicates that the skills assessed in this scale should be viewed separately from the rest of the AI competence.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    In Study II, internal consistencies of all scales and subscales of the MAILS were good (α > .80) or high (α > .90) except for persuasion literacy which was acceptable (α = .74).

    “The internal consistencies of all scales and subscales of the MAILS were good (α > .80) or high (α > .90) except for persuasion literacy which still was acceptable (α = .74).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The study found overall evidence for the validity of the MAILS with some unexpected findings that require further research.

    “We found overall evidence for the validity of the MAILS with some unexpected findings that require further research.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The German and English versions of the MAILS measure the same construct equivalently.

  • May combine two claims.

    Study design and conductown data

    In Study II, internal consistencies for positive attitude (α = .91) and negative attitude (α = .79) were high and acceptable, respectively.

    “Internal consistencies for both scales were high for positive attitude (α = .91) and acceptable for negative attitude (α = .79).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The validity of the MAILS is supported because the instrument is capable of detecting differences between individuals who use AI more often compared to those who use it less often and between those who learn about AI more often compared to those who learn about it less often.

    “Again, the validity of the MAILS is supported as the instrument is capable of detecting differences between individuals who use AI more often compared to those who use it less often and between those who learn about AI more often compared to those who learn about it less often.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The measures from the MAILS are indicators for the individual manifestation of AI literacy and related psychological constructs.

  • May combine two claims.

    Interpretationreasoning

    Given the present and prior evidence, the MAILS is a valid measurement instrument that can be used to assess AI literacy and measure interindividual and intraindividual differences.

    “In total, given the present and prior evidence, we assume that the MAILS is a valid measurement instrument that can be used to assess AI literacy and measure interindividual and intraindividual differences.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The measures from the MAILS are indicators for the individual manifestation of AI literacy and related psychological constructs.
    • The authors state this as an assumption: The measures from the MAILS can be used to evaluate interindividual differences in AI literacy and related psychological competencies.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    This publication contributes to the measurement of AI literacy by providing multifaceted evidence for the validity of the MAILS and highlights the need to continue the validation process in future work.

    “In sum, this publication contributes to the measurement of AI literacy by providing multifaceted evidence for the validity of the MAILS and highlights the need to continue the validation process in future work.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The measures from the MAILS are indicators for the individual manifestation of AI literacy and related psychological constructs.
    • The authors state this as an assumption: The measures from the MAILS can be used to evaluate interindividual differences in AI literacy and related psychological competencies.

  • May combine two claims.

    Interpretationown data

    Following the current paper, we can draw positive conclusions regarding the goals as it was possible to confirm most axioms and find a suitable selection of items for a short version.

    “Following the current paper, we can draw positive conclusions regarding these goals as it was possible to confirm most axioms and find a suitable selection of items for a short version.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The measures from the MAILS are indicators for the individual manifestation of AI literacy and related psychological constructs.
    • The authors state this as an assumption: The measures from the MAILS can be used to evaluate interindividual differences in AI literacy and related psychological competencies.

  • May combine two claims.

    Interpretationcitation

    We expect that emotion regulation facilitates knowledge acquisition independently of the topic of AI, explaining this correlation separately from AI self-competency.

    “Instead, we expect that emotion regulation facilitates knowledge acquisition independently of the topic of AI [35] explaining this correlation separately from AI self-competency.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [35] MacCann, 2020, Emotional intelligence predicts academic performance: a meta-analysis, Psychol. Bull. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The Study I sample showed a rather low use of AI.

    “The sample showed a rather low use of AI.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation

    The correlation of emotion regulation with the AI literacy test scores might be due to emotion regulation being important for knowledge acquisition independently of the topic of AI.

    “Presumably, this might be due to the circumstance that emotion regulation is considered to be important for knowledge acquisition independently of the topic of AI [35].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] MacCann, 2020, Emotional intelligence predicts academic performance: a meta-analysis, Psychol. Bull. source
34 of 34

  • The paper relies on this without support.

    The 34-item MAILS is too long and limits its economic use, motivating a short version.

    “The MAILS consists of 34 items, which limits its economic use.”

  • The paper relies on this without support.

    The German and English versions of the MAILS measure the same construct equivalently.

    “For the first purpose, we combined evidence from three data sets from German- and English-speaking samples.”

  • The paper relies on this without support.

    No standardized procedure for the development of a short version is described in the scientific literature.

    “As no standardized procedure for the development of a short version is described in the scientific literature, we rely on the well-established approach to select the best item/items of each sub-factor of the questionnaire.”

  • The paper relies on this without support.

    Research concerning evidence for the validity of the MAILS so far was scarce.

    “Research concerning evidence for the validity of the MAILS so far was scarce.”

  • The authors state this as an assumption.

    The measures from the MAILS are indicators for the individual manifestation of AI literacy and related psychological constructs.

    “The first five axioms referred to the general assumption that the measures from the MAILS are indicators for the individual manifestation of AI literacy and related psychological constructs.”

  • The authors state this as an assumption.

    The measures from the MAILS can be used to evaluate interindividual differences in AI literacy and related psychological competencies.

    “The second set of axioms referred to the assumption that the measures from the MAILS can be used to evaluate interindividual differences in AI literacy and related psychological competencies.”

  • The paper relies on this without support.

    Knowledge is only one limited part of AI literacy, which also includes different abilities and competencies.

    “Knowledge, however, is only one limited part of AI literacy which also includes different abilities and competencies.”

  • The paper relies on this without support.

    For the abilities and competencies of AI literacy, so far no objective test exists.

    “For these abilities and competencies, so far no objective test exists.”

  • The paper relies on this without support.

    The validity of the other AI literacy self-report scales has not yet been tested besides the factorial structure.

    “This finding, however, is limited in so far as the validity of the other AI literacy self-report scales has not yet been tested besides the factorial structure.”
9 of 9

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Consistentχ²(513) = 886.87reported p < .001 · recomputed p < .001

    “χ2(513) = 886.87, p < .001”

  • Consistentχ²(513) = 781.49reported p < .001 · recomputed p < .001

    “χ2(513) = 781.49, p < .001”

  • Consistentχ²(29) = 171.392reported p < .001 · recomputed p < .001

    “χ2(29) = 171.392, p < .001”

  • Consistentχ²(29) = 163.349reported p < .001 · recomputed p < .001

    “χ2(29) = 163.349, p < .001”

  • Consistentχ²(29) = 154.75reported p < .001 · recomputed p < .001

    “χ2(29) = 154.750, p < .001”

5 of 5

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • AI self-competency is incorrectly described as unrelated to positive attitude

    Major

    The discussion says there was no relationship, but Table 3 reports a significant positive correlation of .35 between AI self-competency and positive attitude toward AI.

  • Study II reverses the definitions of axioms 1b and 1c

    Minor

    Study II labels attitude correlations as axiom 1b and correlations with other AI literacy scales as axiom 1c. The paper defines axiom 1b as concerning other AI literacy scales and axiom 1c as concerning attitude and openness to technology.

  • Study I discussion overlooks a nonsignificant correlation

    Minor

    The discussion says prior use in all three settings correlated significantly with all MAILS scales and subscales. The results explicitly identify an exception: school/university use was not correlated with emotion regulation.

  • General discussion assigns the emotion-regulation exception to the wrong setting

    Minor

    The general discussion says emotion regulation was not related to AI use at work. Table 2 reports a significant work-use correlation (.17**) and a nonsignificant school/university-use correlation (.09).

  • Limitations call tested axiom 1d uninvestigated

    Minor

    The limitations say axiom 1d still needs investigation, whereas Study III reports testing and confirming axiom 1d using an AI literacy test. The paper elsewhere identifies 1e, not 1d, as untested.

5 of 5

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Bandura, 2006, Guide for constructing self-efficacy scales, Self-Effic, Beliefs Adolesc, 5, 307

Cited by

Sorted by citations

Related

Authors
Koch, M. J., Carolus, A., Wienrich, C., & Latoschik, M. E.
Affiliations
University of Würzburg (DE)
Corresponding author
Koch, Martin J.
ORCID
Venue
Heliyon
Publisher
Elsevier BV
Published
2024-11
Volume
10
Issue
21
Pages
e39686
ISSN
2405-8440 (ISSN-L)
DOI
10.1016/j.heliyon.2024.e39686
PMC
PMC11544068
OpenAlex
W4403634296
Semantic Scholar
1178da6e4abdeec2899fee0654147f9760f341ed
Cited by
44
References
34
Citations by year
2026: 40 · 2025: 4
FWCI
16.50
Type
journal article
Language
en
Open access
Licence
Locations
  • Heliyon (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Online Publication Service of Würzburg University (Würzburg University) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Federal Republic of Germany Ministry for Labour and Social Affairs (DKI.00.00030.21)
Topics
  • Physical Sciences › Computer Science › Information Systems › Digital literacy in education
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Gender Studies › Gender and Technology in Education
Keywords
AI literacy, scale validation, short-form scale development, self-assessment tool, factor structure, measurement invariance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-11-04.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-11-04.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review