Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research
Open accessThe DOI matches its Crossref record; 43 of the 45 references it cites were found in a registry or an index; 2 of the 17 sources its claims cite were checked for support.
Abstract
This study introduces and validates the artificial intelligence anxiety scale (AIAS), a novel instrument designed to measure researchers’ anxieties when employing artificial intelligence (AI) tools in academic writing. As AI technologies rapidly infiltrate scholarly work, however, the primary concern grows about their ethical implications, impact on traditional research skills, and the lack of institutional readiness issues that remain underexplored in existing literature. Addressing this critical gap, the AIAS offers a novel framework grounded in real-world academic concerns. Using an inductive approach, data were collected from 219 faculty members and graduate students at Taif University, Saudi Arabia, revealing four core dimensions of AI-related anxiety: (1) concerns about the accuracy of AI outputs, (2) fear of committing plagiarism, (3) lack of institutional guidelines, and (4) fear of losing research skills. Exploratory and confirmatory factor analyses confirmed the existence of these factors, and reliability testing indicated robust internal consistency. By offering the first validated tool specifically tailored to measure AI-related anxiety in academic contexts, this study provides a significant resource for researchers and institutions. Its application can guide universities in devising training and support initiatives to ensure the ethical and practical integration of AI, thereby sustaining the integrity of traditional research competencies and enhancing the overall quality and credibility of academic endeavors.
Overview
CoChat Summary
TL;DR
This study develops and validates the Artificial Intelligence Anxiety Scale (AIAS) to measure researchers’ anxieties about using AI tools in academic writing. Using EFA and CFA with 219 faculty members and graduate students at Taif University, Saudi Arabia, it identifies four anxiety dimensions and reports acceptable reliability and model fit.
Key findings
- EFA produced four factors explaining 66.35% of total variance: concern about AI output accuracy (39.85%), fear of committing plagiarism (12.21%), lack of institutional guidelines (8.72%), and fear of losing research skills (5.54%); KMO was 0.90 and Bartlett’s test was significant (p < 0.01).
- CFA supported the four-factor model: WLSMV χ² = 2.621 (34, N = 219, p < .001), CFI and TLI = 0.96, RMSEA = 0.08 (90% CI: 0.08–0.11), WRMR = 1.074; standardized factor loadings ranged from 0.56 to 0.93, and latent-factor correlations ranged from 0.49 to 0.79 (p < .001).
- Reliability and validity indices were acceptable: Cronbach’s alpha ranged from 0.818 to 0.929, CR from 0.852 to 0.945, AVE from 0.580 to 0.741, HTMT values were below 0.85, and VIF values ranged from 1.427 to 4.488.
- Path coefficients showed positive relationships among anxiety dimensions: F1 → F2 = 0.572, F2 → F3 = 0.440, and F4 → F3 = 0.510; R² values ranged from 0.260 to 0.478.
Methods
- Scale development used an inductive approach: 23 initial items were generated from researcher responses, reviewed by 10 experts, items with ≥80% agreement retained, 3 items deleted, leaving 20 items; the initial version was piloted with 15 participants outside the main sample.
- Participants were 219 faculty members and graduate students at Taif University, Saudi Arabia, recruited via university mailing lists, social media, and academic forums; approximately 400 were invited, yielding a 54.8% response rate. Graduate students were 84.9% (n = 186) and faculty 15.1% (n = 33); disciplines were humanities 73.1%, scientific 21.9%, and medical 5.0%.
- Analyses included EFA in SPSS using principal component analysis with Promax rotation, CFA with WLSMV estimation and a polychoric covariance matrix, and PLS-SEM for reliability, convergent validity, discriminant validity, collinearity, and R² assessment.
Limitations
- The paper states the study was conducted in a specific regional context (Taif University, Saudi Arabia) and calls for cross-cultural and institutional validation; it also notes future multi-group CFA is needed to test measurement invariance across demographic variables.
- The paper states that relatively lower AVE values in some constructs suggest items may not fully capture conceptual richness or distinctiveness, possibly indicating overlapping constructs and a potential threat to discriminant validity; lower R² values (below 0.5 for specific dependent constructs) indicate external factors not included in the model.
- The text makes evident additional constraints: the sample was non-probability (“amenity sampling”), cross-sectional, self-report, and predominantly graduate students, which limits generalizability beyond the sampled roles, disciplines, and setting.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
0 Valid · 2 Not valid · 12 Not checked · 59 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“High CR estimates suggest that the individual items within each factor measure the underlying construct consistently and with minimal measurement error (Fornell & Larcker, 1981;Mohammadkarimi, 2023).”
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > FINDINGS USING PLS-SEM
What the sources say
- Not checked — no full text of the sourcecites Fornell & Larcker, 1981 source
- Could not verifycites Mohammadkarimi, 2023 sourceExperimental — accuracy not yet measured
The Research design section reports a questionnaire reliability check using Cronbach’s alpha (0.87). It does not discuss composite reliability (CR), factor-level item consistency, or measurement error; searches for those terms found no supporting passage in our copy. No page number was provided for the section. (LLM, conf=0.76)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the Fornell-Larcker criterion (Table 7) confirmed that the AVE for each construct exceeded the correlations between constructs, providing further evidence of discriminant validity.”
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > FINDINGS USING PLS-SEM
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The statistical findings of this study reveal a significant correlation between F2 and F1 (plagiarism and accuracy)”
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > DISCUSSION
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, concerns about the accuracy of AI tools are evident, encompassing worries about the reliability and interpretability of the results produced by AI.”
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > CONCLUSION AND FUTURE RESEARCH
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The third factor is the lack of institutional guidelines, revealing apprehension due to the absence of clear guidance from universities and publishers on the appropriate use of AI in research.”
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > CONCLUSION AND FUTURE RESEARCH
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
12 more claims
Create a free account to see all 73 claims and every source verdict.
- Not yet classified. 41
- It may state two separate claims as one. 11
- Probably not a substantial claim. 6
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > Exploratory factor analysis
“Factor 4 was labelled 'Fear of losing research skills due to frequent use of AI tools' and comprised 4 items”
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > CONCLUSION AND FUTURE RESEARCH
“| q18 | | | | 0.814 | | q19 | | | | 0.879 | | q20 | | | | 0.912 | | q21 | | | | 0.879 | | q22 | | | | 0.898 | | q23 | | | | 0.775 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > FINDINGS USING PLS-SEM
“The Fornell-Larcker criterion revealed that the AVE for each construct was greater than its correlations with other constructs, indicating distinctiveness among the factors.”
§ Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research > CONCLUSION AND FUTURE RESEARCH
“| Factor 2 | 0.676 | 0.812 |”
Citations
Cites
Evaluating structural equation models with unobservable variables and measurement error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800104 · Cited by 109,190
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00149 · Cited by 5,925
Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification
Hamari, J., Koivisto, J., & Sarsa, H. (2014) · 2014 47th Hawaii International Conference on System Sciences · doi:10.1109/hicss.2014.377 · Cited by 5,201
A Review of Scale Development Practices in the Study of Organizations
Hinkin, T. R. (1995) · Journal of Management · doi:10.1177/014920639502100509 · Cited by 3,193
Artificial Intelligence, Automation and Work
Acemoglu, D., & Restrepo, P. (2018) · doi:10.3386/w24196 · Cited by 962
The Quest for Artificial Intelligence
Nilsson, N. J. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511819346 · Cited by 783
Wang, Y.-Y., & Wang, Y.-S. (2022) · Interactive Learning Environments · doi:10.1080/10494820.2019.1674887 · Cited by 716
Yusuf, A., Pervin, N., & Román-González, M. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00453-6 · Cited by 554
Research works printed but not linked: 12
- Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.
- Clustering, A. S. (2019). International Arab Conference on Information Technology (ACIT). Higher Education, 165, Article 170.
- DeVellis, R. F. (2016). Scale development: Theory and applications (4th ed.). SAGE.
- Eldow, A., Shakir, M., Talab, M. A., Muttar, A. K., & Tawafak, R. M. (2006). Literature review of authentication layer for public cloud computing: A meta-analysis. Journal of Theoretical and Applied Information Technology, 97, Article 12.
- Evelyn, T. (2021). Surfacing anger and anxiety in graduate research writing. Journal of Multi-Disciplinary Studies, 45(3), 55–70.
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
- Hair Jr, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage Learning.
- Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). The Guilford Press.
- Lei, P.-W., & Shiverdecker, L. K. (2020). Performance of fit indices in multiple-group confirmatory factor models with categorical outcomes. Structural Equation Modeling: A Multidisciplinary Journal, 27(3), 439–461.
- Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
- Wen, B., Xu, C., Wolfe, R., Wang, L. L., & Howe, B. (2024). Mitigating overconfidence in large language models: A behavioral lens on confidence estimation and calibration. In NeurIPS 2024 Workshop on Behavioral Machine Learning.
- Zhou, X., Smith, L., & Patel, R. (2023). Exploring ethical and psychological dimensions of AI in academic writing. Technology and Society Review, 12(4), 76–89.
Cited by
Özkan, Z., & Eyı, S. (2026) · BMC Medical Education · doi:10.1186/s12909-026-09778-4 · Cited by 2
“The literature has shown that the lack of institutional policies and guidance principles directly affects students' ethical concerns regarding AI, and this constitutes a separate psychometric factor [4].”
Alsenan, S. A., Al-Rahmi, W. M., Alyoussef, I. Y., & Dahri, N. A. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1833033 · Cited by 2
Self-citationHelali, M. M., Mostafa, M. M., Aldawsari, H. K., Ibrahim, A. N., Bukhamseen, A. M., & Nemt-allah, M. A. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1810679 · Cited by 2
“This structural validation mirrors findings from other AI-related scales, where factor-analytic results consistently support theoretically proposed multidimensional frameworks (Maharmah et al., 2025; Alshaibani et al., 2025; Sallam et al., 2025).”
Kadirhanoğulları, M. K., & Köse, E. Ö. (2026) · Scientific Reports · doi:10.1038/s41598-026-53637-7 · Cited by 1
“At this stage, the existing literature was thoroughly reviewed; measurement tools used in similar research on artificial intelligence, readiness, and anxiety were examined (e.g., 44 , 54,55,7,56,43,37,42,40,57 ).”
Girgin, Ç. (2026) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/6243455 · Cited by 2
Metadata
- Authors
- Alshaibani, M. H., Al-Rahmi, W. M., & Tawafak, R. M.
- Affiliations
- Taif University (SA)
- Dar Al Uloom University (SA)
- Al-Buraimi University College (OM)
- Corresponding author
- Al-Rahmi, Waleed Mugahed
- Venue
- Contemporary Educational Technology
- Publisher
- Bastas Publications
- Published
- 2025-10-20
- Volume
- 17
- Issue
- 4
- Pages
- ep595
- ISSN
- 1309-517X (ISSN-L)
- OpenAlex
- W4415360798
- Cited by
- 12
- References
- 34
- Citations by year
- 2026: 12
- FWCI
- 12.48
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Contemporary Educational Technology (publishedVersion, cc-by)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- AI anxiety, AIA, academic writing, artificial intelligence in research, plagiarism concerns, institutional guidelines, research skills, scale validation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


