CoChat

Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research

Open access
Alshaibani, Mariam Hejab, Al-Rahmi, Waleed Mugahed, Tawafak, Ragad M. · 2025 · Contemporary Educational Technology

The DOI matches its Crossref record; 43 of the 45 references it cites were found in a registry or an index; 2 of the 17 sources its claims cite were checked for support.

Abstract

This study introduces and validates the artificial intelligence anxiety scale (AIAS), a novel instrument designed to measure researchers’ anxieties when employing artificial intelligence (AI) tools in academic writing. As AI technologies rapidly infiltrate scholarly work, however, the primary concern grows about their ethical implications, impact on traditional research skills, and the lack of institutional readiness issues that remain underexplored in existing literature. Addressing this critical gap, the AIAS offers a novel framework grounded in real-world academic concerns. Using an inductive approach, data were collected from 219 faculty members and graduate students at Taif University, Saudi Arabia, revealing four core dimensions of AI-related anxiety: (1) concerns about the accuracy of AI outputs, (2) fear of committing plagiarism, (3) lack of institutional guidelines, and (4) fear of losing research skills. Exploratory and confirmatory factor analyses confirmed the existence of these factors, and reliability testing indicated robust internal consistency. By offering the first validated tool specifically tailored to measure AI-related anxiety in academic contexts, this study provides a significant resource for researchers and institutions. Its application can guide universities in devising training and support initiatives to ensure the ethical and practical integration of AI, thereby sustaining the integrity of traditional research competencies and enhancing the overall quality and credibility of academic endeavors.

Overview

CoChat Summary

TL;DR

This study develops and validates the Artificial Intelligence Anxiety Scale (AIAS) to measure researchers’ anxieties about using AI tools in academic writing. Using EFA and CFA with 219 faculty members and graduate students at Taif University, Saudi Arabia, it identifies four anxiety dimensions and reports acceptable reliability and model fit.

Key findings

  • EFA produced four factors explaining 66.35% of total variance: concern about AI output accuracy (39.85%), fear of committing plagiarism (12.21%), lack of institutional guidelines (8.72%), and fear of losing research skills (5.54%); KMO was 0.90 and Bartlett’s test was significant (p < 0.01).
  • CFA supported the four-factor model: WLSMV χ² = 2.621 (34, N = 219, p < .001), CFI and TLI = 0.96, RMSEA = 0.08 (90% CI: 0.08–0.11), WRMR = 1.074; standardized factor loadings ranged from 0.56 to 0.93, and latent-factor correlations ranged from 0.49 to 0.79 (p < .001).
  • Reliability and validity indices were acceptable: Cronbach’s alpha ranged from 0.818 to 0.929, CR from 0.852 to 0.945, AVE from 0.580 to 0.741, HTMT values were below 0.85, and VIF values ranged from 1.427 to 4.488.
  • Path coefficients showed positive relationships among anxiety dimensions: F1 → F2 = 0.572, F2 → F3 = 0.440, and F4 → F3 = 0.510; R² values ranged from 0.260 to 0.478.

Methods

  • Scale development used an inductive approach: 23 initial items were generated from researcher responses, reviewed by 10 experts, items with ≥80% agreement retained, 3 items deleted, leaving 20 items; the initial version was piloted with 15 participants outside the main sample.
  • Participants were 219 faculty members and graduate students at Taif University, Saudi Arabia, recruited via university mailing lists, social media, and academic forums; approximately 400 were invited, yielding a 54.8% response rate. Graduate students were 84.9% (n = 186) and faculty 15.1% (n = 33); disciplines were humanities 73.1%, scientific 21.9%, and medical 5.0%.
  • Analyses included EFA in SPSS using principal component analysis with Promax rotation, CFA with WLSMV estimation and a polychoric covariance matrix, and PLS-SEM for reliability, convergent validity, discriminant validity, collinearity, and R² assessment.

Limitations

  • The paper states the study was conducted in a specific regional context (Taif University, Saudi Arabia) and calls for cross-cultural and institutional validation; it also notes future multi-group CFA is needed to test measurement invariance across demographic variables.
  • The paper states that relatively lower AVE values in some constructs suggest items may not fully capture conceptual richness or distinctiveness, possibly indicating overlapping constructs and a potential threat to discriminant validity; lower R² values (below 0.5 for specific dependent constructs) indicate external factors not included in the model.
  • The text makes evident additional constraints: the sample was non-probability (“amenity sampling”), cross-sectional, self-report, and predominantly graduate students, which limits generalizability beyond the sampled roles, disciplines, and setting.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.30935/cedtech/17309

Cites

43 of 45 references verified0 citation errors · 2 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Conceptual research model (Source: Authors)
Figure 1 Conceptual research model (Source: Authors)CC-BY · From the PDF
Figure 2 Scree plot of the EFA (Source: Authors)
Figure 2 Scree plot of the EFA (Source: Authors)CC-BY · From the PDF
Figure 3 Model results (Source: Authors)
Figure 3 Model results (Source: Authors)CC-BY · From the PDF

Verification

Claims the paper makes

0 Valid · 2 Not valid · 12 Not checked · 59 Own results and interpretations

12 more claims

Create a free account to see all 73 claims and every source verdict.

See all claims — free account
  • Not yet classified. 41
  • It may state two separate claims as one. 11
  • Probably not a substantial claim. 6

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 45 references

Research works printed but not linked: 12

  1. Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.
  2. Clustering, A. S. (2019). International Arab Conference on Information Technology (ACIT). Higher Education, 165, Article 170.
  3. DeVellis, R. F. (2016). Scale development: Theory and applications (4th ed.). SAGE.
  4. Eldow, A., Shakir, M., Talab, M. A., Muttar, A. K., & Tawafak, R. M. (2006). Literature review of authentication layer for public cloud computing: A meta-analysis. Journal of Theoretical and Applied Information Technology, 97, Article 12.
  5. Evelyn, T. (2021). Surfacing anger and anxiety in graduate research writing. Journal of Multi-Disciplinary Studies, 45(3), 55–70.
  6. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
  7. Hair Jr, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage Learning.
  8. Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). The Guilford Press.
  9. Lei, P.-W., & Shiverdecker, L. K. (2020). Performance of fit indices in multiple-group confirmatory factor models with categorical outcomes. Structural Equation Modeling: A Multidisciplinary Journal, 27(3), 439–461.
  10. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
  11. Wen, B., Xu, C., Wolfe, R., Wang, L. L., & Howe, B. (2024). Mitigating overconfidence in large language models: A behavioral lens on confidence estimation and calibration. In NeurIPS 2024 Workshop on Behavioral Machine Learning.
  12. Zhou, X., Smith, L., & Patel, R. (2023). Exploring ethical and psychological dimensions of AI in academic writing. Technology and Society Review, 12(4), 76–89.

Cited by

See all 12

Metadata

Authors
Alshaibani, M. H., Al-Rahmi, W. M., & Tawafak, R. M.
Affiliations
  • Taif University (SA)
  • Dar Al Uloom University (SA)
  • Al-Buraimi University College (OM)
Corresponding author
Al-Rahmi, Waleed Mugahed
Venue
Contemporary Educational Technology
Published
2025-10-20
Volume
17
Issue
4
Pages
ep595
ISSN
1309-517X (ISSN-L)
OpenAlex
W4415360798
Cited by
12
References
34
Citations by year
2026: 12
FWCI
12.48
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
Contemporary Educational Technology (publishedVersion, cc-by)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Social Sciences › Education › Academic integrity and plagiarism
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
AI anxiety, AIA, academic writing, artificial intelligence in research, plagiarism concerns, institutional guidelines, research skills, scale validation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.