Critical Evaluation of AI-Generated Academic Information in Higher Education: Empirical Evidence on Trust, AI-Related Digital Literacy and Verification Behaviour

Abstract

Generative artificial intelligence (GenAI) is increasingly used by university students for academic information seeking, explanation, drafting and problem solving. Its usefulness, however, depends not only on ease of use or frequency of interaction but also on users' capacity to evaluate and verify AI-generated information. This study presents an independent secondary-data analysis of a publicly available, anonymised dataset containing responses from 104 university students to 20 five-point Likert-scale items covering perceived ease of use (PEOU), trust in AI, verification behaviour and digital literacy. The analysis examined internal consistency, factorability diagnostics, descriptive statistics, Pearson correlations and multiple linear regression using heteroscedasticity-consistent HC3 standard errors. The four constructs demonstrated acceptable to high internal consistency (Cronbach's α = .782–.898). The KMO measure was .836 and Bartlett's test of sphericity was statistically significant, χ²(190) = 1087.35, p < .001, indicating that the item correlation matrix was suitable for factor-analytic examination. Verification behaviour was positively correlated with AI-related digital literacy (r = .445, p < .001), whereas its bivariate associations with PEOU and trust were not statistically significant. In the multivariable model, AI-related digital literacy showed the largest standardized coefficient (β = .524, p < .001). Trust showed a small negative adjusted association with verification behaviour (β = −.214, p = .048), whereas PEOU and usage frequency were not statistically significant. The model explained 25.3% of the variance in verification behaviour (R² = .253; adjusted R² = .223). The findings suggest that effective academic engagement with GenAI should not be equated with frequent use or perceived ease of use. Instead, evaluative digital-literacy capabilities appear particularly relevant to verification behaviour. Building on these findings, the study proposes the HEAVEN framework (Human-centred Evaluation of AI-generated academic information through Verification, Evidence and Nuanced judgement) as a conceptual framework integrating AI literacy, information verification, evidence appraisal and human judgement in higher education.