An expectancy value theory (EVT) based instrument for measuring student perceptions of generative AI

May combine two claims.

The mean for Q22 was 3.92 and for Q23 was 3.86.

“Thus, based on the means of Q22 (mean = 3.92) and Q23 (mean = 3.86), many of the participants believed that the integration of generative AI technologies in higher education will have a positive impact on teaching and learning in the long run and they envision integrating generative AI technologies into their teaching practices in the future.”

May combine two claims.

Participants recognized the value of ChatGPT (Q7-Q17) in various teaching, learning, and working occasions, except for Q11 and Q10, which had means of 3.31 and 3.38 and tended to be more neutral.

“Additionally, they also recognized the value (Q7-Q17) of ChatGPT in various teaching, learning, and working occasions; except for Q11 (”

May combine two claims.

The majority of participants acknowledged the potential advantages of GenAI in the workplace and its capacity to enhance learning outcomes, including improvement of academic performance and development of digital competence.

“The majority of participants acknowledged the potential advantages of GenAI in the workplace and its capacity to enhance learning outcomes, encompassing the improvement of academic performance and the development of digital competence.”

May combine two claims.

The CR values for the knowledge scale, value scale, and cost scale are 0.822, 0.763, and 0.873 respectively.

“As shown in Table 7, though some of the AVE values are lesser and closer to 0.50, yet with the CR value of the knowledge scale, value scale, and cost scale as 0.822, 0.763, and 0.873 accordingly, which exceed the cutoff point of 0.60, it may be able to conclude that the convergent validity is acceptable (Fornell & Larcker, 1981).”

May combine two claims.

EVT-related factors-knowledge, perceived value (including attainment, intrinsic, and utility values), and perceived cost-are all significantly correlated with students' intention to use GenAI in higher education.

“The findings from Table 7 demonstrates that EVT-related factors, such as knowledge, perceived value (including attainment, intrinsic, and utility values), and perceived cost, are all significantly correlated with students' intention to use GenAI in higher education.”

May combine two claims.

Perceived value has the strongest positive correlation with intention to use, while perceived cost has a weak negative correlation.

“The perceived value has the strongest positive correlation with intention to use, while the perceived cost has a weak negative correlation.”

May combine two claims.

Students were concerned that the use of GenAI could undermine the value of a university education, deprive them of opportunities to interact with others, and hinder the development of transferable skills.

“As shown in the responses to the questionnaire, students were concerned that the use of GenAI could undermine the value of a university education, deprive them of the opportunities to interact with others, and hinder the development of transferable skills.”

May combine two claims.

For perceived value, the TLI is 0.948 and the CFI is higher than 0.95.

“TLI of perceived value (= 0.948) are less than 0.95, yet still higher than 0.90 and its CFI is higher than 0.95.”

May combine two claims.

The HTMT ratios range from 0.660 to 0.002, all lower than the threshold of 0.85.

“The values of HTMT ratios (see Table 8) further denote good discriminant validity, as all the values range from 0.660 to 0.002, lower than the threshold of 0.85 (Henseler et al., 2015).”

• cites Henseler et al., 2015 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Cronbach's alphas for the 5 knowledge, 11 perceived value, and 4 perceived cost items are 0.812, 0.876, and 0.746 respectively.

“Cronbach's alphas for the 5 knowledge, 11 perceived value, and 4 perceived cost items are 0.812, 0.876, and 0.746.”

Classified as probably not a substantial claim.

Missing data were imputed using the multiple imputation procedure in SPSS.

“The missing data were imputed by the multiple imputation procedure in SPSS.”

Classified as probably not a substantial claim.

Expectancy-value theory (EVT) posits that an individual's motivation to engage in a behaviour is influenced by their expectations of success and the value they place on the behaviour.

“EVT posits that individual's motivation to engage in a behaviour is influenced by their expectations of success and the value they place on the behaviour (Wigfield, 1994;Wigfield & Eccles, 2000), working as a framework to examine students' intentions and perceptions from knowledge, perceived value, and perceived cost of the technology in the early stage.”

• cites Wigfield, 1994 sourceNot checked yet

  Not checked in the free run.

• cites Wigfield & Eccles, 2000 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The validity of constructs was measured by average variance extracted (AVE), composite reliability (CR), and heterotrait-monotrait (HTMT) ratio.

“The validity of constructs was measured by Average variance extracted (AVE), composite reliability (CR) (Fornell & Larcker, 1981;Hair et al., 2015), and Heterotrait-monotrait (HTMT) ratio (Henseler et al., 2015).”

• cites Fornell & Larcker, 1981 sourceNot checked yet

  Not checked in the free run.

• cites Hair et al., 2015Not checked yet

  Not checked in the free run.

• cites Henseler et al., 2015 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Many participants believed that integrating GenAI in higher education will have a positive long-term impact on teaching and learning and envisioned integrating GenAI into their teaching practices.

“Thus, based on the means of Q22 (mean = 3.92) and Q23 (mean = 3.86), many of the participants believed that the integration of generative AI technologies in higher education will have a positive impact on teaching and learning in the long run and they envision integrating generative AI technologies into their teaching practices in the future.”

May combine two claims.

Participants had a certain level of understanding and concerns over the limitations of generative AI technology in handling complex tasks.

“They did have a certain level of understanding and concerns over the limitations of generative AI technology in terms of handling complex tasks.”

May combine two claims.

Participants recognized the technology's attainment value (e.g., improving efficiency), its foreseeable utility value in the workplace, and its long-term impacts on learning outcomes.

“Meanwhile, the participants recognized the technology's attainment value such as improving efficiency, and its foreseeable utility value in the workplace as well as the long-term impacts on the learning outcomes (for example, academic performance, creativity-and emotionrelated competencies).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The questionnaire was modified based on pilot study feedback to ensure its accuracy and clarity.

“The questionnaire was modified based on the feedback received during the pilot study to ensure its accuracy and clarity.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Pilot studies were conducted before formal data collection to ensure that the questionnaire items were relevant and clear.

“To ensure that the questionnaire items were relevant and clear, pilot studies were conducted before the formal data collection.”

May combine two claims.

The dataset's S.D., skewness, and kurtosis values indicate a normal distribution, allowing further calculations in stages 2 and 3.

“As summarized by Table 4, S.D., skewness, and kurtosis values indicate a normal distribution of the dataset, which allows further calculations in stage 2 and stage 3.”

May combine two claims.

The knowledge, perceived value, and perceived cost scales have acceptable internal consistency.

“As shown in Table 5, the Cronbach alpha values are all greater than or closer to 0.7, indicating an acceptable internal consistency within the three scales.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The expectancy-value framework has been widely used in educational research to examine students' motivation, learning, and achievement.

“The expectancy-value framework has been widely used in educational research to examine students' motivation, learning, and achievement (Cheng et al., 2020;Sin et al., 2022).”

• cites Cheng et al., 2020 sourceNot checked yet

  Not checked in the free run.

• cites Sin et al., 2022 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Previous research has shown a positive correlation between students' knowledge of technology and their intention to use it, such as via the UTAUT model.

“Previous research has also shown a positive correlation between students' knowledge of technology and their intention to use it such as via the UTAUT model (Venkatesh et al., 2003).”

• cites Venkatesh et al., 2003 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

EVT emphasizes the role of students' knowledge and familiarity with GenAI, which is an essential aspect of this study's research questions.

“Moreover, EVT also emphasizes the role of students' knowledge and familiarity with GenAI, which is an essential aspect of this study's research questions.”

Classified as probably not a substantial claim.

Most questionnaire items were constructed to reflect issues and challenges associated with GenAI use within the EVT framework.

“Further, most of the items were constructed to reflect issues and challenges associated with GenAI use within the EVT framework.”

May combine two claims.

The questionnaire was specifically designed for this study and informed by a review of relevant literature and existing questionnaires on teachers' and students' perceptions of educational technologies based on the EVT framework.

“The questionnaire was specifically designed for the purposes of this study.It was informed by a review of relevant literature and existing questionnaires on teachers' and students' perceptions of educational technologies based on the EVT framework (e.g., Backfisch et al., 2021aBackfisch et al., , 2021b;;Ball et al., 2019;Chen, 2011;Ranellucci et al., 2020).”

• cites Backfisch et al., 2021aNot checked yet

  Not checked in the free run.

• cites Ball et al., 2019 sourceNot checked yet

  Not checked in the free run.

• cites Chen, 2011 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Due to the strong theoretical basis of EVT, only Confirmatory Factor Analysis (CFA) was used without Exploratory Factor Analysis (EFA), following calculation of Cronbach's alpha.

“Regarding the validation stage (i.e.stage 2), due to the strong theoretical basis of EVT, it was decided to use only Confirmatory Factor Analysis (CFA) without using Exploratory Factor Analysis (EFA), following the calculation of Cronbach's alpha.”

Classified as probably not a substantial claim.

EVT presents both expectancy (self-efficacy) and perceived value as having a direct impact on technology adoption in a concurrent manner.

“the EVT presents both expectancy (self-efficacy) and perceived value as having a direct impact on technology adoption in a concurrent manner (Backfisch et al., 2021a(Backfisch et al., , 2021b)).”

• cites Backfisch et al., 2021aNot checked yet

  Not checked in the free run.

• cites Backfisch et al., 2021bNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Prior experience with AI is associated with a positive attitude towards AI (Dahlkemper et al., 2023).

“While Dahlkemper et al (2023) showed that students who had prior experience with AI tended to have a positive attitude towards AI, our study was able to demonstrate a significant relationship between students' frequency of GenAI use and their intention to use GenAI tools.”

• cites Dahlkemper et al (2023)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Previous research has shown that perceived barriers, such as cost, can negatively affect students' intentions to use technology in education.

“Previous research has shown that perceived barriers, such as cost, can negatively affect students' intentions to use technology in education (Chan & Hu, 2023;Flake et al., 2015;Regmi & Jones, 2020;Stüber, 2018).”

• cites Chan & Hu, 2023Not checked yet

  Not checked in the free run.

• cites Flake et al., 2015 sourceNot checked yet

  Not checked in the free run.

• cites Regmi & Jones, 2020Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

CFA is a hypothesis-driven method that allows direct testing of the proposed factor structure.

“According to Brown (2006), CFA is a hypothesis-driven method that allows for direct testing of the proposed factor structure.”

• cites Brown, 2006Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

EVT takes into account the perceived value and cost associated with using GenAI, which are critical factors in determining students' intentions to use such technology.

“EVT is considered more suitable for this study because it takes into account the perceived value and cost associated with using GenAI, which are critical factors in determining students' intentions to use such technology.”

• AssumptionRests on an assumption: The concept of cost is either overlooked or underdeveloped in models such as TAM and UTAUT.

Probably not a substantial claim: may not pass S3 (load-bearing).

Studies have shown that perceived usefulness and perceived ease of use are significant determinants of technology acceptance.

“Studies have shown that perceived usefulness and perceived ease of use are significant determinants of technology acceptance (Davis, 1989;Teo, 2009;Venkatesh et al., 2003).”

• cites Davis, 1989 sourceNot checked yet

  Not checked in the free run.

• cites Teo, 2009 sourceNot checked yet

  Not checked in the free run.

• cites Venkatesh et al., 2003 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Prior studies (Gado et al., 2022; Hu, 2022) found that perceived usefulness of AI had a significant relationship with students' intention to use AI.

“Similarly, Gado et al. (2022) and Hu (2022) found that perceived usefulness of AI had a significant relationship with students' intention to use AI.”

• cites Gado et al. (2022) sourceNot checked yet

  Not checked in the free run.

• cites Hu (2022)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Prior studies found that students' perceived ease of use and perceived usefulness of AI tools positively affect their intention to use the tools.

“This finding resonates with Kim et al. 's (2020) and Hu's (2022) studies that students' perceived ease of use and perceived usefulness of AI tools positively affect their intention to use the tools.”

• cites Kim et al. 's (2020) sourceNot checked yet

  Not checked in the free run.

• cites Hu's (2022)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Using CFA in this study is a parsimonious approach that ensures findings are concise and easier to interpret.

“Moreover, the use of CFA in this study can be considered a parsimonious approach, ensuring that our research findings are concise and easier to interpret.”

May combine two claims.

The study primarily relied on self-reported data, which may be subject to response biases such as social desirability or recall bias.

“Furthermore, the study primarily relied on self-reported data, which may be subject to response biases, such as social desirability or recall bias.”

May combine two claims.

The study was conducted at a single point in time and may not account for potential changes in students' attitudes and perceptions towards GenAI over time.

“Additionally, the study was conducted at a single point in time, and thus, it may not account for potential changes in students' attitudes and perceptions towards GenAI over time.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

The EVT-based instrument addresses the gap in the literature and provides a strong foundation for future research and practice in this area.

“The EVT-based instrument addresses this gap in the literature and provides a strong foundation for future research and practice in this area.”

• AssumptionRests on an assumption: There is a lack of robust, theoretically grounded instruments to assess students' attitudes towards GenAI adoption.

Classified as probably not a substantial claim.

Using only CFA in this study can be considered a justifiable decision given the strong theoretical basis, prior research, and focus on hypothesis testing.

“However, given the strong theoretical basis, prior research, and focus on hypothesis testing, using only CFA in this study can be considered a justifiable decision.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Skipping EFA in the validation can have limitations, such as the potential to overlook alternative factor structures or issues with item loadings.

“It is essential to acknowledge that skipping EFA in the validation can have some limitations, such as the potential to overlook alternative factor structures or issues with item loadings (Fabrigar et al., 1999).”

• cites Fabrigar et al., 1999 sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The sample size of 405 participants may not be fully representative of the larger student population.

“Firstly, the sample size was restricted to 405 participants, which may not be fully representative of the larger student population.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Knowledge is a key component of AI literacy and can increase students' confidence and readiness in using GenAI.

“Knowledge, a key component of AI literacy, can increase students' confidence and readiness in using GenAI (Ng et al., 2021).”

• cites Ng et al., 2021Not checked yet

  Not checked in the free run.

May combine two claims.

Students who have used AI applications and have sound knowledge of AI tend to have a positive view of the technology, resulting in high expectancies for success (Chen et al., 2022).

“Students who have used AI applications and have sound knowledge of AI tend to have a positive view of the technology (Chen et al., 2022), thus resulting in high expectancies for success.”

• cites Chen et al., 2022 sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Expectancy and value are motivational constructs that determine an individual's decision to perform and persist in tasks (Wigfield, 1994).

“Wigfield (1994) refers to expectancy and value as motivational constructs that determine an individual's decision to perform and persist in tasks.”

• cites Wigfield (1994) sourceNot checked yet

  Not checked in the free run.

May combine two claims.

The study offers a validated instrument for measuring students' perceptions of GenAI, which can be used by researchers for further studies of GenAI adoption.

“This study offers a validated instrument for measuring students' perceptions of GenAI, which can be used by researchers for further studies of GenAI adoption.”

May combine two claims.

χ2 is very sensitive to sample size and its p-value tends to be small in a big sample.

“Since χ 2 is very sensitive to sample size and its p-value will tend to be small in a big sample, we decided to report the χ 2 /df ratio instead as recommended by Schermelleh-Engel et al. (2003).”

• cites Schermelleh-Engel et al. (2003)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study obtained approval from the Human Research Ethics Committee (HREC) at the University of Hong Kong, and all participants provided informed consent.

“We conducted our study with full adherence to ethical principles and guidelines for human subject research, and we obtained approval from the Human Research Ethics Committee (HREC) at the University of Hong Kong.All participants provided informed consent, and we did not report any personal identifiers in the study.(Ethical Approval No.: EA230079).”