The Impact of Generative AI Tools on Postgraduate Students’ Learning Experiences: New Insights Into Usage Patterns

Abstract

Aim/Purpose: This study aimed to investigate the impact of generative artificial intelligence (GenAI) tools on postgraduate students’ learning experiences. It examined how these students utilize GenAI tools, identified the perceived benefits of their use, and uncovered the challenges students face in their learning experiences with these tools. Background: Recent advancements in GenAI, particularly tools like ChatGPT, are transforming how postgraduate students engage with learning, offering new opportunities for academic endeavors. However, understanding how students interact with these tools remains unclear, especially relating to their engagement with activities designed to use GenAI tools as learning assistants. Methodology: This study employed a qualitative case study research design, including 11 female postgraduate students enrolled in an educational technology master’s program as the primary data collection method. The researchers only had access to female students because of the gender-segregated education system at most Saudi universities. The participants were interviewed after they engaged in a planned activity called Designing a Training Program with GPT Assistance (DTPGA) and unplanned activities involving other GenAI tools. DTPGA is an activity aimed at integrating innovations in training technologies by designing a training program with ChatGPT assistance. Eleven ChatGPT conversation logs and DTPGA activity answers were analyzed based on specific rubrics. The results of the DTPGA activity analysis support the thematic analysis of the interviews. Contribution: The key contributions of this study include the design and implementation of a planned activity (DTPGA) to understand how students interact with GenAI tools as learning assistants. Evaluating students’ work on a DTPGA activity offers educators a clear vision for assessing GenAI-assisted tasks. In terms of data collection and analysis, this study also offers a deep understanding of the participants’ perspectives and actual usage. The study identified two distinct usage patterns of GenAI among students, each classified with its characteristics and implications, shedding light on the diverse ways these tools are integrated into students’ learning experiences. Furthermore, the findings provide compelling evidence of the risks associated with misunderstandings that can emerge when students misuse GenAI tools. Findings: Students employ GenAI tools to have intellectual conversations, broaden their perspectives, and prototype and test their solutions. The research identified two distinct GenAI usage patterns – the intellectual partner and the information browser – each characterized by unique traits. The first pattern, intellectual partner, uses ChatGPT as a cognitive partner and interacts with it as if it were a real person. This usage tends to be dialogic, collaborative, selectively researching, persistent, and feedback-oriented. The second pattern, information browser, utilizes ChatGPT primarily as a search engine for ideas with the following usage traits: transactional use and illusory understanding. Regarding the perceived benefits, the students appreciated the importance of GenAI tools in enhancing academic skills development, such as academic writing, translation, and self-directed learning. They also acknowledged GenAI tools’ role in boosting productivity in routine tasks by optimizing efforts and enhancing efficiency. In creative tasks, they saved time, offered diverse and innovative ideas, and boosted motivation and confidence. However, the perceived drawbacks included concerns about ethical use, a lack of clear guidelines, limited capabilities, overreliance on GenAI, and potential skills erosion. Recommendations for Practitioners: These findings will help educators guide students in developing metacognitive awareness and refining their use of GenAI tools while addressing the potential risks. Emphasis should be placed on understanding the distinct patterns of GenAI use, which will enable practitioners to scaffold learners. Integration also requires developing innovative strategies through redesigning lessons and activities and carefully selecting adequate assessment methods that foster critical thinking. Additionally, it is important to enhance learners’ skills in effective intellectual communication through GenAI tools, to guide them in evaluating AI-generated content, and to provide ethical frameworks for responsible AI use. Recommendation for Researchers: Researchers need to explore more innovative strategies to integrate GenAI tools into higher education, in line with the nature of the various disciplines, to use GenAI tools as intellectual partners and not superficially. The field requires further studies to investigate how GenAI tools can be integrated to develop students’ intellectual skills, preparing them for future roles that demand advanced abilities and effective use of GenAI for personal and professional growth. Impact on