A Pilot Study Integrating an AI-driven Chatbot in an Introductory Programming Course

Abstract

As AI software tools become more commonplace, their potential to transform the student experience has greatly increased. By integrating these tools into a course, students can begin to receive real-time, around the clock, instructor moderated support. The potential benefits for students are significant. In addition to increasing their ability to work constructively on their schedule, it has the potential to engage students who might otherwise avoid directly interacting with another person. Before these tools can be meaningfully actualized into a course, work must be done to build a knowledge base about the course. For this research, the team sought to develop an intelligent chatbot interface for an introductory computer programming course. The interface had an initially limited knowledge base with the intent that it would be populated based on students' interactions with the chatbot. This model allowed the bot to evolve with the needs of the students. This paper seeks to present the methodology for how the chatbot was developed and integrated into the course, how the knowledge base was developed, the usage during the pilot, and the next steps for improving the chatbot's interface. Additionally, the paper will discuss the mechanisms added to handle issues such as false-positive responses and how faculty may be able to integrate such tools into their own courses as supplementary assistance.