Open Collections

Hunt, Bridgette

This thesis explores the role of generative artificial intelligence (AI), specifically chatbots, in higher education by examining student sentiment, engagement, and the impact on instructor workload. The study focuses on the HelpMe system, a web application help center incorporating a Retrieval- Augmented Generation (RAG)-based chatbot adapted from the ChatEd architecture. Implemented in an upper-year computer science course at UBC Okanagan, the chatbot was designed to assist students with asynchronous questions using a knowledge base built from course materials and instructorprovided answers. To ascertain the chatbot’s effectiveness and students’ perception of it, survey data was collected from 83 participants, accompanied by backend system usage data and instructor-reported email correspondence. The survey gauged attitudes toward AI in education, willingness to use chatbots, preference for course-provided versus external tools, and comparative usefulness against traditional help-seeking methods like email and office hours. Results indicate that a majority of students are open to using chatbots for academic assistance, mainly if provided directly by the institution. While many participants responded neutrally to questions about the chatbot’s capabilities, a significant portion acknowledged its utility and expressed support for broader integration of such tools in other courses. However, usage data revealed that most students interacted with the chatbot only once or twice, with a sharp decline in engagement after the initial weeks of the term. Despite its potential, there was insufficient evidence to determine if the chatbot significantly reduced the volume of student emails or demands on instructor time. The findings suggest that while students are receptive to AI-based tools in the classroom, sustained engagement requires further improvements in usability, integration, and trust. The HelpMe system’s chatbot may serve best as a supplemental resource rather than a replacement for traditional support channels. Ultimately, this thesis highlights both the promise and limitations of generative AI in education, offering insight into how such systems can be refined to better support students and educators alike.

Text

University of British Columbia. COSC_O 449

Vancouver : University of British Columbia Library

10.14288/1.0448870

Science, Irving K. Barber Faculty of (Okanagan); Computer Science, Mathematics, Physics and Statistics, Department of (Okanagan)

Undergraduate

DSpace