Open Collections

Graham, Liam

University of British Columbia

There is increasing evidence that, when interacting with an AI system, users may benefit from having personalized explanations of the system’s behavior. Providing such personalized explanations requires that the AI system can assess user properties that are relevant for personalization. In the absence of prior information on such user properties, the system must rely on being able to predict them during the course of the interaction, in order to deliver personalized explanations as needed. In this thesis, we investigate the feasibility of predicting three user traits – conscientiousness, need for cognition and reading proficiency – that have been shown to impact the effectiveness of explanations when users interact with an Intelligent Tutoring System (ITS). We discuss results on training machine learning models on eye-tracking data, action data and a combination of both as users interact with the ITS. For the eye-tracking data, we test features generated from summative statistics of a user’s gaze, and those from patterns in sequences of areas of interest generated from a user’s gaze. We provide a detailed analysis of the relative efficacy of such models, and show that prediction above baseline classifiers is possible even during early stages of the interaction, which is crucial for the timely personalization of explanations. Lastly, we examine the feature permutation importance of our best models to gain insight into how they work and relate to literature.

Text

2023-01-24

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/83744

Computer Science

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/