Open Collections

A neural architecture for detecting user confusion in eye-tracking data

University of British Columbia

Encouraged by the success of deep learning in a variety of domains, we investigate the effectiveness of a novel application of such methods for detecting user confusion with eye-tracking data. We introduce an architecture that uses RNN and CNN sub-models in parallel to take advantage of the temporal and visuospatial aspects of our data. Experiments with a dataset of user interactions with the ValueChart visualization tool show that our model outperforms an existing model based on a Random Forest classifier, resulting in a 22% improvement in combined sensitivity & specificity. This is a larger improvement in performance than that achieved by either the CNN or RNN when considered alone, though all three deep learning models outperform the Random Forest baseline. To investigate this effect and understand the performance increase achieved by the deep learning models, we carried out preliminary investigations using explainable AI methods, from which we derive future directions for exploring performance gains from combining deep learning models.

Text

2020-09-08

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/75900

Computer Science

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/