UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Gaze tracking for human-ultrasound machine interaction and medical image understanding Zhu, Hongzhi


Despite the rapid advancement in medical imaging technologies, health care systems in Canada as well as in many other countries still cannot provide patients with sufficient medical imaging resources due to the ever-growing demands, lack of imaging devices, and fully trained medical practitioners. What makes the situation worse are persistent and prevalent work-related injury situations for medical practitioners, especially sonographers. To help these problems, the overall objective of this thesis is to improve the user interface design for ultrasound machines and contribute to automated medical imaging-based diagnosis with the help of the gaze tracking technology. Multi-themed study and research were conducted to achieve the objectives. On one hand, for the interface design improvement, we started by analyzing the characteristics of gaze signals through statistical modelling. Then we surveyed among sonographers to understand their daily usage of the ultrasound machines. Also, we analyzed different kinds of ultrasound machines to understand common design patterns and suboptimal control logic. To improve the suboptimal control logic and observed usage difficulties on ultrasound machines, we designed and implemented a gaze tracking contingent ultrasound machine. For the validation of our design ideas and machine effectiveness, comparative user studies were conducted. Additionally, we enhanced the user experience by solving the gaze tracker accuracy deterioration problem during normal usage of the devices by proposing a disruption-free auto-recalibration method. On the other hand, for the automated medical image diagnosis, with the integration of the gaze tracking dataset, we proposed deep learning methods that not only can provide a label that predicts which kind of abnormality is observed in the medical image, but also some reasoning that can guide humans to understand how the prediction is made. Multi-task learning methods and neural network attention mechanisms were used for enhanced automated diagnostic performance and better interpretability of the deep learning models. Our study has demonstrated the usefulness of integrating gaze tracking with human-computer interaction and image understanding in the medical field. Future work can be done to further processing and quantify gaze tracking data.

Item Media

Item Citations and Data


Attribution-NonCommercial-ShareAlike 4.0 International