UBC Undergraduate Research

Multimodal graphical user interface for ultrasound machine control via da Vinci surgeon console : design, development, and initial evaluation Tatasurya, Samuel Radiant


The integration of hands-free modalities for controlling an ultrasound machine from the surgical robot console has the potential to assist the surgeon in manipulating the ultrasound image independently during robot-assisted minimally-invasive surgery (MIS). Hands-free control is especially useful when both of the surgeon’s hands are occupied with the control of surgical tools and no assistant is present to help the surgeon operating the ultrasound machine. This thesis presents the design and development of a graphical user interface (GUI) for da Vinci classic system which enables its user to interact with the SonixTouch ultrasound machine directly from the surgeon console using eye gaze and voice commands. An initial user study was performed as a part of iterative design process in order to assess the GUI usability, and the task chosen for this purpose was an image optimization task. Although current iteration of the GUI resulted in relatively long completion time and high speech recognition error ratio, it demonstrated that integrating such technology in robot-assisted MIS is feasible.

Item Citations and Data


Attribution-NonCommercial-NoDerivs 2.5 Canada

Usage Statistics