Open Collections

Fan, Xu (Neil)

University of British Columbia

By demonstrating the capability of creating enhanced immersive user experiences, Augmented Reality (AR) has received significant attention in recent years. However, the most common interaction methods with this new technology are still largely limited to in-air gesture, voice and gaze controls. Previous works have proven that involving haptic feedback by interacting with real objects can improve the users’ performance and reduce their workload. But they are still not mature enough to be adopted in commercial products. This thesis delves into this challenge, offering novel solutions to make tangible interactions in AR more practical and effective. We begin by addressing touch interactions on ad-hoc surfaces. The vision is to transform everyday, un-instrumented surfaces into expansive touch screens, enhancing AR's interactive domain. Existing systems, however, grapple with issues of touch detection accuracy and latency. By harnessing dynamic fingertip information, we introduce a cutting-edge machine learning approach that elevates touch accuracy and anticipates user interactions, significantly reducing response time. We also analyze the possibility of transforming real-world objects into passive controllers. Precise tracking of objects in a 3D space—represented by six degrees of freedom (DOF)—amplifies AR's spatial control potential. Yet, current methods either necessitate extraneous tracking markers, imposing undue burdens on users, or are too resource-intensive for real-time execution on devices like head-mounted displays. Addressing this, we present "TangiAR." This system identifies and tracks everyday objects in AR without auxiliary trackers. Its efficacy is evidenced through comprehensive tests, assessing its robustness under various challenges, such as occlusions. Moreover, we showcase its practical application, operating seamlessly on a standard HoloLens 2, underlining the immediacy of its integration potential. By bridging these tangible interaction gaps in AR, this thesis not only contributes to technical advancements but also underscores the broader implications for AR's future, emphasizing a more intuitive, responsive, and enriched user experience.

Text

2023-12-21

Attribution-NonCommercial-NoDerivatives 4.0 International

http://hdl.handle.net/2429/86989

Computer Science

University of British Columbia

UBCV

http://creativecommons.org/licenses/by-nc-nd/4.0/