Open Collections

Statistically Informed Multimodal (Domain Adaptation by Transfer) Learning Framework : A Domain Adaptation Use-Case for Industrial Human–Robot Communication

Multidisciplinary Digital Publishing Institute

Cohesive human–robot collaboration can be achieved through seamless communication between human and robot partners. We posit that the design aspects of human–robot communication (HRCom) can take inspiration from human communication to create more intuitive systems. A key component of HRCom systems is perception models developed using machine learning. Being data-driven, these models suffer from the dearth of comprehensive, labelled datasets while models trained on standard, publicly available datasets do not generalize well to application-specific scenarios. Complex interactions and real-world variability lead to shifts in data that require domain adaptation by the models. Existing domain adaptation techniques do not account for incommensurable modes of communication between humans and robot perception systems. Taking into account these challenges, a novel framework is presented that leverages existing domain adaptation techniques off-the-shelf and uses statistical measures to start and stop the training of models when they encounter domain-shifted data. Statistically informed multimodal (domain adaptation by transfer) learning (SIMLea) takes inspiration from human communication to use human feedback to auto-label for iterative domain adaptation. The framework can handle incommensurable multimodal inputs, is mode and model agnostic, and allows statistically informed extension of datasets, leading to more intuitive and naturalistic HRCom systems.

Article

eng

Vancouver : University of British Columbia Library

10.14288/1.0448835

Applied Science, Faculty of; Non UBC; Electrical and Computer Engineering, Department of

10.3390/electronics14071419

Faculty; Researcher

DSpace