Statistically Informed Multimodal (Domain Adaptation by Transfer) Learning Framework : A Domain Adaptation Use-Case for Industrial Human–Robot Communication
department or school: Applied Science, Faculty of
department or school: Non UBC
department or school: Electrical and Computer Engineering, Department of
data provider: DSpace
identifier citation: Electronics 14 (7): 1419 (2025)
creator: Mukherjee, Debasmita
creator: Najjaran, Homayoun
issued: 2025-05-09T16:50:44Z
issued: 2025-03-31
description: Cohesive human–robot collaboration can be achieved through seamless communication between human and robot partners. We posit that the design aspects of human–robot communication (HRCom) can take inspiration from human communication to create more intuitive systems. A key component of HRCom systems is perception models developed using machine learning. Being data-driven, these models suffer from the dearth of comprehensive, labelled datasets while models trained on standard, publicly available datasets do not generalize well to application-specific scenarios. Complex interactions and real-world variability lead to shifts in data that require domain adaptation by the models. Existing domain adaptation techniques do not account for incommensurable modes of communication between humans and robot perception systems. Taking into account these challenges, a novel framework is presented that leverages existing domain adaptation techniques off-the-shelf and uses statistical measures to start and stop the training of models when they encounter domain-shifted data. Statistically informed multimodal (domain adaptation by transfer) learning (SIMLea) takes inspiration from human communication to use human feedback to auto-label for iterative domain adaptation. The framework can handle incommensurable multimodal inputs, is mode and model agnostic, and allows statistically informed extension of datasets, leading to more intuitive and naturalistic HRCom systems.
aggregated CHO: https://circle.library.ubc.ca/rest/handle/2429/91082
note: Academic Editors: Kostas Karpouzis,Bipin Indurkhya, Eleuda Nuñez,Marie-Monique Schaper and CecilioAnguloReceived: 14 February 2025Revised: 22 March 2025Accepted: 28 March 2025Published: 31 March 2025Citation: Mukherjee, D.; Najjaran, H.Statistically Informed Multimodal(Domain Adaptation by Transfer)Learning Framework: A DomainAdaptation Use-Case for IndustrialHuman–Robot Communication.Electronics 2025, 14, 1419.https://doi.org/10.3390/electronics14071419Copyright: © 2025 by the authors.Licensee MDPI, Basel, Switzerland.This article is an open access articledistributed under the terms andconditions of the Creative CommonsAttribution (CC BY) license(https://creativecommons.org/licenses/by/4.0/).ArticleStatistically Informed Multimodal (Domain Adaptation byTransfer) Learning Framework: A Domain Adaptation Use-Casefor Industrial Human–Robot CommunicationDebasmita Mukherjee 1,2,3,† and Homayoun Najjaran 2,*1 Department of Electrical Computer Engineering and J. Herbert Smith Centre for Technology ManagementEntrepreneurship, University of New Brunswick, Fredericton, NB E3B 5A3, Canada;debasmita.mukherjee@unb.ca2 Department of Electrical and Computer Engineering, University of Victoria, Victoria, BC V8P 5C2, Canada3 Department of Mechanical Engineering, The University of British Columbia, Kelowna, BC V1V 1V7, Canada* Correspondence: najjaran@uvic.ca† Current address: University of New Brunswick, 3 Bailey Dr, Fredericton, NB E3B 5A3, Canada.Abstract: Cohesive human–robot collaboration can be achieved through seamless communi-cation between human and robot partners. We posit that the design aspects of human–robotcommunication (HRCom) can take inspiration from human communication to create moreintuitive systems. A key component of HRCom systems is perception models developedusing machine learning. Being data-driven, these models suffer from the dearth of compre-hensive, labelled datasets while models trained on standard, publicly available datasets donot generalize well to app…
has type: Article
is shown at: 10.14288/1.0448835
language: eng
peer review status: Reviewed
provider: Vancouver : University of British Columbia Library
publisher: Multidisciplinary Digital Publishing Institute
publisher DOI: 10.3390/electronics14071419
rights: CC BY 4.0
rights URI: https://creativecommons.org/licenses/by/4.0/
scholar level: Faculty
scholar level: Researcher
subject: human–robot communication
subject: iterative domain adaptation
subject: personalized machine learning
subject: feedback enabled labelling
subject: machine perception
subject: model personalization
type: Text
identifier URI: http://hdl.handle.net/2429/91082