Applied Science, Faculty of
Mechanical Engineering, Department of
DSpace
UBCV
Chen, Delun
Master of Applied Science - MASc
University of British Columbia
Carbon fiber-reinforced polymer (CFRP) has been widely used in the automotive and aerospace industries. The current manufacturing process associated with vacuum bagging requires at least two trained technicians with expertise to manually manipulate the materials. To alleviate the physical demands on technicians, a robot arm with end-of-arm tooling (EOAT) was developed to collaborate with them and assist with pleat formation and migration tasks. However, encountering this robot without prior understanding of its behaviour can lead to frustration, confusion, or a high learning curve. This thesis evaluates robot visual cues designed for human-robot collaboration in the context of a simulated CFRP manufacturing vacuum bagging process. By communicating gripper force information to technicians, these cues aim to enhance the interpretability of the robot's behaviour and reduce the workload experienced during the human-robot interaction. This study first implemented the mechatronic system to control the robot's gripper force under both semi-automatic and automatic pleating modes, and then developed four visual cues based on light brightness, colour gradient, loading progress bar, and flashing light to convey system status to the operator under both modes. The results indicate that the colour gradient and loading progress bar are significantly more suitable for indicating gripper force compared to the other patterns. These patterns exhibit significantly higher legibility, intuitiveness and interactivity, leading to reduced human perceived mental load, effort, and frustration. Participants prefer operating the robot using the automatic mode, which improves the overall performance of the robot. Additionally, all designed cues outperform the no cue condition, highlighting the importance of visual feedback for users to understand the robot’s behaviour during human-robot collaboration.
Design and Evaluation of Visual Cues for Human-Robot Collaboration in Carbon Fiber-Reinforced Polymer (CFRP) Manufacturing by Delun Chen B.A.Sc., The University of British Columbia Okanagan, 2019 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF APPLIED SCIENCE in The Faculty of Graduate and Postdoctoral Studies (Mechanical Engineering) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) September 2023 © Delun Chen, 2023 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the thesis entitled: Design and Evaluation of Visual Cues for Human-Robot Collaboration in Carbon Fiber-Reinforced Polymer (CFRP) Manufacturing submitted by Delun Chen in partial fulfilment of the requirements for the degree of Master of Applied Science in Mechanical Engineering Examining Committee: Dr. H.F. Machiel Van der Loos, Mechanical Engineering, University of British Columbia Supervisor Dr. Kefei Wen, Mechanical Engineering, University of British Columbia Supervisory Committee Member Dr. Ahmad Mohammadpanah, Mechanical Engineering, University of British Columbia Supervisory Committee Member iii Abstract Carbon fiber-reinforced polymer (CFRP) has been widely used in the automotive and aerospace industries. The current manufacturing process associated with vacuum bagging requires at least two trained technicians with expertise to manually manipulate the materials. To alleviate the physical demands on technicians, a robot arm with end-of-arm tooling (EOAT) was developed to collaborate with them and assist with pleat formation and migration tasks. However, encountering this robot without prior understanding of its behaviour can lead to frustration, confusion, or a high learning curve. This thesis evaluates robot visual cues designed for human-robot collaboration in the context of a simulated CFRP manufacturing vacuum bagging process. By communicating gripper force information to technicians, these cues aim to enhance the interpretability of the robot's behaviour and reduce the workload experienced during the human-robot interaction. This study first implemented the mechatronic system to control the robot's gripper force under both semi-automatic and automatic pleating modes, and then developed four visual cues based on light brightness, colour gradient, loading progress bar, and flashing light to convey system status to the operator under both modes. The results indicate that the colour gradient and loading progress bar are significantly more suitable for indicating gripper force compared to the other patterns. These patterns exhibit significantly higher legibility, intuitiveness and interactivity, leading to reduced human perceived mental load, effort, and frustration. Participants prefer operating the robot using the automatic mode, which improves the overall performance of the robot. Additionally, all designed cues outperform the no cue condition, highlighting the importance of visual feedback for users to understand the robot’s behaviour during human-robot collaboration. iv Lay Summary Large-scale composite manufacturing processes, particularly vacuum injection moulding (VIM), require experienced workers to spend hours manually manipulating multiple layers of fabric to produce the desired composite part. A promising solution is to utilize robots as assistants to semi-automate the repetitive fabric-handling tasks. To increase user acceptance and prevent confusion during this collaboration, my research focuses on developing a robotic system that uses sensors to automate fabric handling and display the status of certain task parameters. The system uses a special-purpose gripper, called vacuum infusion moulding end-effector (VIMEE), in tandem with a KUKA LBR arm. My user interface study explores the effectiveness of different visual cues, using LED light strips, that convey physical parameters such as pinching force and fabric position inside the gripper. Validated user interface questionnaires are used to compile usability statistics with four display types across two operating modes. Using a two-way ANOVA, the results will determine which types of displays are most intuitive and least mentally demanding for the two modes. v Preface With the support of my supervisor, Dr. Machiel Van der Loos, I conceptualized and designed the research program described in this thesis. A UBC MECH capstone student team conducted the initial design of the research platform, Vacuum Infusion Moulding End-Effector (VIMEE). Due to the COVID-19 pandemic, the Capstone project was not able to fully complete the implementation. I undertook the remaining development of the robot, including sensor calibration, software and hardware integration. A version of Section 3.1, regarding the robot gripper’s mechatronics design, has been published [Chen D, Chan WP, Quek J, Van der Loos HFM. Mechatronic Design of Two-Jaw Pleating End-Effector for Large-Scale Carbon Fibre Reinforced Polymer Manufacturing. Materials Science Forum. 1086:85–90, 2023]. I was the lead investigator, responsible for all major testing, data analysis, and manuscript composition. Quek J was involved in sensor calibration and data collection. Chan WP and Van der Loos HFM were the supervisory authors on this project and were involved throughout the project in concept formation and manuscript revision. The design of robot visual cues in Section 3.2 was based on a preliminary survey study in collaboration with Yushan Liu, an MEng student in Mechanical Engineering at the University of British Columbia. The UBC Ethics ID for this study is H22-01934. The preliminary study laid the foundation for me to finalize the design of the visual cues. Kyle Ah Von and Patrick Thangarajah, both volunteers during their B.A.Sc. studies in Mechanical Engineering at the University of British Columbia, assisted with code programming, testing and data collection. I conducted a literature review to frame the experimental design and closely supervised the testing and user study. The data analysis in Section 3.6 and the writing of the results in Chapter 4 are entirely my own contributions. The UBC Ethics ID for this user study is H22-03470. vi Table of Contents Abstract .......................................................................................................................................... iii Lay Summary ................................................................................................................................. iv Preface ............................................................................................................................................. v Table of Contents ........................................................................................................................... vi List of Tables ................................................................................................................................... x List of Figures ............................................................................................................................... xii List of Abbreviations.................................................................................................................... xiv Acknowledgements ....................................................................................................................... xv Dedication .................................................................................................................................... xvi Chapter 1: Introduction ................................................................................................................... 1 1.1 Thesis Outline .................................................................................................................. 7 Chapter 2: Background and Motivation .......................................................................................... 9 2.1 Artificial Cues in Robotics ............................................................................................... 9 2.1.1 Modalities for Human-Robot Communication ....................................................... 10 2.1.2 Non-verbal Visual Cues .......................................................................................... 15 2.1 Research Objectives ....................................................................................................... 20 Chapter 3: Method and Materials ................................................................................................. 21 3.1 Robot Research Platform ............................................................................................... 22 3.1.1 Pleat Loading Function ........................................................................................... 22 3.1.2 Pleat Migration Function ........................................................................................ 23 3.1.3 Operating Modes ..................................................................................................... 24 vii 3.1.4 VIMEE Gripper Design .......................................................................................... 24 3.2 Design of Robot LED Cues ............................................................................................ 27 3.2.1 Light Brightness ...................................................................................................... 28 3.2.2 Loading Progressing Bar ......................................................................................... 29 3.2.3 Flashing Light ......................................................................................................... 30 3.2.4 Colour Gradient ...................................................................................................... 31 3.3 Experimental Scenario ................................................................................................... 33 3.4 Experimental Procedure and Questionnaire Design ....................................................... 35 3.4.1 Suitability Likert Scale ........................................................................................... 37 3.4.2 Interpretability Likert Scale .................................................................................... 37 3.4.3 NASA Task Load Index (TLX) ............................................................................... 38 3.4.4 Follow-Up Questions .............................................................................................. 39 3.5 Participants ..................................................................................................................... 39 3.6 Data Analysis .................................................................................................................. 40 Chapter 4: Results ......................................................................................................................... 43 4.1 Suitability Scale.............................................................................................................. 43 4.1.1 Hypothesis H1.1 ...................................................................................................... 44 4.1.2 Hypothesis H1.2 ...................................................................................................... 46 4.1.3 Preference for Robot LED Cues ............................................................................. 48 4.2 Interpretability Scales ..................................................................................................... 49 4.2.1 Legibility Scale ....................................................................................................... 50 4.2.2 Intuitiveness Scale .................................................................................................. 51 viii 4.2.3 Interactivity Scale ................................................................................................... 53 4.3 NASA TLX (Task Load Index) Scales ........................................................................... 56 4.3.1 Mental Demand Scale ............................................................................................. 57 4.3.2 Effort Demand Scale ............................................................................................... 59 4.3.3 Frustration Scale ..................................................................................................... 62 Chapter 5: Discussion ................................................................................................................... 66 5.1 Suitability of Robot LED Cues ...................................................................................... 66 5.2 Interpretability of Robot LED Cues ............................................................................... 67 5.3 Workload Demand of Robot LED Cues ......................................................................... 68 5.4 Comparative Analysis of Robot LED Cues .................................................................... 70 5.5 Limitations ..................................................................................................................... 73 5.5.1 Limitations in Participant Recruitment ................................................................... 73 5.5.2 Limitations in Experimental Setup ......................................................................... 73 5.6 Future Work .................................................................................................................... 74 Chapter 6: Conclusion................................................................................................................... 76 Bibliography ................................................................................................................................. 80 Appendices .................................................................................................................................... 86 Appendix A: Robot Hardware and Software Implementation Details ...................................... 86 A.1 Hardware and Mechatronics Design ............................................................................... 86 A.2 Software Implementation ................................................................................................ 95 Appendix B: User Study Consent Form and Questionnaire ..................................................... 111 B.1 Consent Form ................................................................................................................. 111 ix B.2 Questionnaire ................................................................................................................ 113 Appendix C: ANOVA Assumption Tests and Results of Likert-Scale Analysis ..................... 130 C.1 Normality Test ............................................................................................................... 130 C.2 Homogeneity of Variance Test ...................................................................................... 130 C.3 Sphericity Test ............................................................................................................... 131 C.4 Reliability Test .............................................................................................................. 131 C.5 ANOVA Tests on Suitability Scale ................................................................................ 131 C.6 ANOVA Tests on Legibility Scale ................................................................................. 134 C.7 ANOVA Tests on Intuitiveness Scale ............................................................................ 136 C.8 ANOVA Tests on Interactivity Scale ............................................................................. 139 C.9 ANOVA Tests on Mental Demand Scale ....................................................................... 141 C.10 ANOVA Tests on Effort Demand Scale ....................................................................... 144 C.11 ANOVA Tests on Frustration Scale ............................................................................. 147 x List of Tables Table 3.1 VIMEE task performance in different grip force ranges ............................................... 27 Table 4.1 2x5 Repeated Measures ANOVA on suitability scale ................................................... 44 Table 4.2 2x5 Repeated Measures ANOVA on interpretability scales .......................................... 49 Table 4.3 2x5 Repeated Measures ANOVA on workload demand scales ..................................... 57 Table A.1 Sensitivity comparison for different load ranges .......................................................... 94 Table C.1 Tests of normality ....................................................................................................... 130 Table C.2 Levene's test of equality of error variances ................................................................ 130 Table C.3 Mauchly's test of sphericity ........................................................................................ 131 Table C.4 Reliability statistics .................................................................................................... 131 Table C.5 ANOVA tests of suitability within-subjects effects .................................................... 131 Table C.6 Pairwise comparisons of operating modes on suitability scale .................................. 133 Table C.7 Pairwise comparisons of LED cue patterns on suitability scale ................................. 133 Table C.8 ANOVA tests of legibility within-subjects effects ...................................................... 134 Table C.9 Pairwise comparisons of LED cue patterns on legibility scale .................................. 135 Table C.10 ANOVA tests of intuitiveness within-subjects effects .............................................. 136 Table C.11 Pairwise comparisons of LED cue patterns on intuitiveness scale ........................... 138 Table C.12 ANOVA tests of interactivity within-subjects effects ............................................... 139 Table C.13 Pairwise comparisons of operating modes on interactivity scale ............................. 140 Table C.14 Pairwise comparisons of LED cue patterns on interactivity scale ........................... 140 xi Table C.15 ANOVA tests of mental demand within-subjects effects.......................................... 141 Table C.16 Pairwise comparisons of operating modes on mental demand scale ........................ 143 Table C.17 Pairwise comparisons of LED cue patterns on mental demand scale ...................... 143 Table C.18 ANOVA tests of effort demand within-subjects effects ............................................ 144 Table C.19 Pairwise comparisons of operating modes on effort demand scale .......................... 145 Table C.20 Pairwise comparisons of LED cue patterns on effort demand scale ........................ 146 Table C.21 ANOVA tests of frustration within-subjects effects ................................................. 147 Table C.22 Pairwise comparisons of operating modes on frustration scale ............................... 148 Table C.23 Pairwise comparisons of LED cue patterns on frustration scale .............................. 148 xii List of Figures Figure 1.1 Set-up and equipment for vacuum infusion moulding process ..................................... 3 Figure 1.2 The structure of the VIMEE gripper .............................................................................. 6 Figure 2.1 Types of non-anthropomorphic robots that use visual cues for HRI ........................... 16 Figure 3.1 Simulated CFRP pleating procedure ........................................................................... 23 Figure 3.2 Design layout of the robot research platform: the VIMEE gripper ............................. 26 Figure 3.3 LED cue based on the light brightness effect .............................................................. 29 Figure 3.4 LED cue based on the loading progress light effect .................................................... 30 Figure 3.5 Mapping of the LED flashing rate across different force ranges ................................ 31 Figure 3.6 LED cue based on the colour gradient effect .............................................................. 32 Figure 3.7 Robot research platform with experimental setups ..................................................... 33 Figure 3.8 Joypad control map and functionalities ....................................................................... 34 Figure 3.9 Overview of the experimental procedure .................................................................... 35 Figure 4.1 Pairwise comparisons of cue patterns on suitability scale ........................................... 45 Figure 4.2 Pairwise comparisons of operating modes on suitability scale ................................... 47 Figure 4.3 Interaction of cue patterns and modes on suitability scale .......................................... 48 Figure 4.4 Pairwise comparisons of cue patterns on legibility scale ............................................ 51 Figure 4.5 Pairwise comparisons of cue patterns on intuitiveness scale ...................................... 53 Figure 4.6 Pairwise comparisons of operating modes on interactivity scale ................................ 54 Figure 4.7 Pairwise comparisons of cue patterns on interactivity scale ....................................... 55 xiii Figure 4.8 Attributes of cue patterns on interpretability scales .................................................... 56 Figure 4.9 Pairwise comparisons of operating modes on mental demand scale ........................... 58 Figure 4.10 Pairwise comparisons of cue patterns on mental demand scale ................................ 59 Figure 4.11 Pairwise comparisons of operating modes on effort demand scale ........................... 60 Figure 4.12 Pairwise comparisons of cue patterns on effort demand scale .................................. 61 Figure 4.13 Interaction of cue patterns and modes on effort demand scale .................................. 62 Figure 4.14 Pairwise comparisons of operating modes on frustration scale ................................. 63 Figure 4.15 Pairwise comparisons of cue patterns on frustration scale ........................................ 64 Figure 4.16 Attributes of operating modes on NASA TLX scales................................................ 65 Figure 4.17 Attributes of cue patterns on NASA TLX scales ....................................................... 65 Figure A.1 VIMEE gripper’s functionalities in a flow diagram ................................................... 87 Figure A.2 Gripper’s overall mechanical architecture .................................................................. 88 Figure A.3 PCB shield design ....................................................................................................... 90 Figure A.4 Load cell calibration within a small range .................................................................. 92 Figure A.5 Load cell calibration within a medium range ............................................................. 92 Figure A.6 Load cell calibration within a large range .................................................................. 93 Figure A.7 Evolution of the VIMEE gripper ................................................................................ 94 xiv List of Abbreviations ANOVA Analysis of variance CARIS Lab Collaborative Advanced Robotics and Intelligent Systems Lab CFRP Carbon fibre reinforced polymer EOAT End-of-arm tooling HRI Human-robot interaction LED Light emitting diode ROS Robot Operating System UBC University of British Columbia VIM Vacuum infusion moulding VIMEE Vacuum infusion moulding end-effector xv Acknowledgements I wish to thank my committee members, Dr. Machiel Van der Loos, Dr. Kefei Wen, and Dr. Ahmad Mohammadpanah, who have been more than generous with their expertise and precious time. Thank you for agreeing to serve on my committee to examine my thesis and defense. I wish to express my sincere gratitude specially to my supervisor, Dr. Van der Loos, for his invaluable guidance, support, and feedback throughout my entire time in the UBC CARIS Lab. His commitment to excellence in education and research has greatly enriched my learning experience and shaped my academic path in a positive direction. I am grateful to him for creating a friendly, supportive, and mentally healthy research environment. It was a fruitful journey to explore various novel research aspects in the field of HRI, while learning from his invaluable insights. I would like to acknowledge and thank all the members of the CARIS Lab for their participation in my study. Thank you to Wesley Chan and Maram Sakr for their insightful guidance and technical support. Thank you to Mahsa Khalili for her thorough feedback on my writing and data analysis. Thank you to Mark Dong and Yushan Liu for their significant contributions to the trajectory planning on the KUKA arm and robot cues research. Kyle Ah Von, Patrick Thangarajah, and Evin Li volunteered countless hours to contribute to the study in this thesis, as well as the early prototyping work on the VIMEE gripper. The UBC 2019s Capstone team #7 deserves credit for the development of the early prototype of VIMEE. Thanks to our industry partner, the German Aerospace Centre, DLR, and their researchers for contributing to the gripper frame design and sharing their knowledge on CFRP manufacturing to guide our research. I am deeply grateful to the UBC Mechanical Engineering Department, the Natural Sciences and Engineering Council of Canada, and the UBC Designing for People program for providing financial support. Lastly, a special feeling of gratitude goes to my loving parents and girlfriend for their ongoing support, understanding, and encouragement. xvi Dedication To the loving memory of my paternal grandfather and maternal grandmother, whose lives and accomplishments continue to inspire me.1 Chapter 1: Introduction In recent years, the progress in collaborative service robots has been primarily attributed to the rapid development of multi-sensory end-of-arm tooling (EOAT) systems [1]. EOATs are becoming useful robotic devices for a wide area of interest for engineers, scientists and clinicians, such as industrial manufacturing applications [2], undersea grasping and manipulation [3], minimally invasive surgery [4], and assistive services [5]. Industrial grippers are one of the most commonly used robot-EOATs that provide increased autonomy and safety for manifold manufacturing and assembly practices [6]. The collaboration with End-of-Arm Tools (EOATs) empowers humans to delegate heavy-duty tasks, such as managing repetitive cycles, transporting heavy loads, and operating in extreme environments, thus freeing humans to focus on high-level decision-making. With the rise of modern industrial strategies, including Industry 4.0 [7] and intelligent manufacturing, researchers have directed their focus toward the development of robust and reliable robotic workforces. Robot-EOATs have facilitated the availability, flexibility, and efficiency of the manufacturing process [6]. The associated demand for grasping tools has been on the upswing in recent years. As per the World Robotics Industrial Robots report, more than 3 million industrial EOATs were employed in factories, laboratories, and various commercial settings in 2021 [8]. Additionally, 384,000 units were shipped worldwide in 2020, with a modest 0.5% growth in new robot sales despite the global pandemic [8]. Especially within the manufacturing sector, these 2 robots offer significant technological and economic benefits to their clients and stakeholders. In the realm of composite material manufacturing, a multitude of innovative materials, such as carbon fibre reinforced polymer (CFRPs), have been widely used in the automotive and aerospace industries. CFRP is a composite material that combines the strength and stiffness of carbon fibers with the versatility of a polymer matrix. Due to its exceptional mechanical properties and lightweight nature, this composite material plays a vital role in various applications. For instance, automobile bumpers employ this material to withstand both tensile and compressive loads, thereby enhancing fuel efficiency and engine speed [9]. More importantly, the airframes of long-range aircraft are comprised of CFRPs to provide corrosion resistance, high durability, and high-temperature efficiency [10]. In the process of manufacturing CFRP, Vacuum Infusion Moulding (VIM) stands out as one of the most promising composite manufacturing techniques. This process involves infusing liquid resin into a dry fiber preform under vacuum pressure to form a solid, durable, and lightweight composite structure. The setup and equipment are summarized in Figure 1.1. Layers of dry reinforcement materials are placed within a mould, which is then covered with a plastic vacuum bag on top. A vacuum pump generates a pressure differential that facilitates the uniform flow of resin through the reinforcement layers. After the reinforcement is fully impregnated with resin, the assembly is set for curing at room temperature, allowing the resin to solidify and harden, forming the composite 3 structure. This technique has been widely adopted for large-scale CFRP manufacturing of aircraft bulkheads. Based on research conducted by the German Aerospace Centre (DLR) on materials and processing technology, the utilization of optimized vacuum infusion automated manufacturing technology in 2010 led to a reduction of approximately 40% in manufacturing costs and about 30% in weight for metal-constructed fuselages, compared to traditional manufacturing techniques [11]. Figure 1.1 Set-up and equipment for vacuum infusion moulding process [12] One of the critical steps is manipulating the vacuum bagging film and forming proper pleats, which allow the vacuum bag to move and conform properly to the part under vacuum. However, this process requires sophisticated manipulations of film materials. The current pleat manipulation process at DLR requires at least two trained technicians with the expertise to manually form and migrate pleats to remove air pockets at corners and fillet areas. This manual step dominates the production process and increases production costs and rejection rates. A promising solution is to utilize industrial EOATs to alleviate the physical demands of the pleating tasks. 4 The rapid development of robotics and automation in manufacturing has yielded significant efficiency and productivity improvements. However, numerous processes have not yet been fully automated due to their inherent complexity and flexibility. For instance, the formation and migration of pleats with dynamic and unpredictable geometries over moulds can be complex and specialized. Such tasks require a combination of human decision-making, problem-solving, and expertise, along with the precision and repeatability of robots. Collaborative robots with EOATs can serve as human-assisted systems in such Human-Robot Interaction (HRI) scenarios to alleviate the physical demands on human operators. Nevertheless, the current interfaces, such as computer consoles and teach pendants, are complex, constraining, and lacking intuitiveness for real-time interactions [13]. As a result, operators need appropriate training in order to understand and engage with these robots in an effective manner. To facilitate the process of human-robot collaboration, researchers focus on improving the robots' interpretability by establishing communication through signs to convey information. Visual signals, such as blinking or coloured LEDs, are frequently used to report the robots’ task states in real-time [14]. Peters et al. conducted an observational study to develop a concept of spatial prompting robots [15]. They found that users prefer to have visual indicators on the mobile robots to signal in which direction it will drive. Visual cues are also commonly used to mark specific functions. For instance, the design of robots’ mechanical joints is often visually emphasized to display the mechanical function (e.g., form follows function) [14]. Moreover, visual cues are widely applied 5 in HRI to improve users’ perception and understanding of the robots’ status during collaboration tasks. Arévalo et al. presented the design of robot visual cues in augmented reality, which can assist pick-and-place tasks, avoid attention splitting, and increase situational awareness during human-robot collaborations [16]. Thus, visual cues play a critical role in HRI, providing users with relevant information and facilitating effective communication between the user and the robot. A well-designed robotic system can facilitate user experience and task performance, but it can also become a source of disruption when the HRI is not properly designed. In the context of CFRP manufacturing, a multitude of resources come into play, including humans, machinery, materials, equipment, tools and information. The inherent complexity of this industrial application highlights the critical need for effective collaboration between humans and robots [17]. Given this circumstance, designing appropriate robot cues that facilitate seamless information exchange becomes a crucial undertaking. Therefore, I focus on developing visual cues for industrial robots to communicate their sensor input while performing specialized manufacturing functions (e.g., loading and migrating pleats) in the context of CFRP vacuum bagging process. This thesis explores the research question, “How should industrial robots communicate their behaviours to operators in shared workspaces?” For the robot research platform, I conducted the design of an industrial EOAT named the Vacuum Infusion Moulding End-Effector (VIMEE) [18] for the manipulation of CFRP vacuum bagging 6 process. As illustrated in Figure 1.2, VIMEE is a two-jaw roller-based gripper equipped with integrated sensors, actuators, and control units designed to detect and manipulate the composite material while operating in tandem with a KUKA LBR IIWA robot arm [19]. A testbed was created using a transparent plastic nylon sheet, which was placed on a flat surface with edges sealed onto the surface with adhesive to simulate the vacuum bagging setup. The gripper rollers can rotate in both directions to accomplish the tasks of pleat formation and migration. Figure 1.2 The structure of the VIMEE gripper developed for the CFRP vacuum bagging process, (a) the simulated model of the VIMEE gripper; (b) the layout of electronic components The purpose of the visual cues designed for this thesis is to communicate the gripper's force information to operators. Based on VIMEE's early prototype testing, the applied grip force was identified as a critical parameter in the manipulation of CFRP bagging material [18]. It is important to continuously adjust the grip force within an appropriate range that allows the roller system to effectively complete the pleat loading and migration (i.e. minimizing slippage or jamming). The robot was programmed to allow operators to manipulate the VIMEE gripper based on the visual force feedback indicator in both semi-automatic and automatic modes. During the vacuum bagging 7 process, the programmed functions, including loading and migrating pleats, can automatically control the motion of the robot arm and gripper rollers based on the real-time data of grip force and pleat height detected by the sensors. The semi-automatic mode enables manual grip force adjustment using a joypad. In the automatic mode, the gripper is capable of autonomously adjusting the force throughout the task. The operator can observe the visual feedback and collaborate with the robot to prevent slippage and jamming conditions. The detailed experimental procedure is described in Section 3.3. 1.1 Thesis Outline Chapter 2 presents the background through a literature review regarding the development of artificial cues in robotics. The literature explores different communication modalities in verbal and non-verbal HRI applications. The literature explores various communication modalities in both verbal and non-verbal HRI applications. Existing literature identifies research gaps and motivates the objectives of this study, which guide the design of robot visual cues. Chapter 3 describes the development of the VIMEE gripper in the context of a simulated CFRP manufacturing vacuum bagging process. Four patterns of novel LED cues based on light brightness, colour gradient, loading progress bar, and flashing light are designed to visually communicate the robot’s grip force feedback. This chapter also describes the experimental procedure for human-robot collaboration tasks and the data analysis of the survey evaluation. Chapter 4 investigates the evaluation results using statistical analysis techniques. The performance of each LED cue is evaluated in terms of 8 suitability, interpretability, and workload demand. The findings based on the statistical results are then discussed in Chapter 5. This chapter also highlights the corroborations and differences between this study and the existing literature, along with the challenges and limitations encountered throughout the experiment process. Chapter 6 concludes this thesis with implications for the designed robot visual cues in the field of HRI, as well as the directions for future work. 9 Chapter 2: Background and Motivation Chapter 1 introduces the trends in industrial robots and CFRP manufacturing technologies, as well as the definition of HRI in the context of a simulated vacuum bagging process. This chapter explores the literature on various existing human-robot communication techniques that provide motivation for the HRI design in this study. A key research objective is to develop robots that are easy to interact with and understand by users. To achieve this, researchers aim to interpret robot behaviours using meaningful expressions and movements that resemble human-human communication language and body cues [20]. In the context of HRI, these forms of communication are artificially implemented in robotic systems. Section 2.1 reviews the literature on verbal and non-verbal communication modalities and their role in guiding the HRI design for the industrial EOAT in this study (Chapter 3). The overall research objective is outlined in Section 2.2. 2.1 Artificial Cues in Robotics This section reviews the literature on artificial signals and how robots use different communication modalities to explicitly express their interactive behaviours in a clear and effective manner. Section 2.1.1 explores the verbal and non-verbal modalities commonly employed by collaborative social robots. Section 2.1.2 focuses on the study of visual cues through LEDs, which has demonstrated their crucial role in non-verbal HRI for non-anthropomorphic robots in complex environments. 10 2.1.1 Modalities for Human-Robot Communication Interactive robots are capable of communicating with human users by following specific rules, hence producing appropriate behaviours in accordance with users' requirements [21]. This can be accomplished through various communication modalities based on natural language processing, gesture recognition, and computer vision [22]. These technologies can be integrated into robots to dynamically represent the system's functional state and alter the behaviour of users in different HRI scenarios. Researchers have focused on establishing effective forms of communication through verbal and non-verbal techniques for robot decision-making and collaborative tasks [23]. Verbal human-robot communication is a widely used interaction modality that leverages natural language processing technology to enable robots to communicate and collaborate with human partners in shared workspaces. The goal of this communication modality is to improve the robot's social acceptability by establishing a natural and human-like interaction. Gnjatović et al. proposed an adaptive and assistive dialogue strategy based on automatic speech recognition and text-to-speech synthesis [24]. Their robot was able to process linguistic contents spoken by users and collaborate with them to complete the manipulation task. The dialogue system managed the HRI and processed spontaneously formulated user utterances, and adapted to different user speech patterns. The paper demonstrated the communication efficiency and robustness of their proposed robot dialogue system. Jia et al. developed a natural language-based robot programming framework implemented on a mobile manipulator system [25]. This system was able to learn from 11 the user’s verbal instructions and execute multiple on-site manufacturing tasks. Their experiment showed the effectiveness of verbal interaction in programming robots for different collaborative tasks. Nikolaidis et al.’s research focused on mutual-adaptation verbal commands that allow the robot to inform the user about its internal state during decision-making [26]. This enhanced the robot’s trustworthiness and the user’s situational awareness during collaborative tasks. The abovementioned research studies explored human-robot verbal communication based on different natural language techniques. Many advantages have been highlighted in terms of communication efficiency, effectiveness, and social presence. However, there are still technical limitations to implementing natural language technology in current robot applications. For instance, robots are facing difficulties in capturing sound from a distant voice source [27], cancelling ambient acoustic noise [28], handling speech interruption from users [29], and recognizing the associated voice commands from multiple acoustic sources [30]. Additionally, the user’s age can also impact the performance of human-robot verbal communication. Hämäläinen et al. discussed the challenge of recognizing children's speech due to a lack of training data and benchmarking for children's voice recognition models that identify pitch characteristics of children’s voices and speech disfluencies during their developmental changes [31]. Natural language interactions are not yet practical for elderly populations either, particularly for dysarthric speakers, as their impairments to normal communication flow and slow or slurred speech may result in distorted meaning or misunderstanding [32]. Hence, building a reliable and natural HRI requires a well-12 designed interaction modality that can adapt to users' needs in a supportive manner. In our social environment, non-verbal behaviours including gestures, postures, facial expressions, and eye movements are commonly interpreted as an expression of intention. These behaviours have six main functions that influence the conveyed meaning by repeating, contradicting, substituting, complementing, accenting or regulating the conversational flow [33]. Research indicates that over 60% of the meaning communicated in interactions is derived from non-verbal behaviours [34]. Integrating these behaviours into robots designed for human interaction has become a promising approach to fostering more intuitive and accessible user experiences, particularly for novice users. A literature review in HRI highlighted four key features of non-verbal communication: kinesics, proxemics, haptics, and chronemics [35]. These features can be incorporated into robots for collaborative tasks through interface design [36], interaction strategy [37] and collaboration models [38]. Kinesics-based HRI encompasses the use of robots’ body movements, positioning, visual and facial expressions to convey shared meaning [35]. Krenn et al.’s research on human-robot pick-and-place tasks found that gestures and light signals are effective non-verbal communication cues [39]. In their study, an industrial robot arm was able to produce human-like deictic gestures and project light signals on its gripper in a virtual reality environment to indicate it is the user’s turn to pick up an object. The results showed that both non-verbal cues were helpful in attracting the user’s 13 attention and correctly interpreting the robot’s intention. Another non-verbal communication strategy is based on proxemics, which leverages the variations in social distances between objects, users and robot agents to affect human perception [35]. Shi et al. investigated the robot’s distance and velocity constraints associated with levels of safety and comfort perceived by users [40]. The results of their experiments showed that a safe distance-dependent velocity was critical for high-speed robots moving among people. Users felt unsafe and uncomfortable when the robot approached too fast or too close. Haptics, also known as tactile interaction, is another fundamental mode of non-verbal communication that conveys messages through the senses of touch [41]. Fukuda et al. conducted a study on the effect of robot touch on a person’s feelings during an experimental economic game called ultimatum game [42]. The study employed a Robbie-mR2 upper-torso humanoid robot to interact with users in two conditions: a touch condition, where the robot gently touched the participant's arm during unfair proposals, and a no-touch condition in the same scenario. Participants' sense of unfairness was measured by the amplitude of medial frontal negativity using electroencephalogram. The results suggested that haptic feedback from robots can be an effective mode of non-verbal communication to enhance users’ positive feelings during decision-making. 14 Chronemics in non-verbal communication focuses on a person's experience with time [43]. Relevant research in HRI has explored the effect of hesitation in robot action on users during conflict resolution in human-robot collaboration. Moon et al. investigated the communicative capabilities reflected in the robot’s trajectory characteristics of hesitation gestures for users with limited robotics knowledge during collaborative reaching scenarios [44]. The researchers recorded human hesitation gestures for reach-and-retract tasks and programmed them into a CRS A460 industrial robot arm. Their experimental results showed that users were able to effectively recognize the anthropomorphic hesitation gestures demonstrated by the articulated robot, even when only the human’s wrist trajectories were replicated. In conclusion, the aforementioned literature review on the topic of non-verbal communication investigated the effect of various modalities in robots, such as gestures, postures, touch, and light signals. The role of non-verbal communication is of indisputable importance for coordinating actions in collaborative activities [45]. It allows people to directly perceive the information communicated through robot movements and visual representations. In the context of social HRI, non-verbal behaviours have shown positive effects in conveying awareness, clarifying ambiguous meanings, augmenting verbal references, promoting positive feelings and safety. Robots can take advantage of these behaviours to enhance the effectiveness of collaborations and reduce the communication barriers between humans and robots. As the demand for collaborative tasks increases, robots’ non-verbal behaviours significantly improve people’s performance in terms of 15 recall accuracy and completion time [46]. However, incorporating non-verbal behaviours into the design of robots that exhibit human-like social cues through movements and lights can be challenging. Most human-human interactions rely on non-verbal behaviours that are instinctual and consistent across cultures [47]. To achieve fluid human-robot communication, robots are required to learn the specific social norms that are adequate for the designed application [48]. 2.1.2 Non-verbal Visual Cues The preceding section reviewed verbal and non-verbal communication modalities in the field of HRI. The majority of research in this area focuses on enhancing the efficiency and reliability of communication in social settings. In addition to anthropomorphic social robots, other forms of robotic systems shown in Figure 2.1, such as zoomorphic robots [49], powered prosthetic limbs [50], aerial drones [51], mobile service robots [52], and industrial robot [53] have become increasingly prevalent in a broad range of HRI applications. However, people may find it challenging to interpret the communication cues embodied by these robots due to their distinct appearances and limited human-like behaviours [54]. Interactions with robots in complex and dynamic environments, such as a room filled with obstacles or a noisy open space, can be challenging, as the robots’ movements and natural language interactions may be limited or hindered by the surroundings. As a result, non-verbal expressions, especially visual cues through LED displays, have come to be an important communication channel in these contexts. This section explores the use of visual cues by non-anthropomorphic robots to convey their behavioural 16 information or task states during human-robot collaboration. Figure 2.1 Types of non-anthropomorphic robots that use visual cues for HRI: (a) quadruped robot; (b) prosthetic hands; (c) flying robot; (d) mobile service robot; (e) industrial robot arm. Images with permission from Wolters Kluwer Health, Inc., 2012; from the author, Bilge Mutlu, 2015; from IEEE, 2018, 2022 Prior HRI studies have shown that for ensuring users’ physical safety, it is not sufficient to provide them with psychological safety, which refers to feeling comfortable to engage in open and honest communication and to establish a positive relationship with the robot agents [55]. Therefore, it is essential to build transparency of a robot’s capabilities, limitations, awareness and intent to enhance mutual understanding. Multiple studies have investigated the implementation of visual cues through LEDs in HRI to assist humans in understanding robots’ behaviours. Sanoubari et al. designed LED cues for a quadruped robot to communicate its awareness of surrounding obstacles and people, intent and error information [49]. The messages were mapped to corresponding LED behaviours using the RGB colour space, the length of the LED strip and the frequency of the light pattern. The researchers compared communication through LEDs, gesture cues, and a combination 17 of both in five experimental scenarios: the status of processing, noticing, turning, greeting and listening behaviours. The results showed that LED cues are more effective in communicating the robot’s task states and enhancing users’ psychological safety compared to gesture cues. In the context of manipulating a prosthetic hand, Engeberg et al.’s research focused on designing visual feedback to prevent slippage during grip motion [50]. To improve the precision and intuition of prosthesis control for amputees, the researchers used a red-green bicolour LED to provide real time visual indication of the grip force being applied. Through their experiments, the research team evaluated the effectiveness of the visual force feedback by having participants manipulate a powered prosthetic hand to grip a fragile object. The results demonstrated a significant reduction in the breakage and slippage of the grasped object when visual feedback was used. This research also provided evidence for the usefulness of visual force indication through LEDs in facilitating the manipulation of prosthetic hands. Szafir et al. developed a set of LED signals for flying robots to explicitly communicate directionality during interactions with humans in close proximity [51]. The robot's intentional movements were encoded in abstract LED behaviours displayed via a ring of lights surrounding the quadcopter. The researchers conceived of four LED signal patterns, including blinker design, warning beacon, thruster signal, and gaze signal, to communicate the robot's flight trajectories during transitions between hovering and moving states. Study participants evaluated the 18 performance of LED signals, and a no-signaling baseline as the flying robot approached programmed targets in different locations. The outcome of this study revealed that LED signaling patterns improved the predictability of the robot's movements compared to the baseline. The blinker, thruster, and gaze signals significantly outperformed the baseline in terms of response time and predictive accuracy. Fernandez et al. conducted research on the design of LED cues for mobile service robots in crowded and human-populated environments [52]. The goal was to visually indicate the robot's intended lane-changing behaviours. The mobile robot was equipped with two LED strips that blinked on either side when the robot makes turns. In a hallway environment, participants and the robot approached each other from opposite ends. Participants were asked to coordinate their movements and pass the robot when a conflict occurred on their paths, based on observing the LED signals from the robot. The experimental results demonstrated that using LED-based signals to indicate the mobile robot's intended direction of travel effectively disambiguated the robot’s behaviours and improved human interpretability in human-robot special interactions. Furthermore, Hetherington et al. designed and evaluated non-verbal robot motion legibility cues that communicate a mobile robot’s motion intention to pedestrians in public spaces [56]. The researchers used projected arrows and flashing lights to indicate the robot’s forthcoming motion and major waypoints. The results showed that projected arrows can communicate path information, 19 while flashing lights can communicate goal information of a mobile robot. Implementing these motion legibility cues to a mobile robot’s design facilitated the social acceptance of human-robot spatial interactions in public spaces. The reviewed literature suggests that using non-verbal visual signals is an effective human-robot communication modality in complex environments where traditional communication methods may be impeded by obstacles and/or noise. Additionally, LED cues have become a signaling paradigm in human-robot interactions, in which people may not have a well-established understanding of the messages conveyed by non-anthropomorphic robots [49]. However, the study of visual cues in industrial robots is still an under-explored research area with limited literature and applications. Existing studies are primarily restricted to implementing visual cues within a simulated virtual environment, rather than in a real-world industrial setting. For instance, Sreng et al. presented the use of visual cues of contact to improve interactive manipulation of virtual objects in industrial assembly/maintenance simulations [57]. Arevalo et al.’s research explored visual cues in the teleoperation of a robotic arm in manipulation and grasping tasks by using augmented reality [53]. adapting the existing HRI cues to effectively accommodate the high production flexibility driven by increasingly diversified customer needs in the manufacturing industry is challenging [58]. Thus, further research is necessary to enhance human-robot interactions in industrial settings. 20 2.2 Research Objectives The literature reviewed in Section 2.1.2 shows that mapping robots’ informative messages to abstract visual behaviours significantly improves the predictability and interpretability of robots' status during human-robot interactions. LED-based visual cues can serve as an effective means of communication for non-anthropomorphic robots in complex and volatile environments. However, limited evidence exists to prove the effectiveness of LED cues for industrial robots, such as the VIMEE gripper. In light of CFRP manufacturing, maintaining an appropriate grip force during the vacuum bagging process is crucial for the successful manipulation of fabric materials. Insufficient or excessive grip force can directly impact the robot's manipulation task (e.g., the pleating material slipping and pulling out from the gripper) as well as the quality of the final product. Therefore, the overarching research objective is to investigate the design of LED cues for a manufacturing robot gripper, VIMEE, to communicate its force feedback information during the pleating procedure of a simulated CFRP manufacturing vacuum bagging process. 21 Chapter 3: Method and Materials In this study, I focused on designing visual cues that can be integrated onto a robotic gripper (e.g., VIMEE) to provide force feedback information during CFRP manufacturing. The designed visual cues explicitly communicate the robot's applied grip force information to the users, aiming to enhance the interpretability of the robot's behaviour. The robot described in Section 3.1 served as the research platform for prototyping the visual cues based on four different LED animation patterns. Each LED cue pattern was tested while the robot was operated in semi-automatic mode and automatic mode. Section 3.2 elaborates the design of these LED cues in detail. A user study was conducted to test the following hypotheses: H1.1 : All LED cues are equally suitable for indicating the robot’s information. H1.2 : All LED cues in the semi-automatic mode are equally suitable to those in the automatic mode. In addition to testing these hypotheses, this study investigated additional research questions that are listed below: 1. Which LED cues are more suitable for indicating the robot's force ? 2. Is the LED cue pattern more suitable than the condition with no visual feedback? 3. Do participants comprehend the information conveyed by the robot through the LED cues? 4. Do the LED cues affect participants' workload during the human-robot collaboration task? 22 Human experiments were conducted to test the above hypotheses and research questions. The details of the experiment and questionnaire design are described in Section 3.3 and 3.4. The remainder of this chapter presents the participant recruitment and the data analysis procedures. 3.1 Robot Research Platform This section describes the robot research platform designed for this study. Given the high labour and time costs associated with the current manual vacuum bagging process in CFRP manufacturing, DLR and UBC’s Collaborative Advanced Robotics and Intelligent Systems (CARIS) Lab collaborated on the development of the VIMEE gripper to reduce the physical demands of the task incurred by human operators. The two main pleating functions implemented in the gripper are outlined in Section 3.1.1 and 3.1.2. The programmed operating modes are presented in Section 3.1.3. The details of the gripper design are elaborated in Section 3.1.4 3.1.1 Pleat Loading Function The first main function developed on the VIMEE gripper is loading material slack and forming a pleat inside the gripper jaws. This step mimics the technicians' hand motion to smooth and manipulate the small wrinkles on the surface of the vacuum bag material prior to the pleat migration process. As the procedures illustrated in Figure 3.1 (a), a small pre-formed wrinkle is created and positioned between the gripper rollers. Then, the gripper jaws are closed to gradually apply force on the wrinkle. As the gripper rollers pinch the wrinkle at the set force, the programmed 23 function activates the rollers to spin inwards to load the pleat. The material slack is gathered and rolled into the gripper jaws, forming a 15 cm vertical pleat. The gripper automatically ends the pleat loading function once the pleat is properly formed. Figure 3.1 Simulated CFRP pleating procedure: (a) loading a wrinkle to form the pleat; (b) migrating the pleat horizontally to the right 3.1.2 Pleat Migration Function During the vacuum bagging process, pleats are typically positioned along corners in the part to prevent resin pooling, which can lead to the bridging of vacuum material across those corners. Technicians must manually manipulate the pleat over the mould, particularly around the corner area. Therefore, the pleat migration function simulates this process on the VIMEE gripper. As the Figure 3.1 (b) shown, the gripper performs this process by spinning both rollers in the same direction, while the KUKA arm translates the gripper along a linear path. During the migration, the rollers grab pleating materials to generate opposing shear forces inside the pleat. Therefore, a sufficient and appropriate amount of clamping force and roller torque are essential in providing friction resistance to migrate the pleat. 24 3.1.3 Operating Modes The pleat loading and migration functions can be operated in both semi-automatic and automatic modes. In the automatic mode, the gripper can automatically adjust the grip force and maintain it at the programmed level during the tasks. However, in the semi-automatic mode, the operator needs to adjust the gripper jaws and control the grip force using a joypad. For both operating modes, the gripper rollers are automatically triggered to perform the commanded functions when the grip force reaches the programmed level. The gripper detects the pleat height and roller speed to determine the occurrence of pleat slippage or jamming conditions. The trajectory of the KUKA robot arm is activated only when the gripper rollers are engaged to migrate the pleat. 3.1.4 VIMEE Gripper Design The mechanical design of the VIMEE gripper adopts a roller-based structure, which can achieve a pinch grasp motion for pleat loading and migration tasks. The dimension of the gripper is 40 cm x 30 cm x 12 cm, and the overall weight is about 5 Kg. The gripper’s major components are illustrated in Figure 3.2. A customized flange at the top of the gripper is designed to fit the mount on the KUKA LBR IIWA robot arm, allowing for integration. The powered jaw on the right side can push the left compliant jaw to rotate around a pivot joint, while the horizontal grip force is converted into a vertical load force measured via the load cell. Based on a Wheatstone bridge configuration, its internal strain gauges with a measurement range of 0 to 23 kgf can sense the resultant grip force from the compliant jaw. The load cell assembly contains a preloaded spring 25 mechanism that modulates the powered jaw’s applied grip force. A two-jaw aluminum frame and cylindrical clamping roller with high-friction Neoprene rubber (durometer 40 A, μ= 0.8) provide the current VIMEE prototype with the capability to manipulate the fabric pleats. Furthermore, the powered jaw is driven by a NEMA 17 stepper motor [59], which can generate a maximum grip force of 150 N. An array of Time-of-Flight (ToF) sensors [60] mounted inside the powered jaw can detect the height of the pleat. When the gripper jaws close, two DC gear motors control the rubber-coated rollers to manipulate the pleating materials. The roller design is constructed with a two-piece cylindrical clamping mechanism to prevent slipping of the adhesive-backed rubber on the rollers. An optical encoder attached to the roller detects its rotational speed. The VIMEE gripper can monitor the pleat's status based on sensor feedback of grip force, roller speed, and pleat height. With the integration of these sensors and actuators, the gripper collaborates with the KUKA robot arm to execute the programmed pleat loading and migration functions in the simulated vacuum bagging process. The control program was embedded in the Robot Operating System (ROS), in coordination with the control of the movement of a KUKA robotic arm. The detailed software implementation is attached in Appendix A. 26 Figure 3.2 Design layout of the robot research platform: the VIMEE gripper To successfully load and migrate the pleat, the grip force is critical in providing friction resistance during the manipulation of the pleat. Based on the results of our early prototype testing, Table 3.1 summarizes VIMEE’s task performance under three different grip force ranges [18]. To mitigate roller slipping and jamming conditions, the grip force must be maintained at a sufficient level between 25 N and 40 N. The grip force and robot status information were mapped to LED indicators to provide cues to the operator, as described in the next section. 27 Table 3.1 VIMEE task performance in different grip force ranges Grip Force Range Robot Status Description Insufficient (< 25 N) Slipping Slight slippage occurs between the pleat and roller surface due to insufficient grip force. Sufficient (25 N – 40 N) Optimal Rollers successfully load and migrate the pleat with their maximum torque. Excessive (> 40 N) Jamming Rollers are stalled or rotate in snatches due to excessive grip force. 3.2 Design of Robot LED Cues This section elaborates on the design and functionality of the visual cues implemented in this study. During the pleat loading and migration tasks, grip force was found to be a critical parameter that can affect the robot’s task performance. Therefore, I prototyped four LED-based cue patterns to communicate the robot's grip force information in pleat manipulation tasks. The light source is a programmable NeoPixel LED RGB strip with 10 pixels connected to an Arduino microcontroller. This LED stick is mounted horizontally on the powered jaw of the gripper to provide force feedback information. The code programmed on the Arduino translates the output data from the load cell sensor into digital signals for the LED modules to execute light animation. Based on the existing expressive lights designed in other HRI research, I animated four LED cue patterns, including light brightness, loading progress bar, flashing light and colour gradient, to indicate grip force information. For the colour gradient animation, I selected a colour scheme that is intuitive, similar to traffic lights, using yellow, green, and red to indicate the corresponding force 28 ranges. Crozier’s research found that blue is more neutral and less susceptible to extremes of judgment compared to other colours [61]. Therefore, the remaining patterns, including light brightness, loading progress bar, and flashing light, were animated in blue to minimize the potential bias from colours. The following sub-sections describe the design of each LED cue pattern. 3.2.1 Light Brightness As Engeberg et al.’s research described in Section 2.1.2 showed, the use of LED light intensity was able to enhance the visual feedback for controlling the grasp force of a prosthetic hand [50]. With this result, the first animation pattern presented in Figure 3.3 was designed based on light intensity. I mapped the grip force from lowest to highest to a range of LED brightness from 0 to 220 mcd. As the applied gripper force increases, the blue LED strip gradually becomes brighter to provide the participant with visual feedback on the grip force level. As the force decreases, the LED strip becomes dimmer accordingly. 29 Figure 3.3 The LED cue based on the light brightness effect: (a) the LED strip with brightness in the insufficient force range; (b) the LED brightness in the sufficient force range; (c) the LED brightness in the excessive force range; (d) the mapping of LED brightness across different force ranges 3.2.2 Loading Progressing Bar The second LED pattern was animated by using the progress light effect, as illustrated in Figure 3.4. As the powered jaw moves to the left to apply force on the compliant jaw, a right-to-left progress bar grows to indicate the generated gripper force in real-time. The growing portion of the strip that is also lit in a blue colour. A similar animation light effect was used in Baraka et al.’s research to indicate the task progress of a collaborative mobile robot [62]. Their experimental results show that expressive lights significantly improved people’s understanding of the robot’s 30 state and actions when minimal contextual clues were provided in the experimental scenarios. Figure 3.4 The LED cue based on the loading progress light effect: (a) the LED strip with the progress light in the insufficient force range; (b) the progress light in the sufficient force range; (c) the progress light in the excessive force range; (d) the mapping of the number of LED pixels across different force ranges 3.2.3 Flashing Light Figure 3.5 depicts the mapping of LED flashing rate to indicate the grip force information. This cue pattern was designed based on Sanoubari et al.’s research, which used the flashing frequency of a light pattern to indicate the activity state of a quadruped robot as one of robot’s affective expressions [49]. Their results show that the flashing LED was effective in communicating the robot’s states. Researchers mapped the activity value to a range of frequency between 400ms (~1.3 31 Hz) and 4000ms (~0.1 Hz) to initiate the duration for which the LEDs were on each time. Therefore, I used the same LED flashing rates to indicate the change in gripper force. Figure 3.5 The mapping of the LED flashing rate across different force ranges 3.2.4 Colour Gradient The design of the colour LED cue, as illustrated in Figure 3.6, is based on the concept of using colour variations to convey the robot's informative messages [49]. Engeberg et al.’s research also demonstrates a significant improvement in controlling the grip force applied by a prosthetic hand to upper limb amputees when the colour LEDs are implemented to visualize the force feedback [50]. In this cue pattern, the gripper force was mapped to the RGB colour space, ranging from red to green. I used red, green, and yellow to indicate the levels of excessive force, sufficient force, and insufficient force, respectively. 32 Figure 3.6 The LED cue based on the colour gradient effect: (a) the LED strip displays yellow colour in the insufficient force range; (b) the LED displays green colour in the sufficient force range; (c) the LED displays red in the excessive force range; (d) the mapping of the LED colours across different force ranges These four LED cue patterns were designed for use in both semi-automatic and automatic modes, where the participant collaborates with the VIMEE robot to achieve the required pleating tasks based on visual feedback. Since the previous reviewed literature had not directly compared these cue patterns in this particular context, I designed an experiment to analyze the performance of these patterns in order to fill this research gap. The details of the experimental setup and procedures are described in Section 3.3 and 3.4. 33 3.3 Experimental Scenario To simulate the vacuum bagging process in CFRP manufacturing, I created a testbed using a nylon bagging film. As the experimental setup shown in Figure 3.7, the nylon material was placed on a flat surface and its edges were sealed onto the surface with adhesive. The VIMEE gripper was mounted onto the KUKA robot arm, which moved linearly above the testbed to perform pleat formation and migration tasks. Figure 3.7 The industrial robot research platform on the simulated CFRP manufacturing vacuum bagging setup A gamepad was provided to the participant to control the robot during the experiment. Figure 3.8 shows the mapping of the control buttons to their corresponding functionality. The participant can use the gamepad to move the gripper jaw, activate the pleat formation or migration functions, and stop the robot accordingly. In the pleat formation task, the robot arm remains stationary while the 34 gripper rollers load the pleat from a small pre-formed wrinkle. In the pleat migration task, the robot arm moves the gripper along a preprogrammed 50 cm linear path on the testbed while the gripper rollers migrate the loaded pleat horizontally. A portion of the material slack was intentionally taped on the testbed to cause pleat slippage, allowing the participant to interact with the VIMEE robot and resolve the issue during the task. Figure 3.8 Joypad control map and functionalities This study evaluated four types of LED cues that were animated to indicate the gripper force in real-time during pleat manipulation. The experiment was conducted in two operation modes: semi-automatic mode and automatic mode. Participants needed to manually adjust the grip force using the joystick while operating the robot in the semi-automatic mode. In contrast, the automatic mode enabled the robot to adjust the force by itself. Each experimental trial included a pleat formation task and a pleat migration task. Additionally, as a control condition, this study included scenarios with no LED cues. In total, 10 trials per participant were included in this study. 35 3.4 Experimental Procedure and Questionnaire Design In this experiment, I designed a digital questionnaire to collect user feedback for each LED cue pattern in two task scenarios (semi-automatic mode and automatic mode). Figure 3.9 presents a flow diagram of the experimental procedure. Figure 3.9 Overview of the experimental procedure 36 After introducing participants to the background information about this study, I obtained their written consent following UBC Research Ethics Board protocol (Ethics ID: H22-03470). The consent form attached in Appendix B was provided to the participant for review and signing before starting the experimental task. During the explanation of the experimental design and the task procedure, Figure 3.8 and Figure 3.9 were presented to the participant. The first task requires the participant to use a gamepad to perform the pleat formation task under the semi-automatic operation mode. The participant presses the “▲” button to activate the pleat formation function. By observing the LED cues, the participant manipulates the stick on the gamepad to move the gripper jaw and adjust the grip force. Once the pre-programmed force range is reached, the rollers are triggered to load the pleat from a small pre-formed wrinkle. Once the pleat is properly formed inside the gripper, the gripper jaw automatically returns to the initial position to signal the completion of the pleat formation task. The participant then presses the “■” button to activate the pleat migration function, which enables the gripper to roll the loaded pleat linearly along a pre-programmed path. The participant is asked to use the gamepad to manipulate the gripper jaw to generate a sufficient amount of force to trigger the motion of migration. During the task, manual adjustment is required to maintain an appropriate grip force range. When the pleat slips out of the gripper and causes the robot to halt as programmed, the participant is asked to reposition the pleat inside the gripper and resume the migration process to complete the task. During each trial, the participant is also asked to fill out the survey questions related to the LED cue pattern. After evaluating the LED cue patterns in the semi-automatic mode, the participant proceeds with the 37 same task procedure to operate the VIMEE gripper in the automatic mode. In this mode, the robot can control its gripper force within an appropriate range and start the task automatically. The participant repeats the pleat formation and migration tasks to evaluate all LED cue patterns. To minimize data distortion and carry-over effects caused by sequencing bias, participants follow a Latin Square Design [63] to determine the order of the LED cues in the experiment. 3.4.1 Suitability Likert Scale After operating the robot with each visual cue pattern, participants responded to the first section of the questionnaire with the following question on a 7-point Likert scale: 1.1 How suitable is the visual cue pattern in indicating the robot's gripper force information? I proposed a suitability scale comprised of the Likert-type evaluation presented in Question 1.1. Participants choose the score that represents the level of suitability for the specific visual cue based on their general impression after each trial. The suitability scale was also used in a pilot online survey study with 22 participants to verify their preliminary understanding of the animated visual cue design. 3.4.2 Interpretability Likert Scale The study reviewed in Section 2.1.2 used a similar strategy to evaluate the acceptability of a social robot’s motion legibility cues [56]. To further investigate the participants’ interpretability of the 38 visual cues in this study, I proposed Questions 2.1, 2.2, and 2.3 to assess the cue pattern in terms of intuitiveness, legibility and interactivity. These are common sub-constructs of human interpretability in the non-verbal HRI studies reviewed in Section 2.1.2 (e.g., Fernandez et al. [52]). 2.1 How easily did you understand the robot's states through the visual cues? 2.2 How intuitive was the robot's communication to you? 2.3 Do the visual cues allow you to intervene the robot's pleating process with confidence? 3.4.3 NASA Task Load Index (TLX) The last section of the questionnaire comprises three questions on a Likert scale based on a NASA-TLX [64], which is a widely used assessment tool to rate perceived workload by participants. Some studies used it to measure participants’ mental workload affected by the visual cues during human-robot collaboration (e.g., Eimontaite et al. [65]). Therefore, I focused on assessing the mental demand, effort and safety in this study. The following questions are included in the second part of the questionnaire. 3.1 How mentally demanding was the task? 3.2 How hard did you have to work to accomplish your level of performance? 3.3 How insecure, discouraged, irritated, stressed, and annoyed were you? 39 3.4.4 Follow-Up Questions At the end of the questionnaire, participants answered three follow-up short questions: 4.1 Do the visual cues used in the pleating process have enough contrast to represent different force levels? 4.2 Can the cue patterns be appropriately distinguished from each other? 4.3 Are there any potential improvements you would like to suggest? Based on the performance of each LED cue pattern, participants can respond to these open-ended questions to elaborate on the pros and cons, and to provide additional comments about the design. 3.5 Participants The participants in this experiment mainly consisted of university researchers and students. They were mainly recruited from different research labs and design clubs. Participants sent emails to sign up for a 90-minute experiment session according to their availability. The study was conducted in the UBC CARIS lab. Each experiment session involved only one participant to test all levels of the LED cue pattern design. The questionnaire was distributed to the participants online using the Google Forms survey tool. Participants evaluated each LED cue after completing the robot operation task under each condition. They were allowed to withdraw from the experiment at any time during the study procedure. Appendix B includes the questionnaire used in this experiment. 40 Human experiments were conducted to test the above hypotheses and research questions. The sample size was computed through a priori analysis for the five cue patterns (light brightness, flashing light, loading progress bar, colour gradient, no light) under two modes (semi-automatic, automatic) using analysis of variance (ANOVA) estimates [66]. Based on the power analysis in Wright et al.’s study [67], a medium effect size (i.e., 𝑓 = 0.25), a power of 0.80, and a significance level of 0.05 were applied to determine the smallest sample size required for this experiment. The results indicate that a minimum of 22 participants would be needed in order to achieve the desired level of significance. To mitigate the potential effects of attrition, a total of 39 participants were recruited to operate the robot tasks and respond to the survey. All participants were able to fully complete the required experiment procedure in this study. 3.6 Data Analysis I analyzed the suitability scale data to test hypotheses H1.1 and H1.2. This analysis also answers the research questions, “Which LED cues are more suitable for indicating the robot's force compared to others?” and “Is the cue pattern more suitable with LED than without?” The suitability scale was analyzed by using a 2 x 5 repeated measures ANOVA test. This test was also applied to analyze the intuitiveness, legibility and interactivity scale data, which answer the research question, “Do participants comprehend the message conveyed by the robot?” The results of NASA-TLX scale were used to analysis the participants’ workload in terms of mental demand, effort, safety and effort. This provides the answer for the research question, “Do the LED cues affect 41 participants' workload during the human-robot collaboration task?” The within-subject factors were operating modes and cue patterns. As the example given in Sullivan et al.’s research, it is valid to consider the Likert-type items as interval-scale continuous data and then to perform parametric tests [68]. The general assumptions of a two-way ANOVA include normally distributed data, homogeneity of variance within each group, and sphericity [69]. Norman shows that parametric methods are robust with respect to violations of assumptions [70]. Despite the survey data collected in this study violating some of these assumptions, the test is still robust against departures from the assumptions. In Hetherington’s study, the researcher applied a mixed-model ANOVA test to analyze the data with such violations [71]. I conducted normality tests for each LED cue pattern. The results show that only the pattern with light brightness and the pattern with flashing light in the semi-automatic mode are normally distributed. The distributions of all other cue patterns are non-normal. However, according to the central limit theorem, the distribution of sample means tends to approximate a normal distribution when the population distributions are skewed and non-normal, as long as the sample size of each group is sufficiently large [72]. As a rule of thumb, a sample size of over 30 subjects is considered adequate [73]. Given that this study involved 39 subjects in each combination of factor levels, the ANOVA test is mathematically valid and robust to violations of normality. For the assumption of homogeneity of variance, I conducted Levene's test [74] which shows the error variance of dependent variable is unequal across groups. 42 Nonetheless, the ANOVA test is generally robust to violations of the assumption if the ratio of the largest to smallest group being less than 1.5 [75]. As the group sizes in this study are equal (i.e., 𝑁 = 39), thus it is valid to utilize ANOVA to analyze these data. The assumption of sphericity was tested by using Mauchly's test [76], which shows variances of the differences between all combinations of factor levels are equal. Therefore, the data in this study obey the assumption of sphericity to ensure the validity and reliability of ANOVA results. Appendix C includes the results of the assumption testing. To test the hypotheses and answer the research questions, post-hoc pairwise comparisons were conducted to analyze the scale data. The Bonferroni correction method [77] was employed to adjust the significance level to 𝛼 = 0.005 , which helps reduce the risk of a type I error by lowering the family-wise error rate (FWER) to 0.049. The p-values in the pairwise comparisons were also adjusted using the Bonferroni method. It is important to note that the follow-up questions regarding the performance of LED cues (Question 4.1 and 4.2) and future improvements (Question 4.3) were not subjected to statistical analysis. Therefore, this chapter primarily focuses on summarizing the statistical results based on the Likert-type items. 43 Chapter 4: Results This chapter elaborates on the results of the analysis following the abovementioned procedure. The statistical results of hypothesis tests regarding H1.1 and H1.2 as well as survey responses are presented. Appendix C attaches tables with complete results for the ANOVAs and the post-hoc pairwise comparison tests. The evaluation score is computed as the mean response to the Likert scale questions presented in Section 3.4.1, 3.4.2 and 3.4.3. I conducted an internal reliability analysis to verify the consistency of the results based on Cronbach’s α [78] for each combination of cue pattern and operation mode. The computed α coefficient of 0.836 shows that the results have high internal consistency, which are above the acceptable level at 0.7 [79]. 4.1 Suitability Scale Participants' responses to the suitability of LED cues were analyzed using a 2x5 mixed ANOVA. The results are used to test hypotheses H1.1 and H1.2. The ANOVA calculated the within-subjects' effects of operating modes and cue patterns based on the suitability score. Table 4.1 summarizes the statistical results for this test. These results show a significant main effect in the mode (𝐹(1, 38) = 11.749, 𝑝 = 0.001, 𝜂𝑝2 = 0.236) and the pattern (F(4, 152) = 93.650, p <0.001, ηp2 = 0.711). However, the interaction scenario is not further analyzed since the results are non-significant at the 𝛼 = 0.005 level, and its effect size of 𝜂𝑝2 = 0.023 is small [80]. 44 Table 4.1 2x5 Repeated Measures ANOVA on suitability scale Effect Effect DOF Error DOF F-statistic p-value Effect size 𝜼𝒑𝟐* Mode 1 38 11.749 0.001 0.236 Pattern 4 152 93.650 <0.001 0.711 Mode * Pattern 4 152 0.907 0.462 0.023 4.1.1 H1.1: All LED cues are equally suitable for indicating the robot’s information. The ANOVA tests detected a significant main effect of LED cue patterns on the suitability score (p < 0.001, 𝜂𝑝2 = 0.711). However, no significant interaction effect is found between cue pattern and cue mode. I conducted a post-hoc pairwise comparison with Bonferroni correction to determine which LED cues are significantly more suitable compared to the others. The results in Figure 4.1 illustrate that the LED cues with the colour gradient and the loading progress bar have the highest score on indicating the level of gripper force. Therefore, Hypothesis H1.1 can be rejected because the statistically significant evidence at 𝛼 < 0.001 shows different means of suitability scores between these LED cue patterns. * Partial eta-squared (𝜂𝑝2) effect sizes of 0.01, 0.06, and 0.14 are considered small, medium, and large, respectively [80]. 45 Figure 4.1 The mean suitability score based on the LED cue pattern Pairwise comparisons reveal that the LED cue pattern with the loading progress bar is significantly more suitable compared to other patterns, such as the light brightness (𝑀𝐷 = 1.359, 𝑝 <0.001, 𝑑 = 0.920 ) * , flashing light (𝑀𝐷 = 2.231, 𝑝 < 0.001, 𝑑 = 1.385 ), and the no light condition (𝑀𝐷 = 4.154, 𝑝 < 0.001, 𝑑 = 2.251). The results indicate that the loading progress bar has a slightly higher mean compared to the colour gradient, but no significant difference is found between them. Additionally, the mean of the colour gradient is significantly higher than that of the light brightness ( 𝑀𝐷 = 1.256, 𝑝 < 0.001, 𝑑 = 0.756 ), flashing light ( 𝑀𝐷 = 2.128, 𝑝 <0.001, 𝑑 = 1.246 ), and no light condition (𝑀𝐷 = 4.051, 𝑝 < 0.001, 𝑑 = 2.046 ). The third * "MD" represents the absolute mean difference in the post-hoc pairwise comparison. The p-value indicates the level of marginal significance (with the Bonferroni correction adjusting the significance level to 𝛼 = 0.005). Cohen's d is an effect size used to measure the difference between the means of two groups in the pairwise comparisons. Effect sizes of 0.2, 0.5, and 0.8 are considered small, medium, and large, respectively [81]. 0 1 2 3 4 5 6 7No LightFlashing LightLoading Progress BarColour GradientLight BrightnessSuitability ScoreMean of Suitability Score by LED Cue Patterns46 preferred cue pattern is the light brightness, which is significantly more suitable than the flashing light ( 𝑀𝐷 = 0.872, 𝑝 < 0.001, 𝑑 = 0.530 ) and the no light condition ( 𝑀𝐷 = 2.795, 𝑝 <0.001, 𝑑 = 1.433). The least suitable pattern is the flashing light, which is a significantly more suitable pattern than the no light condition (𝑀𝐷 = 1.923, 𝑝 < 0.001, 𝑑 = 0.924). 4.1.2 H1.2: All LED cues in the semi-automatic mode are equally suitable to those in the automatic mode. The ANOVA tests also reveal a significant main effect of the operating mode on the suitability score ( 𝑝 = 0.001, 𝜂𝑝2 = 0.236 ). Pairwise comparisons were conducted to determine the significant differences between these modes. The means of suitability scores with respect to the operating mode were plotted in Figure 4.2. The results show that the automatic mode is significantly more suitable than the semi-automatic mode (𝑀𝐷 = 0.359, 𝑝 = 0.009, 𝑑 = 0.302). Therefore, Hypothesis H1.2 can be rejected when comparing the means of the suitability scores based on the operating modes. Despite the result showing a statistically significant difference between these two modes, this difference is relatively small, as indicated by the suitability scores. 47 Figure 4.2 Pairwise comparisons for the suitability score based on the operating mode Although a non-significant interaction effect between mode and pattern is found in the ANOVA tests, it is still worthwhile to investigate the Hypothesis H1.2 by comparing the suitability means for each individual LED cue with respect to the mode and determining if there are any significant differences. Figure 4.3 illustrates the mean suitability scores for each LED cue under both the semi-automatic and automatic modes. As ANOVA tests reveal significant differences among the cue modes, it is important to note that not every LED cue pattern exhibited such significant differences. The pairwise comparisons show that the colour gradient in the automatic mode has a significantly higher suitability mean compared to the semi-automatic mode (𝑀𝐷 = 0.410, 𝑝 =0.024, 𝑑 = 0.375 ). A significant difference between the modes is also found in the no light condition (𝑀𝐷 = 0.615, 𝑝 = 0.004, 𝑑 = 0.493). The other cue patterns do not show significant differences between these modes. 3.9 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9Semi-automatic ModeAutomatic ModeSuitability ScoreMean of Suitability Score by Operating Mode48 Figure 4.3 Pairwise comparisons for the suitability score of the LED cue patterns under the semi-automatic and automatic modes 4.1.3 Preference for Robot LED Cues Based on the abovementioned ANOVA tests on the suitability Likert responses to the LED cue pattern and mode, the results provide answers to the research questions, “Which LED cues are more suitable for indicating the robot's force compared to others?” and “Is the cue pattern more suitable with LED than without?” The results in H1.1 reveal a significant difference between the LED cue patterns. The colour gradient and the loading progress bar are more suitable for indicating the gripper force information. Moreover, the results in H1.2 reveal that the colour gradient is significantly more suitable in the automatic mode than in the semi-automatic mode. The next 49 preferred LED cue is the light brightness, followed by the flashing light. All the designed LED cues are found to be significantly more suitable than the no-light condition. 4.2 Interpretability Scales This section presents the results of ANOVA tests on the interpretability Likert responses to the LED cues in Question 2.1, 2.2 and 2.3. These questions were designed by using a 7-point Likert scale to assess the general interpretability of the LED cues in terms of legibility, intuitiveness and interactivity. I conducted ANOVA tests to detect the main effects of the LED cue patterns, operating modes, and their interactions. Table 4.2 provides a summary of these statistical results. The Bonferroni post-hoc comparisons (𝛼 = 0.005 ) were then performed to determine the significant differences between each combination of factor levels. The results described in the following sub-sections contribute to answering the research question, "Do participants comprehend the message conveyed by the robot?" Table 4.2 2x5 Repeated Measures ANOVA on interpretability scales Interpretability factor Effect Effect DOF Error DOF F-statistic p-value Effect size 𝜼𝒑𝟐 Legibility Mode 1 38 2.618 0.114 0.064 Pattern 4 152 163.279 <0.001 0.811 Mode * Pattern 4 152 0.729 0.513 0.021 50 Intuitiveness Mode 1 38 1.776 0.191 0.045 Pattern 4 152 133.303 <0.001 0.778 Mode * Pattern 4 152 0.729 0.573 0.019 Interactivity Mode 1 38 10.371 0.003 0.214 Pattern 4 152 110.642 <0.001 0.744 Mode * Pattern 4 152 1.102 0.358 0.028 4.2.1 Legibility Scale The legibility Likert scale in Question 2.1 was designed to measure how easily the messages in robot LED cues can be read and understood. The results of the ANOVA reveal a significant main effect in the LED cue pattern (𝐹(4, 152) = 163.279, 𝑝 < 0.001, 𝜂𝑝2 = 0.811 ). However, no significant results are found for the mode and the interaction between mode and pattern. The analysis focuses on the post-hoc tests that compare the differences between the LED cue patterns. The results of pairwise comparisons on the legibility scale are illustrated in Figure 4.4. The colour gradient and loading progress bar are the two most easily readable and understood LED cue patterns. Their means of legibility scores are significantly higher than the other patterns. The mean score of the colour gradient is slightly higher than that of the loading progress bar, but no significant difference is found between them. The light brightness is the third highest scoring pattern. It has a significantly higher mean compared to the flashing light (𝑀𝐷 = 0.936, 𝑝 <51 0.001, 𝑑 = 0.656) and the no light condition (𝑀𝐷 = 3.128, 𝑝 < 0.001, 𝑑 = 1.977). The flashing light shows the lowest mean score, which is below the median score of 4 on the legibility scale. Results show that all the designed LED cues are significantly easier to be read and understood than the no light condition. Figure 4.4 The mean legibility score based on the LED cue pattern 4.2.2 Intuitiveness Scale The intuitiveness Likert scale in Question 2.2 evaluates how clear the LED cue is in communicating the gripper force information. As the results of ANOVA tests summarized in Table 4.2, a significant main effect is detected in the cue pattern ( 𝐹(4, 152) = 133.303, 𝑝 <0.001, 𝜂𝑝2 = 0.778 ). However, there were no significant results at 𝛼 = 0.005 between the modes or in the interaction of mode and pattern. Thus, pairwise comparisons of the intuitiveness 0 1 2 3 4 5 6 7No LightFlashing LightLoading Progress BarColour GradientLight BrightnessLegibility ScoreMean of Legibility Score by LED Cue Patterns52 scale in this section focuses on the cue pattern only. As depicted in Figure 4.5, the pairwise comparisons reveal the mean of intuitiveness score for each pattern. The colour gradient received a significantly higher mean in intuitiveness compared to the light brightness (𝑀𝐷 = 1.782, 𝑝 < 0.001, 𝑑 = 1.108), flashing light (𝑀𝐷 = 2.577, 𝑝 <0.001, 𝑑 = 1.488), and no light condition (𝑀𝐷 = 4.577, 𝑝 < 0.001, 𝑑 = 2.876). Similarly, the loading progress bar also has a significantly higher score than the light brightness (𝑀𝐷 =1.577, 𝑝 < 0.001, 𝑑 = 1.046), flashing light (𝑀𝐷 = 2.372, 𝑝 < 0.001, 𝑑 = 1.321), and no light condition (𝑀𝐷 = 4.372, 𝑝 < 0.001, 𝑑 = 2.621). Although the mean score for the colour gradient is slightly higher than that of the loading progress bar, no significant difference is observed between these patterns. The light brightness is found to be significantly clearer than the flashing light (𝑀𝐷 = 0.795, 𝑝 < 0.001, 𝑑 = 0.557) and no light condition (𝑀𝐷 = 2.795, 𝑝 < 0.001, 𝑑 =1.587). Furthermore, the flashing light is significantly clearer than the no light condition (𝑀𝐷 =2.000, 𝑝 < 0.001, 𝑑 = 1.041). Both the flashing light and the no light condition have means lower than the median score of 4. All the designed LED cues exhibit significantly higher intuitiveness scores than the no light condition. 53 Figure 4.5 The mean intuitiveness score based on the LED cue pattern 4.2.3 Interactivity Scale The interactivity scale measures the level of confidence when intervening in the robot’s pleat manipulation process. ANOVA tests were conducted to determine the significant results for each combination of factor levels based on the Likert responses from Question 2.3. The results in Table 4.2 reveal that cue mode (𝐹(1, 38) = 10.371, 𝑝 = 0.003, 𝜂𝑝2 = 0.214) and pattern (𝐹(4, 152) = 110.642, 𝑝 < 0.001, 𝜂𝑝2 = 0.744) have a statistically significant main effect on the interactivity scale. However, the interaction effect between mode and pattern is not found to be significant. Post-hoc tests were conducted to compare the differences in modes and patterns. 0 1 2 3 4 5 6 7No LightFlashing LightLoading Progress BarColour GradientLight BrightnessIntuitiveness ScoreMean of Intuitiveness Score by LED Cue Patterns54 For the pairwise comparisons between the modes, Figure 4.6 depicts the interactivity scores in both semi-automatic and automatic modes. The results show that operating the robot in the automatic mode significantly increases the levels of confidence compared to that in the semi-automatic mode (𝑀𝐷 = 0.359, 𝑝 = 0.003, 𝑑 = 0.261 ). Both modes show a mean interactivity score above the median score of 4. Figure 4.6 Pairwise comparisons for the interactivity score based on the operating mode Pairwise comparisons were also conducted to analyze the differences between each pattern. As depicted in Figure 4.7, the Likert responses on the interactivity scale display a similar distribution to the legibility and intuitiveness scales. During the human-robot interaction task, both the colour gradient and the loading progress bar exhibit significantly higher levels of confidence compared to the other patterns. Moreover, the light brightness shows a significantly higher mean score than the flashing light (𝑀𝐷 = 0.846, 𝑝 = 0.002, 𝑑 = 0.510 ) and the no light condition (𝑀𝐷 =2.756, 𝑝 < 0.001, 𝑑 = 1.535). The mean score of the flashing light is also significantly higher 3.7 3.8 3.9 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7Semi-automatic ModeAutomatic ModeInteractivity ScoreMean of Interactivity Score by Operating Mode55 than the no light condition (𝑀𝐷 = 1.910, 𝑝 < 0.001, 𝑑 = 1.068), but both are below the median score of 4. Overall, the designed patterns demonstrate significantly higher levels of confidence compared to the no light condition. Figure 4.7 The mean interactivity score based on the LED cue pattern In summary, the analysis of the interpretability scale for the LED cue patterns shows consistent results in terms of legibility, intuitiveness, and interactivity. These statistical findings provide an answer to the research question: "Do participants comprehend the message conveyed by the robot?" The significant main effect of the mode on the interactivity scale shows that operating the robot in the automatic mode generally provides increased interactivity compared to the semi-automatic mode. Regarding the cue patterns, their attributes on the interpretability scales are compared in Figure 4.8. The results indicate the outperformance of the colour gradient and loading 0 1 2 3 4 5 6 7No LightFlashing LightLoading Progress BarColour GradientLight BrightnessInteractivity ScoreMean of Interactivity Score by LED Cue Patterns56 progress bar patterns in terms of legibility, intuitiveness, and interactivity. The subsequent patterns are light brightness and flashing light. The overall score of the flashing light is below the median at 4. The no light condition received the lowest overall score, suggesting that the designed LED cue patterns can improve the robot’s interpretability during human-robot communication. Figure 4.8 Attributes of LED cue patterns on interpretability scales 4.3 NASA TLX (Task Load Index) Scales This section presents the ANOVA results based on participant-reported subjective data obtained from the NASA TLX questionnaire in the Question 3.1, 3.2 and 3.3. A part of the standard NASA TLX questions was used to primarily measure the level of workload experienced by participants while performing tasks, including mental demands, effort demand, and frustration. The ANOVA tests were performed to detect the main effects of these scales with respect to the cue mode, pattern, and their interactions. Results are shown in Table 4.3. When a significant effect is observed, 57 Bonferroni post-hoc comparisons are conducted to determine the differences between each combination of within-subject factors. The findings are then used to answer the research question, "Do the LED cues affect participants' workload during the human-robot collaboration task?" Table 4.3 2x5 Repeated Measures ANOVA on workload demand scales Workload demand factor Effect Effect DOF Error DOF F-statistic p-value Effect size 𝜼𝒑𝟐 Mental Demand Mode 1 38 34.753 <0.001 0.478 Pattern 4 152 51.928 <0.001 0.577 Mode * Pattern 4 152 1.575 0.184 0.040 Effort Demand Mode 1 38 24.079 <0.001 0.388 Pattern 4 152 45.163 <0.001 0.543 Mode * Pattern 4 152 4.009 0.004 0.095 Frustration Mode 1 38 26.691 <0.001 0.413 Pattern 4 152 30.364 <0.001 0.444 Mode * Pattern 4 152 1.884 0.116 0.047 4.3.1 Mental Demand Scale The mental demand scale in Question 3.1 assesses the cognitive workload and mental effort required to perform the task. The ANOVA results in Table 4.3 indicate a significant main effect for both mode (𝐹(1, 38) = 34.753, 𝑝 < 0.001, 𝜂𝑝2 = 0.478) and pattern (𝐹(4, 152) = 51.928, 𝑝 <58 0.001, 𝜂𝑝2 = 0.577). Since no interaction effect is detected from the ANOVA tests, the post-hoc pairwise comparison for the mental demand scale focuses on identifying significant results in mode and pattern. In the pairwise comparisons based on the operating mode, the results indicate that the automatic mode requires significantly less mental load than the semi-automatic mode (𝑀𝐷 = 0.882, 𝑝 <0.001, 𝑑 = 0.610). Figure 4.9 shows the comparison results, indicating that both semi-automatic and automatic modes have mean mental demand scores below the median score. Figure 4.9 Pairwise comparisons for the mental demand score based on the operating mode The ANOVA tests also detected a significant main effect of cue pattern on the mental demand scale (𝑝 < 0.001, 𝜂𝑝2 = 0.577). The distribution of the mean score for the pattern is shown in Figure 4.10. Pairwise comparisons reveal that the colour gradient and the loading progress bar are significantly less mentally demanding patterns compared to the others. The mental load required 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0Semi-automatic ModeAutomatic ModeMental Demand ScoreMean of Mental Demand Score by Operating Mode59 for the light brightness is relatively lower than that of the flashing light. However, the difference between these patterns is non-significant. Overall, the designed LED cues show a significantly lower level of mental demand than the no light condition. The no light condition is the only pattern with a mean score above the median of 4. Figure 4.10 The mean mental demand score based on the LED cue pattern 4.3.2 Effort Demand Scale As part of the workload assessment, the Likert responses to the task difficulty level associated with the perceived effort in Question 3.2 were analyzed by ANOVA tests. This scale was designed to measure the amount of physical and mental exertion required to complete the task. The results of ANOVA tests show significant main effects for both mode (𝐹(1, 38) = 24.079, 𝑝 < 0.001, 𝜂𝑝2 =0.388 ) and pattern (𝐹(4, 152) = 34.163, 𝑝 < 0.001, 𝜂𝑝2 = 0.543 ). Additionally, a significant 0 1 2 3 4 5 6 7No LightFlashing LightLoading Progress BarColour GradientLight BrightnessMental Demand ScoreMean of Mental Demand Score by LED Cue Patterns60 interaction effect between mode and pattern is found (𝐹(4, 152) = 4.009, 𝑝 = 0.004, 𝜂𝑝2 =0.095 ). Therefore, I conducted pairwise comparisons to determine the significant differences between these variables. For the pairwise comparison between the operating modes, Figure 4.11 shows the distribution of their effort demand score means. The results reveal that the automatic mode generally requires significantly less effort demand than the semi-automatic mode (𝑀𝐷 = 0.995, 𝑝 < 0.001, 𝑑 =0.594). The mean score of effort demand is lower than the median of 4 for both modes. Figure 4.11 Pairwise comparisons for the effort demand score based on the operating mode The mean of effort demand score for each pattern is plotted in Figure 4.12. For the comparisons between the LED cue patterns, the results show that operating the robot with the colour gradient and the loading progress bar demands a significantly lower level of effort compared to other patterns. The mean score of the light brightness is relatively lower than the flashing light, but the result is non-significant. All the designed patterns have an effort demand score lower than the 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0Semi-automatic ModeAutomatic ModeEffort Demand ScoreMean of Effort Demand Score by Operating Mode61 median at 4 and require significantly less effort compared to the no light condition. Figure 4.12 Pairwise comparisons for the effort demand score based on the LED cue patterns As the ANOVA tests also reveal a significant interaction effect with a medium effect size (𝑝 =0.004, 𝜂𝑝2 = 0.095 ), pairwise comparisons were conducted for each pattern under different modes. The results in Figure 4.13 indicate that all patterns, including light brightness (𝑀𝐷 =0.872, 𝑝 = 0.001, 𝑑 = 0.553 ), colour gradient (𝑀𝐷 = 0.564, 𝑝 = 0.049, 𝑑 = 0.326 ), loading progress bar (𝑀𝐷 = 0.769, 𝑝 = 0.002, 𝑑 = 0.541), flashing light (𝑀𝐷 = 1.513, 𝑝 < 0.001, 𝑑 =0.859), and the no light condition (𝑀𝐷 = 1.256, 𝑝 < 0.001, 𝑑 = 0.709), require a significantly reduced level of effort in the automatic mode compared to the semi-automatic mode. Additionally, the flashing light pattern exhibits a relatively larger reduction in effort demand compared to other patterns when operating in the automatic mode. Therefore, the results of the interaction effect in the effort demand show that the effect of the operating mode depends on the level of the pattern. 0 1 2 3 4 5 6 7No LightFlashing LightLoading Progress BarColour GradientLight BrightnessEffort Demand ScoreMean of Effort Demand Score by LED Cue Patterns62 Figure 4.13 Pairwise comparisons for the effort demand score of the LED cue patterns under the semi-automatic and automatic modes 4.3.3 Frustration Scale The frustration scale was designed to assess the level of mental discomfort and negative emotional responses associated with the task. This helps in understanding the extent to which participants may have felt stressed, overwhelmed, or dissatisfied regarding the designed patterns and modes. The results of the ANOVA tests in Table 4.3 indicate a significant main effect for both mode (𝐹(1, 38) = 26.691, 𝑝 < 0.001, 𝜂𝑝2 = 0.413) and pattern (𝐹(4, 152) = 30.364, 𝑝 <0.001, 𝜂𝑝2 = 0.444 ). However, their interaction effect is found to be non-significant. In this section, the results of pairwise comparisons were analyzed based on mode and pattern only. 63 For the pairwise comparisons of the modes, the results of frustration mean scores in Figure 4.14 show a consistent trend to the mental demand and effort demand. Operating the robot in the automatic mode is significantly less frustrating than in the semi-automatic mode ( 𝑀𝐷 =0.774, 𝑝 < 0.001, 𝑑 = 0.559 ). The means of frustration scores for both modes are below the median on the frustration scale. Figure 4.14 Pairwise comparisons for the frustration score based on the operating mode Figure 4.15 displays the means of Likert responses for each pattern on the frustration scale. The results of pairwise comparisons show that the colour gradient and loading progress bar patterns have a significantly lower level of frustration compared to the other patterns. The light brightness is significantly less frustrating than the no light condition (𝑀𝐷 = 1.282, 𝑝 < 0.001, 𝑑 = 0.650). Meanwhile, the flashing light has a relatively lower mean than the no light condition, but no significant difference is detected between them. All designed patterns, except the flashing light, are significantly less frustrating than the no light condition, and they have a mean score lower than the median at 4. 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5Semi-automatic ModeAutomatic ModeFrustration ScoreMean of Frustration Score by Operating Mode64 Figure 4.15 Pairwise comparisons for the frustration score based on the LED cue patterns In summary, the NASA TLX questionnaire was used to assess the workload of interacting with the robot during the task. The analysis of the mental demand, effort, and frustration scales reveals main effects in the mode and pattern. Additionally, an interaction effect between mode and pattern is found in the effort scale. These results help evaluate the psychological aspect of the task load and provide insights to answer the research question, "Do the LED cues affect participants' workload during the human-robot collaboration task?" Figure 4.16 and Figure 4.17 summarize the performance of the modes and cue patterns based on the NASA TLX scales. Generally, the automatic mode reduces the overall workload of the task compared to the semi-automatic mode. The loading progress bar and the colour gradient are the two least workload demanding patterns, which have significantly reduced levels of mental demand, effort, and frustration compared to other patterns. All LED cues outperform the no light condition, effectively alleviating the overall workload demand. Meanwhile, the interaction effect in the effort scale shows that each pattern has 0 1 2 3 4 5 6 7No LightFlashing LightLoading Progress BarColour GradientLight BrightnessFrustration ScoreMean of Frustration Score by LED Cue Patterns65 significantly reduced effort demand in the automatic mode than in the semi-automatic mode. However, the level of effort demand depends on the combination of mode and pattern. Therefore, operating the robot with different patterns and modes indeed impacts the extent of participants' workload in terms of mental demand, effort demand, and frustration. Figure 4.16 Attributes of operating modes on NASA TLX scales Figure 4.17 Attributes of LED cue patterns on NASA TLX scales 66 Chapter 5: Discussion This chapter presents key interpretations of the findings derived from the statistical analysis of the results in the previous chapter, as well as the limitations and future work. The study evaluated four LED cue patterns plus a light-off condition and compared them across two operating modes (automatic and semi-automatic). These cue patterns had not previously been directly compared in this context. Sections 5.1, 5.2, and 5.3 discuss the results in terms of suitability, interpretability and workload demand. Section 5.4 discusses the correlations and differences between this study and previous literature. Limitations and recommended future steps are discussed in Section 5.5 and 5.6 respectively. 5.1 Suitability of Robot LED Cues The suitability scale evaluated the overall performance of each pattern based on participants' subjective impressions. The ANOVA test showed significant differences among the LED cue patterns. Therefore, Hypothesis H1.1: all LED cues are equally suitable for indicating the robot’s information, can be rejected. Our findings revealed that the colour gradient and loading progress bar patterns are significantly more suitable for indicating the gripper force compared to the other patterns (light brightness, flashing light). The outperformance of these patterns is attributed to their easily readable and intuitive animations, thus enhancing participants' confidence during interactions with the robot. The significant differences between the light conditions and the light-off condition demonstrate the effectiveness of the LED cues in terms of facilitating the robot's 67 interpretability while alleviating the participants' workload in understanding the robot's status. Additionally, operating the robot in different modes can impact participants' performance in terms of confidence levels and workload demands. The automatic mode provided slightly higher suitability compared to the semi-automatic mode. Therefore, comparing the suitability scores with respect to the modes leads us to reject Hypothesis H1.2: all LED cues in the semi-automatic mode are equally suitable to those in the automatic mode. However, the two-way interaction effect analysis did not indicate significant differences in every pairwise comparison. The results showed that the improvement by using the automatic mode is statistically significant in the colour gradient and no light conditions. In terms of the scale score, it is important to note that the mean differences between the two modes are not substantial in each pattern. Responses to the suitability scale showed that participants perceive these two modes as relatively similar. 5.2 Interpretability of Robot LED Cues To investigate the reasons that led participants to their responses to the suitability scale, I evaluated the performance of each LED cue in the interpretability scale. The results showed participants preferred the colour gradient and loading progress bar because these patterns were significantly more comprehensible in terms of legibility, intuitiveness, and interactivity than the light brightness and flashing light patterns. Participants' preference on the interpretability scales explains their feedback indicated on the suitability scale. The LED with a colour scheme similar to traffic lights 68 was reported to be clear and intuitive for visually supplying force feedback, which enhances the precision of grip force control [50]. Furthermore, robot expressive LEDs with the loading progress bar was proven to be an effective pattern that significantly helps people understand the robot’s state and actions, thereby building a more trustworthy relationship between humans and robots [62]. The next preferred pattern was light brightness, which outperformed the flashing light in the context of this study. The overall LED cues exhibited significantly higher interpretability compared to the no-light condition. This suggests that the designed visual feedback is crucial for empowering non-anthropomorphic robots to effectively convey information during HRI [49]. In the comparison of robot operating modes, the main effect indicates that the automatic mode increases participants' confidence levels when interacting with the robot. This suggests that the programmed automatic mode allows for a higher level of robot autonomy, thereby enhancing the safety and effectiveness of human-robot collaboration. 5.3 Workload Demand of Robot LED Cues The workload perceived by participants while performing collaborative tasks with the robot was evaluated using NASA TLX. The main focus was on the cognitive workload, specifically related to mental demand, effort demand, and frustration level. The main effects revealed the impact of patterns and modes on the overall cognitive workload experienced by participants. The results demonstrated that the colour gradient and loading progress had significantly lower workload 69 demands compared to the other patterns. These patterns were effective in communicating the intended messages and signaling robots’ awareness and intent, resulted in higher levels of psychological safety. This further clarifies the findings of the interpretability scale. The outperformance of the colour gradient and loading progress bar patterns in terms of legibility, intuitiveness, and interactivity allows participants to easily and clearly receive the messages conveyed by the robot. As a result, this reduces participants' cognitive demand and effort required to comprehend the robot's behaviours, as well as preventing any potential frustration caused by miscommunication with the robot. Additionally, participants preferred to operate the robot in the automatic mode as it reduced their mental workload, effort, and frustration. Regarding the interaction effect on the effort scale, the automatic mode diminished the effort differences between the patterns. This improvement was more significant in the flashing light, which had high effort demands in the semi-automatic mode. Moreover, when participants operated the robot with no light condition, it consistently required a higher workload, mainly due to the lack of visual feedback to maintain the gripper force within an appropriate range [50]. In summary, participants showed a preference for the colour gradient and loading progress bar patterns when operating the robot in collaborative tasks. These patterns were deemed more suitable for indicating the gripper force, as their animations enhanced the interpretability of the robot's 70 behaviour and alleviated the perceived workload of the participants. Moreover, the majority of responses in the follow-up Question 4.1 highlighted the colour gradient, in particular, as having better contrast in representing different levels of force compared to other patterns. Participants also specified that light brightness and flashing light patterns required relatively more cognitive workload for recognizing the indication of force levels. The results in Chapter 4 demonstrate that the automatic mode enhances the performance of the LED cues in comparison to their performance in the semi-automatic mode. The automatic mode elevates participants' confidence levels during human-robot interaction tasks, specifically in pleat manipulation. Furthermore, the automatic mode was proved to be more supportive than the semi-automatic mode, leading to a significant reduction in participants' mental demand, effort demand, and frustration. Lastly, the feedback from Question 4.2 indicated that all participants were able to appropriately distinguish each LED cue pattern. These patterns outperformed the no light condition in all aspects. Therefore, participants prefer to have visual feedback to better understand the robot's behaviour. 5.4 Comparative Analysis of Robot LED Cues The abovementioned results in Section 5.1, 5.2, and 5.3 align with several findings of the reviewed literature, supporting and reinforcing existing HRI research. Notably, this study substantiated the 71 advantages of employing LED cues for the VIMEE robot conveying force feedback information. The use of LED patterns, such as the colour gradient and loading progress bar, enhances the interpretability of the robot's behaviours while mitigating participants' perceived workload. This corroborates previously reviewed literature that emphasizes the importance of clear and easily interpretable visual cues with coloured LEDs in facilitating participants' situational awareness and promoting mutual understanding between humans and robots [51]. On the other hand, the use of coloured LED cues to convey affective expressions for a quadruped robot [49] and to provide force feedback for a prosthetic hand [50] yields positive results in improving effective communication of the robot's intent and state, as well as promoting psychological safety for individuals interacting with them. In my study, the LED cues with colours and the loading progress bar patterns for grip force feedback indication were equally preferred within the context of the CFRP vacuum bagging process. These patterns exhibited a high level of legibility, intuitiveness, and interactivity, resulting in reduced mental and effort demand, as well as less frustration for participants. Furthermore, Baraka et al.’s research showed these patterns played a significant role in helping individuals understand the robot's behaviour when there were limited contextual cues available [62]. In the HRI scenario, a robot with visual feedback generally enhances communication, providing a higher level of interpretability, effectiveness, and safety compared to a robot without visual feedback [51]. The consistency of these results with existing research supports the notion that well-designed LED cues can improve individuals’ sense of psychological safety, enhance their situational awareness while engaging in HRI, and establish transparency of the robot's behavioural status. 72 While the previously discussed outcomes align with some reviewed literature trends, there are a few differences observed in the LED cues, including the flashing light and light brightness. The use of these patterns for a quadruped robot's affective expressions [49] and a prosthetic hand's force feedback [50] demonstrated their effectiveness in signaling the robot's awareness and conveying intended information. However, the performance of these patterns was less satisfactory in the context of this study. Several factors may have contributed to this divergence in results, including variations in the operational interface, research platform, and task complexity. In Sanoubari et al.’s research, the evaluation was conducted using an animated robot, and participants did not need to operate the robot during the interaction [49]. This setup may have allowed participants to pay more attention to observing and interpreting the message in the flashing LED. Whereas, in my study, participants were required to operate the robot gripper and interact with it while observing the visual feedback. This process could potentially elevate the perceived workload for participants, resulting in a distinct HRI experience when interpreting the same light cues. Additionally, in Engeberg et al.’s research, participants were tasked with controlling a prosthetic hand using electromyographic preamplifiers attached to their forearms [50]. Despite the LED with changing light brightness for grip force feedback were employed in both our studies, the experience of operating a prosthetic hand through a wearable control device could be fundamentally different from operating a robot gripper using a joystick. Moreover, Engeberg et al. utilized a combination of coloured LEDs with a changing brightness effect to convey the force information [50]. In contrast, I evaluated the effects of the colour gradient and light brightness 73 patterns individually. This approach could yield distinct results for these patterns in this study. 5.5 Limitations This section outlines some limitations of this research. 5.5.1 Limitations in Participant Recruitment Our study participants were mainly recruited from the university setting, which may lead to experimenting with a less diverse sample. University students and researchers typically lack an industrial background, especially in the field of CFRP manufacturing. Therefore, having a relatively small and homogenous sample of participants could potentially limit the generalizability of the results to the relevant populations (e.g., technicians at DLR's facilities). To reduce the influence of this limitation, I attempted to advertise the study recruitment widely to mechanical engineering students and researchers who potentially have relatively more experience in robotics, automation, or manufacturing compared to other disciplines. While this study had a relatively small sample size, the number of recruited participants was sufficient to ensure statistically valid results are obtained from hypothesis tests. The required minimum sample size was determined by the power analysis presented in Section 3.5. 5.5.2 Limitations in Experimental Setup In this study, a robot gripper equipped with LEDs was prototyped for collaboration with the participants. A simplified version of the testbed with a nylon membrane was used to assess the 74 gripper's performance in the pleating procedure of a simulated CFRP manufacturing vacuum bagging process. However, it is important to acknowledge that real-world workpieces can be much larger and more complex than the simulated testbed used in the laboratory. The gripper's behaviour may vary when encountering different sizes of materials. This experiment aimed to replicate the setup as closely as possible to the industrial setting in DLR's facilities. The results demonstrated the effectiveness of the VIMEE gripper as a proof-of-concept. Future tests can focus on evaluating the performance with a large-scale setup. Another limitation, as mentioned by some participants in the follow-up Questions 4.2 and 4.3, is regarding the latency of the LEDs when there is a quick force change. This latency is mainly attributed to the limited computational capability of the microcontroller on the VIMEE gripper. 5.6 Future Work The current study provides valuable insights into the design and evaluation of visual cues for human-robot collaboration tasks. In addition, participant responses to Question 4.3 suggested improvements that can be expanded upon for future research, such as having an additional robot cue to indicate the occurrence of slippage or jamming conditions, improving the delay from LEDs, and using softer hues instead of blue to avoid discomfort for participants' eyes. While the designed LED cues show promising results, further exploration of other novel cue patterns for gripper force indication, such as combining the colour gradient and loading progress 75 bar, could be conducted to enrich the understanding of their effects on user performance and perception. Additional improvements could also focus on implementing auxiliary robot cues for other pleating functionalities. To provide for future studies, this thesis provides a comprehensive description of the gripper design in Section 3.1, which includes a description of modular sensors and actuators that can be easily recalibrated or upgraded. In order to improve the robot’s response time, the gripper can be upgraded by replacing the current microcontroller with a more powerful processor. The wiring of the gripper for power supply and Arduino communication can be optimized to enhance the robot's mobility and improve its compatibility for more complex trajectory planning in future applications. Moreover, the pleat manipulation tests can be continued to explore the use of alternate high-friction roller materials to better prevent slippage of the pleat and enhance the overall performance of the gripper. The primary objective of this study, which has been successfully achieved, is to prototype the VIMEE gripper with a visual feedback system as a proof-of-concept and to investigate the impact of the designed LED cue patterns and modes on human-robot collaboration tasks. 76 Chapter 6: Conclusion This thesis presents a comprehensive investigation into the design and evaluation of visual cues that provide force feedback for a robot gripper in a simulated CFRP vacuum bagging process. In this study, I explored the acceptance and appropriateness of LED-based expressive lights, which are one of the most important modalities in human-robot non-verbal communication. The study was guided by literature review and a pilot survey, which led to the formulation of specific research questions and the experimental design. The results validated the effectiveness and generalizability of the four designed novel visual cue patterns: light brightness, colour gradient, light progress bar, and flashing light. Each pattern had specific programmed light animations on an LED strip to indicate the force information while the VIMEE gripper performed pleat manipulation tasks, including loading and migration of pleats. A total of 39 participants were recruited to conduct the experiments and characterize these LED cues in terms of suitability, interpretability, and workload demand. A questionnaire with 7-point Likert scales was employed to assess the performance of the various cues under both semi-automatic and automatic modes. The results were analyzed using 2x5 repeated measures ANOVA tests, comparing the differences between the developed novel cues. The suitability Likert scale was used to assess the effectiveness and appropriateness of the designed LED cues for communicating the gripper force information during human-robot collaborative tasks. The ANOVA results revealed that operating the robot with different cue patterns and modes led to statistically significant differences in participants’ performance. Therefore, both hypotheses 77 H1.1 and H1.2 can be rejected. The automatic mode was found to be more suitable than the semi-automatic mode. The colour gradient and loading progress bar were the most suitable cue patterns. The next preferred pattern is light brightness, followed by the flashing light. Overall, participants determined that the LED cue patterns were significantly more suitable than the no light condition. These findings highlight the significance of visual feedback in enhancing the participants' understanding of the robot's behaviour. Based on the results of the responses to the interpretability scales, I assessed the performance of the LED cues in terms of legibility, intuitiveness, and interactivity, aiming to answer the question, "Do participants comprehend the message conveyed by the robot?" The results exhibited a similar trend as the suitability scale results. Specifically, the colour gradient and loading progress bar patterns were found to be significantly more comprehensible and easier to read compared to the other patterns. Additionally, these patterns provided participants with a significantly increased level of confidence during their interactions with the robot. On the other hand, the light brightness and flashing light patterns were generally perceived as less interpretable across these scales. All LED cues significantly outperformed the no light condition. This indicates that participants were able to comprehend the information communicated by the robot through visual feedback during the tasks. The overall design of expressive LEDs can help disambiguate the robot’s behaviours and improve interpretability in HRI. 78 The results of the NASA TLX scales provide insights into the question, "Do the LED cues impact participants' workload during the human-robot collaboration task?" The colour gradient and loading progress bar patterns were found to significantly reduce participants' workload in terms of mental demand, effort, and frustration. However, the light brightness and flashing light patterns were perceived as relatively more demanding in terms of workload. The no light condition consistently showed the highest workload demand, as it lacked visual feedback for participants during their interactions with the robot. Overall, the results of the interpretability and workload scales support the trend observed in the suitability scale. Both the colour gradient and loading progress bar were found to be the most effective visual feedback patterns for communicating the gripper force information. These patterns exhibited significantly higher legibility, intuitiveness, and safety, resulting in lower levels of mental demand, effort demand, and frustration compared to the other patterns. Additionally, the automatic mode was preferred over the semi-automatic mode, as it improved the overall performance for the cue patterns and significantly reduced participants' perceived workload. The fact that all the LED cues outperformed the no light conditions further demonstrates their critical and beneficial role for participants when interacting with the robot in a simulated CFRP manufacturing vacuum bagging process. The results showed that implementing visual cues on an industrial robot significantly helped people in establishing a safe and trustworthy relationship with a robot agent. The mutual understanding in the human-robot interaction scenario was achieved by 79 enhancing the interpretability and transparency of the robot’s capabilities and limitations. 80 Bibliography [1] Chaudhari, N., & Shinde, V. B. (2014). Recent Trends in Robotic End of Arm Tooling (gripper). International Journal of Mechanical Engineering and Robotics Research, 3(2), 171. [2] Reddy, P. V. P., & Suresh, V. V. N. S. (2013). A review on importance of universal gripper in industrial robot applications. Int. J. Mech. Eng. Robot. Res, 2(2), 255-264. [3] Bemfica, J. R., Melchiorri, C., Moriello, L., Palli, G., & Scarcia, U. (2014, May). A three-fingered cable-driven gripper for underwater applications. In 2014 IEEE International Conference on Robotics and Automation (ICRA) (pp. 2469-2474). [4] Stephan, M., Rognini, G., Sengul, A., Beira, R., Santos-Carreras, L., & Bleuler, H. (2010, October). Modeling and design of a gripper for a robotic surgical system integrating force sensing capabilities in 4 DOF. In ICCAS 2010 (pp. 361-365). IEEE. [5] Alqasemi, R., Mahler, S., & Dubey, R. (2007, June). Design and construction of a robotic gripper for activities of daily living for people with disabilities. In 2007 IEEE 10th International Conference on Rehabilitation Robotics (pp. 432-437). [6] Samadikhoshkho, Z., Zareinia, K., & Janabi-Sharifi, F. (2019, May). A brief review on robotic grippers classifications. In 2019 IEEE Canadian Conference of Electrical and Computer Engineering (CCECE) (pp. 1-4). [7] Lasi, H., Fettke, P., Kemper, H. G., Feld, T., & Hoffmann, M. (2014). Industry 4.0. Business & Information Systems Engineering, 6(4), 239-242. [8] Jurkat, A., Klump, R., & Schneider, F. (2022). Tracking the Rise of Robots: The IFR Database. Jahrbücher für Nationalökonomie und Statistik. [9] Hegde, S., Shenoy, B. S., & Chethan, K. N. (2019). Review on carbon fiber reinforced polymer (CFRP) and their mechanical performance. Materials Today: Proceedings, 19, 658-662. [10] Al-Lami, A., Hilmer, P., & Sinapius, M. (2018). Eco-efficiency assessment of manufacturing carbon fiber reinforced polymers (CFRP) in aerospace industry. Aerospace Science and Technology, 79, 669-678. [11] Herbeck, L., Wilmes, H., & Kleineberg, M. (2004). Material and processing technology for CFRP fuselage. SAMPE 2004. [12] Glast, F. (2021). Vacuum Infusion Complete Guide. Dostopno na: https://www. fibreglast. com/product/vacuum-infusion-Guide/Learning_Center, ogled, 26, 18. [13] Chan, W. P., Hanks, G., Sakr, M., Zuo, T., Van der Loos, H. M., & Croft, E. (2020, October). An augmented reality human-robot physical collaboration interface design for shared, large-scale, labour-intensive manufacturing tasks. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 11308-11313). [14] Hegel, F., Gieselmann, S., Peters, A., Holthaus, P., & Wrede, B. (2011, July). Towards a typology of meaningful signals and cues in social robotics. In 2011 RO-MAN (pp. 72-78). 81 IEEE. [15] Peters, A., Spexard, T. P., Hanheide, M., & Weiss, P. (2011). Hey robot, get out of my way-A survey on a spatial and situational movement concept in HRI. In Behaviour Monitoring and Interpretation–BMI (pp. 147-165). IOS Press. [16] Arévalo Arboleda, S., Dierks, T., Rücker, F., & Gerken, J. (2020, March). There's More than Meets the Eye: Enhancing Robot Control through Augmented Visual Cues. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (pp. 104-106). [17] Vidal, G. H., Coronado-Hernández, J. R., & Minnaard, C. (2022). Measuring Manufacturing System Complexity: A Literature Review. Journal of Intelligent Manufacturing, 1-24. [18] Chen, D., Chan, W. P., Quek, J., & Van der Loos, H. M. (2023, May). Mechatronic Design of Two-Jaw Pleating End-Effector for Large-Scale Carbon Fibre Reinforced Polymer Manufacturing. In Materials Science Forum (Vol. 1086, pp. 85-90). Trans Tech Publications Ltd. [19] Stürz, Y. R., Affolter, L. M., & Smith, R. S. (2017). Parameter identification of the KUKA LBR iiwa robot including constraints on physical feasibility. IFAC-PapersOnLine, 50(1), 6863-6868. [20] Smith, J. M., & Harper, D. (2003). Animal signals. Oxford University Press. [21] Marin Vargas, A., Cominelli, L., Dell’Orletta, F., & Scilingo, E. P. (2021). Verbal communication in robotics: a study on salient terms, research fields and trends in the last decades based on a computational linguistic analysis. Frontiers in Computer Science, 63. [22] Perzanowski, D., Schultz, A. C., Adams, W., Marsh, E., & Bugajska, M. (2001). Building a multimodal human-robot interface. IEEE Intelligent Systems, 16(1), 16-21. [23] Mavridis, N. (2015). A review of verbal and non-verbal human–robot interactive communication. Robotics and Autonomous Systems, 63, 22-35. [24] Gnjatović, M., Tasevski, J., Nikolić, M., Mišković, D., Borovac, B., & Delić, V. (2012, September). Adaptive multimodal interaction with industrial robot. In 2012 IEEE 10th Jubilee International Symposium on Intelligent Systems and Informatics (pp. 329-333). [25] Jia, Y., She, L., Cheng, Y., Bao, J., Chai, J. Y., & Xi, N. (2016, August). Program robots manufacturing tasks by natural language instructions. In 2016 IEEE International Conference on Automation Science and Engineering (CASE) (pp. 633-638). [26] Nikolaidis, S., Kwon, M., Forlizzi, J., & Srinivasa, S. (2018). Planning with verbal communication for human-robot collaboration. ACM Transactions on Human-Robot Interaction (THRI), 7(3), 1-21. [27] Kumatani, K., Arakawa, T., Yamamoto, K., McDonough, J., Raj, B., Singh, R., & Tashev, I. (2012, December). Microphone array processing for distant speech recognition: Towards real-world deployment. In Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (pp. 1-10). IEEE. 82 [28] Jensen, B., Tomatis, N., Mayor, L., Drygajlo, A., & Siegwart, R. (2005). Robots meet humans-interaction in public spaces. IEEE Transactions on Industrial Electronics, 52(6), 1530-1546. [29] Kennedy, J., Lemaignan, S., Montassier, C., Lavalade, P., Irfan, B., Papadopoulos, F., ... & Belpaeme, T. (2017, March). Child speech recognition in human-robot interaction: evaluations and recommendations. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (pp. 82-90). [30] Gomez, R., Inoue, K., Nakamura, K., Mizumoto, T., & Nakadai, K. (2014, September). Speech-based human-robot interaction robust to acoustic reflections in real environment. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1367-1373). [31] Hämäläinen, A., Candeias, S., Cho, H., Meinedo, H., Abad, A., Pellegrini, T., ... & Dias, M. S. (2014, September). Correlating ASR errors with developmental changes in speech production: A study of 3-10-year-old European Portuguese children's speech. In Workshop on Child Computer Interaction-WOCCI 2014 (pp. 1-8). [32] Kumar, S. A., & Kumar, C. S. (2016, September). Improving the intelligibility of dysarthric speech towards enhancing the effectiveness of speech therapy. In 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (pp. 1000-1005). IEEE. [33] Beebe, S. A. (1979). Nonverbal Communication in Business: Principles and Applications. [34] Dolcos, S., Sung, K., Argo, J. J., Flor-Henry, S., & Dolcos, F. (2012). The Power of a Handshake: Neural Correlates of Evaluative Judgments in Observed Social Interactions. Journal of Cognitive Neuroscience, 24(12), 2292-2305. [35] Saunderson, S., & Nejat, G. (2019). How robots influence humans: A Survey of Nonverbal Communication in Social Human–Robot Interaction. International Journal of Social Robotics, 11(4), 575-608. [36] Threatt, A. L., Green, K. E., Brooks, J. O., Merino, J., Walker, I. D., & Yanik, P. (2013, July). Design and evaluation of a nonverbal communication platform between assistive robots and their users. In International Conference on Distributed, Ambient, and Pervasive Interactions (pp. 505-513). Springer, Berlin, Heidelberg. [37] Vasconez, J. P., Guevara, L., & Cheein, F. A. (2019, April). Social robot navigation based on HRI non-verbal communication: a case study on avocado harvesting. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (pp. 957-960). [38] Yang, Y., & Williams, A. B. (2021, March). Improving human-robot collaboration efficiency and robustness through non-verbal emotional communication. In Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (pp. 354-356). [39] Krenn, B., Reinboth, T., Gross, S., Busch, C., Mara, M., Meyer, K., ... & Layer-Wagner, T. (2021). It's your turn!--A collaborative human-robot pick-and-place scenario in a virtual industrial setting. arXiv preprint arXiv:2105.13838. 83 [40] Shi, D., Collins Jr, E. G., Goldiez, B., Donate, A., Liu, X., & Dunlap, D. (2008, May). Human-aware robot motion planning with velocity constraints. In 2008 International Symposium on Collaborative Technologies and Systems (pp. 490-497). IEEE. [41] Austin, W. M. (1965). Some Social Aspects of Paralanguage. Canadian Journal of Linguistics/Revue canadienne de linguistique, 11(1), 31-39. [42] Fukuda, H., Shiomi, M., Nakagawa, K., & Ueda, K. (2012, March). 'Midas touch'in human-robot interaction: Evidence from event-related potentials during the ultimatum game. In Proceedings of the 7th Annual ACM/IEEE International Conference on Human-Robot Interaction (pp. 131-132). [43] Bruneau, T. (2012). Chronemics: Time-binding and the construction of personal time. ETC: A Review of General Semantics, 72-92. [44] Moon, A., Parker, C. A., Croft, E. A., & Van der Loos, H. M. (2011, September). Did you see it hesitate?-Empirically grounded design of hesitation trajectories for collaborative robots. In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 1994-1999). [45] Breazeal, C., Kidd, C. D., Thomaz, A. L., Hoffman, G., & Berlin, M. (2005, August). Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. In 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 708-713). [46] Admoni, H., Weng, T., Hayes, B., & Scassellati, B. (2016, March). Robot nonverbal behavior improves task performance in difficult collaborations. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (pp. 51-58). [47] Ekman, P., & Friesen, W. V. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. semiotica, 1(1), 49-98. [48] Urakami, J., & Seaborn, K. (2022). Nonverbal Cues in Human-robot Interaction: A Communication Studies Perspective. ACM Transactions on Human-Robot Interaction. [49] Sanoubari, E., David, B., Kew, C., Cunningham, C., & Caluwaerts, K. (2022, August). From Message to Expression: Exploring Non-Verbal Communication for Appearance-Constrained Robots. In 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) (pp. 1193-1200). [50] Engeberg, E. D., & Meek, S. (2012). Enhanced visual feedback for slip prevention with a prosthetic hand. Prosthetics and Orthotics International, 36(4), 423-429. [51] Szafir, D., Mutlu, B., & Fong, T. (2015, March). Communicating directionality in flying robots. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction (pp. 19-26). [52] Fernandez, R., John, N., Kirmani, S., Hart, J., Sinapov, J., & Stone, P. (2018, August). Passive demonstrations of light-based robot signals for improved human interpretability. In 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp. 234-239). 84 [53] Arevalo Arboleda, S., Rücker, F., Dierks, T., & Gerken, J. (2021, May). Assisting manipulation and grasping in robot teleoperation with augmented reality visual cues. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-14). [54] Varney, V., Henke, C., & Janssen, D. (2020). Living and Interacting with Robots: Engaging Users in the Development of a Mobile Robot. In Becoming Human with Humanoid-From Physical Interaction to Social Intelligence. IntechOpen. [55] Lasota, P. A., & Shah, J. A. (2015). Analyzing the effects of human-aware motion planning on close-proximity human–robot collaboration. Human Factors, 57(1), 21-33. [56] Hetherington, N. J., Croft, E. A., & Van der Loos, H. M. (2021). Hey robot, which way are you going? nonverbal motion legibility cues for human-robot spatial interaction. IEEE Robotics and Automation Letters, 6(3), 5010-5015. [57] Sreng, J., Lécuyer, A., Mégard, C., & Andriot, C. (2006). Using visual cues of contact to improve interactive manipulation of virtual objects in industrial assembly/maintenance simulations. IEEE Transactions on Visualization and Computer Graphics, 12(5), 1013-1020. [58] Kopp, T., Baumgartner, M., & Kinkel, S. (2021). Success Factors for Introducing Industrial Human-Robot Interaction in Practice: An Empirically Driven Framework. The International Journal of Advanced Manufacturing Technology, 112, 685-704. [59] Carrier, D. S. M. D., & Carrier, D. S. M. D. (2001). Stepper Motors. [60] Komarizadehasl, S., Mobaraki, B., Lozano-Galant, J. A., & Turmo, J. (2020, September). Detailed evaluation of low-cost ranging sensors for structural health monitoring applications. In Proceedings of the International Conference of Recent Trends in Geotechnical and Geo-Environmental Engineering and Education, RTCEE/RTGEE (pp. 8-12). [61] Crozier, W. R. (1999). The meanings of colour: Preferences among hues. Pigment & Resin Technology, 28(1), 6-14. [62] Baraka, K., Rosenthal, S., & Veloso, M. (2016, August). Enhancing human understanding of a mobile robot's state and actions using expressive lights. In 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp. 652-657). [63] Richardson, J. T. (2018). The use of Latin-square designs in educational and psychological research. Educational Research Review, 24, 84-97. [64] Hart, S. G. (2006, October). NASA-task load index (NASA-TLX); 20 years later. In Proceedings of The Human Factors And Ergonomics Society Annual Meeting (Vol. 50, No. 9, pp. 904-908). Sage CA: Los Angeles, CA: Sage publications. [65] Eimontaite, I. (2022). Human–Robot Collaboration Using Visual Cues for Communication. The 21st Century Industrial Robot: When Tools Become Collaborators, 71-85. 85 [66] Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149-1160. [67] Wright, K. D., Stewart, S. H., & Allen Finley, G. (2010). When Are Parents Helpful? A Randomized Clinical Trial of The Efficacy of Parental Presence for Pediatric Anesthesia. Canadian Journal of Anesthesia/Journal Canadien D'anesthésie, 57(8), 751-758. [68] Sullivan, G. M., & Artino Jr, A. R. (2013). Analyzing And Interpreting Data from Likert-Type Scales. Journal of Graduate Medical Education, 5(4), 541-542. [69] Sawyer, S. F. (2009). Analysis of variance: the fundamental concepts. Journal of Manual & Manipulative Therapy, 17(2), 27E-38E. [70] Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15, 625-632. [71] Hetherington, N. J. (2020). Design and evaluation of nonverbal motion cues for human-robot spatial interaction (Doctoral dissertation, University of British Columbia). [72] Kwak, S. G., & Kim, J. H. (2017). Central Limit Theorem: The Cornerstone of Modern Statistics. Korean journal of anesthesiology, 70(2), 144-156. [73] Daniel, W. W., & Cross, C. L. (2018). Biostatistics: a foundation for analysis in the health sciences. Wiley. [74] Glass, G. V. (1966). Testing Homogeneity of Variances. American Educational Research Journal, 3(3), 187-190. [75] Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donahue, B., ... & Levin, J. R. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses. Review of educational research, 68(3), 350-386. [76] Singh, V., Rana, R. K., & Singhal, R. (2013). Analysis of Repeated Measurement Data in the Clinical Trials. Journal of Ayurveda and Integrative Medicine, 4(2), 77. [77] Armstrong, R. A. (2014). When to use the Bonferroni correction. Ophthalmic and Physiological Optics, 34(5), 502-508. [78] Gliem, J. A., & Gliem, R. R. (2003). Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales. Midwest research-to-Practice Conference in Adult, Continuing, and Community Education. [79] Yang, Y., & Green, S. B. (2011). Coefficient Alpha: A Reliability Coefficient For The 21st Century?. Journal of Psychoeducational Assessment, 29(4), 377-392. [80] Khalilzadeh, J., & Tasci, A. D. (2017). Large sample size, significance level, and the effect size: Solutions to perils of using big data for academic research. Tourism Management, 62, 89-96. [81] Rice, M. E., & Harris, G. T. (2005). Comparing effect sizes in follow-up studies: ROC Area, Cohen's d, and r. Law and Human Behavior, 29, 615-620. 86 Appendices Appendix A : Robot Hardware and Software Implementation Details This section provides implementation details for the robot platform used in this study. The robot system includes two main components: the VIMEE gripper and the KUKA robot arm. The hardware and mechatronics work were mainly focused on the VIMEE gripper. The software implementation includes KUKA trajectory planning and visualization in ROS, as well as ROSifying the VIMEE’s Arduino program. A.1 Hardware and Mechatronics Design The main pleating functions of the VIMEE system involve pleat detection, loading, rolling, and fine adjustment inside the gripper jaws. Figure A.1 outlines the functional processes of the system in an architecture block diagram. When the gripper is positioned over a pleat, the gripper jaws remain in the open position for ultrasonic sensors to detect the presence of pleat material. The linear actuator then closes the gripper jaws to clamp the target pleat until a predetermined amount of grip force is measured by the load cell unit. Once the roller system rolls the pleat up tightly into the gripper, the microcontroller initiates manipulation functions to migrate the pleat. As the robot arm moves along a line path, the rollers roll in the same direction to translate the pleat horizontally. The force and distance feedback from the load cell and ToF sensors ensure the target pleat maintains a correct holding position inside the gripper. 87 Figure A.1 The VIMEE gripper’s functionalities in a flow diagram During the auto pleat manipulation, a ToF sensor array constantly measures the height of the pleat and compares it against the initial measured value. A decrease in height indicates a pleat slipping or jamming condition, which may trigger an anti-slip algorithm to halt the pleat migration process and repeatedly adjust the pleat's height and position in the jaws. 88 The gripper design is based on a two-jaw structure. The gripper's body frame is cast from aluminum alloy, enabling the system to achieve high strength and durability cost-effectively. The gripper jaws consist of four trapezoid-shaped aluminum plates. This design offers flexibility for the integration of the sensor and actuators. The gripper’s mechanical layout is depicted in Figure A.2. The two-jaw structure allows the gripper to move linearly and perform pinch grasp movements for pleat manipulation tasks. To move the powered jaw mounted on the lead screw, a stepper motor is equipped to enable the jaw to slide within a range of 0 - 150 mm. As the grip force increases, the gripper jaws move closer to each other. The powered jaw causes the compliant jaw to slightly rotate around a pivot joint, converting horizontal grip force into horizontal load force. Above the compliant jaw, a load cell assembly measures the resulting load force. This assembly includes a spring mechanism that allows fine modulation of force application. Figure A.2 Gripper’s overall mechanical architecture 89 Position sensors are placed above the linear actuator and vertically mounted inside the powered jaw to position the gripper and monitor the pleat's height. Between the two jaws, a downward-facing ultrasonic sensor detects the presence of pleat material before manipulation. Additionally, the electronics enclosure above the left jaw houses the Arduino, PCB shield, motor drivers, and wiring connections. As the pleating process requires the roller to move the fabric material within a light vacuum environment, having an adequate clamping force and roller torque is crucial to provide friction resistance. To quantify these parameters, the Capstone team conducted a physical experiment to simulate the shearing motion. Sandpaper exhibited the lowest Coefficient of Friction (CoF) for rolling the fabric membrane. However, the abrasive nature of the surface raised concerns about damaging the membrane under high compressive forces. In contrast, the rubber materials showed an average CoF of around 0.85. Neoprene crosshatched rubber sheet (with a durometer of 40 A and a coefficient of friction of 0.8), featuring an adhesive surface finish, demonstrated the ability to generate sufficient friction between the roller and the exterior material, effectively gripping the pleats. This allowed the inner surfaces of the pleat to still slide with respect to each other, facilitating the roller's grip and rolling action. Considering the dust-free operating environment at DLR, this material is likely to retain its adhesive properties over an extended duration. As a result, the rubber sheet with a durometer of 40A and cross texture emerges as the most suitable candidate. 90 As the electrical design progressed from individual breadboard prototyping to an integrated protoboard, instances of interference due to a floating ground was identified. To address this issue, a customized modular circuit board was manufactured to consolidate the electronics connections within the gripper system. The PCB shield incorporates multiple JST male connectors (Figure A.3), which facilitate the circuit connections for end-stops, load cells, encoders, ultrasonic sensors, ToF arrays, and motor drivers. Figure A.3 PCB shield design A Wheatstone bridge load cell unit with a range of up to 50 lbf has been incorporated to enable force measurement capabilities for the VIMEE gripper, thereby allowing it to adapt to the pleat dynamics during the loading process. As grip force is applied, the compliant jaw pivots and transmits the resulting force through the load cell for measurement. To precisely control the increments of applied force, a pre-tensioning clamp system consisting of a retaining clip and an internal spring has been mounted onto the load cell assembly, preventing an immediate exertion of maximum force. The output data from the load cell is then amplified from 0-200 mV to readable voltages (0-2V) using an Op-Amp circuit in the PCB shield. This configuration enables the microcontroller (Arduino Mega) to receive voltage signals from the load cell. Additionally, the 91 output can be measured by an oscilloscope for calibration purposes. Calibration testing was conducted across four load ranges: low (0 - 500 g), mid (500 g - 1500 g), and high (1500 g - 7200 g). The increments increased from 20 g to 100 g to 900 g as the load range expanded. By applying the incremental loads to the load cell, a linear trend in the calibration curve can be observed in Figure A.4, Figure A.5, and Figure A.6 respectively. 92 Figure A.4 Load cell calibration within a small range Trial 1 Sensor gain – 0.0068 V/kg Off-set – 65.463 mV Trial 2 Sensor gain – 0.0085 V/kg Off-set – 65.132 mV Figure A.5 Load cell calibration within a medium range Trial 1 Sensor gain – 0.0549 V/kg Off-set – 50.164 mV Trial 2 Sensor gain – 0.0496 V/kg Off-set – 54.3 mV 93 Figure A.6 Load cell calibration within a large range Trial 1 Sensor gain – 0.0536 V/kg Off-set – 49.498 mV Trial 2 Sensor gain – 0.0544 V/kg Off-set – 57.24 mV Based on the data collected from each load range, a linear regression model was obtained and used to compare the accuracy of weight measurements across the low-range, mid-range, and high-range categories. With the load cell's output voltages and the corresponding measured weights, the associated sensitivity (K) was calculated and presented in Table A.1. Notably, a significant fluctuation in voltage signals is observed within the low load range, resulting in a difference of 85.33% in comparison to the full-range loads. In contrast, the differences for the mid-range and high-range loads are 3.33% and 0.74%, respectively. These results highlight the outperformance of the load cell sensor when operating in higher load ranges with larger increments. However, the large calibration error within the low-range load can potentially lead to reduced accuracy in small weight measurements. The load cell's sensitivity is inherently determined by its design. 94 Table A.1 Sensitivity comparison for different load ranges Range Actual Sensitivity (K) Full-Range Sensitivity(K) % Difference Low Range 0.00765 V/kg 0.054 V/kg 85.33 Mid-Range 0.0522 V/kg 0.054 V/kg 3.33 High Range 00544 V/kg 0.054 V/kg 0.74% After calibrating the sensors, I proceeded with the hardware integration work on the VIMEE gripper. The initial state in which the gripper was prototype is depicted in Figure A.7 (a). All wires from the protoboard were disconnected, as they are replaced with a PCB. To arrange the intricate network of wires, the sensor cables were correctly labeled, arranged in order, and hidden within the gripper jaws. As the gripper’s final state illustrated in Figure A.7 (b). The gripper, equipped with a 3D-printed protective cover, was subsequently integrated onto the KUKA robot for the purposes of this study. Figure A.7 Evolution of the VIMEE gripper: (a) initial state; (b) final integration 95 A.2 Software Implementation The program for the KUKA robot arm was developed based on an open-source package, IIWA Stack, which integrates KUKA robot’s control features into the ROS platform. ROS uses a communication model based on nodes, topics, messages, services, and actions to facilitate communication and coordination between different components of a robotic system. The functions of the VIMEE gripper were programmed using an Arduino Mega 2560 microcontroller board, enabling communication with the KUKA robot in ROS via the rosserial. The rosserial uses a ROS communication protocol over the UART of the Arduino. It allows the Arduino to be a full-fledged ROS node which can directly publish and subscribe to ROS messages. Figure A.1 High-level schematic of the robot communication network. 96 The software implementation of the KUKA arm in ROS was primarily contributed by Mark Dong, an MEng student in Mechanical Engineering at the University of British Columbia in 2022. I used his program framework to customize the pleat loading and migration functions. Therefore, the relevant sections of Mark’s Capstone Project Report are attached below to describe the development of the KUKA arm's trajectory control and simulation packages in ROS. 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 Appendix B : User Study Consent Form and Questionnaire B.1 Consent Form 112 113 B.2 Questionnaire 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 Appendix C : ANOVA Assumption Tests and Results of Likert-Scale Analysis C.1 Normality Test Table C.1 Tests of normality Kolmogorov-Smirnova Shapiro-Wilk Statistic df Sig. Statistic df Sig. (a) Manual Light Intensity .143 39 .042 .945 39 .054 (b) Manual Colour Gradient Effect .304 39 .000 .813 39 .000 (c) Manual Loading Progress Bar .253 39 .000 .820 39 .000 (d) Manual Blink Frequency .130 39 .092 .954 39 .108 (e) Manual No LED cue .385 39 .000 .582 39 .000 (a) Auto Light Intensity .162 39 .011 .926 39 .014 (b) Auto Gradient Effect .309 39 .000 .752 39 .000 (c) Auto Loading Progress Bar .261 39 .000 .797 39 .000 (d) Auto Blink Frequency .179 39 .003 .933 39 .023 (e) Auto No LED cue .333 39 .000 .738 39 .000 a. Lilliefors Significance Correction C.2 Homogeneity of Variance Test Table C.2 Levene's test of equality of error variances Levene Statistic df1 df2 Sig. Suitability Based on Mean 4.674 9 380 .000 Based on Median 2.649 9 380 .005 Based on Median and with adjusted df 2.649 9 258.465 .006 Based on trimmed mean 4.235 9 380 .000 Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a. Dependent variable: Suitability b. Design: Intercept + Mode + Pattern + Mode * Pattern 131 C.3 Sphericity Test Table C.3 Mauchly's test of sphericity C.4 Reliability Test Table C.4 Reliability statistics C.5 ANOVA Tests on Suitability Scale Table C.5 ANOVA tests of suitability within-subjects effects Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared mode Sphericity Assumed 12.564 1 12.564 11.749 .001 .236 Greenhouse-Geisser 12.564 1.000 12.564 11.749 .001 .236 Huynh-Feldt 12.564 1.000 12.564 11.749 .001 .236 Lower-bound 12.564 1.000 12.564 11.749 .001 .236 Measure: MEASURE_1 Within Subjects Effect Mauchly's W Approx. Chi-Square df Sig. Epsilonb Greenhouse-Geisser Huynh-Feldt Lower-bound mode 1.000 .000 0 . 1.000 1.000 1.000 pattern .663 14.956 9 .092 .833 .923 .250 mode * pattern .907 3.571 9 .937 .948 1.000 .250 Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix. a. Design: Intercept Within Subjects Design: mode + pattern + mode * pattern b. May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in the Tests of Within-Subjects Effects table. Cronbach's Alpha N of Items .836 70 132 Error(mode) Sphericity Assumed 40.636 38 1.069 Greenhouse-Geisser 40.636 38.000 1.069 Huynh-Feldt 40.636 38.000 1.069 Lower-bound 40.636 38.000 1.069 pattern Sphericity Assumed 918.503 4 229.626 93.650 .000 .711 Greenhouse-Geisser 918.503 3.332 275.671 93.650 .000 .711 Huynh-Feldt 918.503 3.691 248.881 93.650 .000 .711 Lower-bound 918.503 1.000 918.503 93.650 .000 .711 Error(pattern) Sphericity Assumed 372.697 152 2.452 Greenhouse-Geisser 372.697 126.612 2.944 Huynh-Feldt 372.697 140.240 2.658 Lower-bound 372.697 38.000 9.808 mode * pattern Sphericity Assumed 2.256 4 .564 .907 .462 .023 Greenhouse-Geisser 2.256 3.793 .595 .907 .458 .023 Huynh-Feldt 2.256 4.000 .564 .907 .462 .023 Lower-bound 2.256 1.000 2.256 .907 .347 .023 Error(mode*pattern) Sphericity Assumed 94.544 152 .622 Greenhouse-Geisser 94.544 144.121 .656 Huynh-Feldt 94.544 152.000 .622 Lower-bound 94.544 38.000 2.488 133 Table C.6 Pairwise comparisons of operating modes on suitability scale Dependent Variable: Suitability (I) Mode (J) Mode Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Semi-automatic Mode Automatic Mode -.359* .136 .009 -.626 -.092 Automatic Mode Semi-automatic Mode .359* .136 .009 .092 .626 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. Table C.7 Pairwise comparisons of LED cue patterns on suitability scale Dependent Variable: Suitability (I) Pattern (J) Pattern Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Light Brightness Colour Gradient -1.256* .215 .000 -1.862 -.650 Loading Progress Bar -1.359* .215 .000 -1.965 -.753 Flashing Light .872* .215 .001 .266 1.478 No Light 2.795* .215 .000 2.189 3.401 Colour Gradient Light Brightness 1.256* .215 .000 .650 1.862 Loading Progress Bar -.103 .215 1.000 -.709 .504 Flashing Light 2.128* .215 .000 1.522 2.734 No Light 4.051* .215 .000 3.445 4.657 Loading Progress Bar Light Brightness 1.359* .215 .000 .753 1.965 Colour Gradient .103 .215 1.000 -.504 .709 Flashing Light 2.231* .215 .000 1.625 2.837 No Light 4.154* .215 .000 3.548 4.760 Flashing Light Light Brightness -.872* .215 .001 -1.478 -.266 Colour Gradient -2.128* .215 .000 -2.734 -1.522 Loading Progress Bar -2.231* .215 .000 -2.837 -1.625 134 No Light 1.923* .215 .000 1.317 2.529 No Light Light Brightness -2.795* .215 .000 -3.401 -2.189 Colour Gradient -4.051* .215 .000 -4.657 -3.445 Loading Progress Bar -4.154* .215 .000 -4.760 -3.548 Flashing Light -1.923* .215 .000 -2.529 -1.317 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. C.6 ANOVA Tests on Legibility Scale Table C.8 ANOVA tests of legibility within-subjects effects Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Mode Sphericity Assumed 2.308 1 2.308 2.618 .114 .064 Greenhouse-Geisser 2.308 1.000 2.308 2.618 .114 .064 Huynh-Feldt 2.308 1.000 2.308 2.618 .114 .064 Lower-bound 2.308 1.000 2.308 2.618 .114 .064 Error(Mode) Sphericity Assumed 33.492 38 .881 Greenhouse-Geisser 33.492 38.000 .881 Huynh-Feldt 33.492 38.000 .881 Lower-bound 33.492 38.000 .881 Pattern Sphericity Assumed 1140.882 4 285.221 163.279 .000 .811 Greenhouse-Geisser 1140.882 3.358 339.776 163.279 .000 .811 Huynh-Feldt 1140.882 3.722 306.491 163.279 .000 .811 Lower-bound 1140.882 1.000 1140.882 163.279 .000 .811 135 Error(Pattern) Sphericity Assumed 265.518 152 1.747 Greenhouse-Geisser 265.518 127.595 2.081 Huynh-Feldt 265.518 141.451 1.877 Lower-bound 265.518 38.000 6.987 Mode * Pattern Sphericity Assumed 1.846 4 .462 .822 .513 .021 Greenhouse-Geisser 1.846 3.676 .502 .822 .505 .021 Huynh-Feldt 1.846 4.000 .462 .822 .513 .021 Lower-bound 1.846 1.000 1.846 .822 .370 .021 Error(Mode*Pattern) Sphericity Assumed 85.354 152 .562 Greenhouse-Geisser 85.354 139.693 .611 Huynh-Feldt 85.354 152.000 .562 Lower-bound 85.354 38.000 2.246 Table C.9 Pairwise comparisons of LED cue patterns on legibility scale (I) Pattern (J) Pattern Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Light Brightness Colour Gradient -1.590* .225 .000 -2.260 -.920 Loading Progress Bar -1.308* .203 .000 -1.914 -.702 Flashing Light .936* .197 .000 .348 1.523 No Light 3.128* .220 .000 2.472 3.784 Colour Gradient Light Brightness 1.590* .225 .000 .920 2.260 Loading Progress Bar .282 .151 .699 -.169 .733 Flashing Light 2.526* .226 .000 1.851 3.200 136 No Light 4.718* .184 .000 4.168 5.268 Loading Progress Bar Light Brightness 1.308* .203 .000 .702 1.914 Colour Gradient -.282 .151 .699 -.733 .169 Flashing Light 2.244* .228 .000 1.565 2.923 No Light 4.436* .221 .000 3.776 5.095 Flashing Light Light Brightness -.936* .197 .000 -1.523 -.348 Colour Gradient -2.526* .226 .000 -3.200 -1.851 Loading Progress Bar -2.244* .228 .000 -2.923 -1.565 No Light 2.192* .244 .000 1.465 2.920 No Light Light Brightness -3.128* .220 .000 -3.784 -2.472 Colour Gradient -4.718* .184 .000 -5.268 -4.168 Loading Progress Bar -4.436* .221 .000 -5.095 -3.776 Flashing Light -2.192* .244 .000 -2.920 -1.465 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. C.7 ANOVA Tests on Intuitiveness Scale Table C.10 ANOVA tests of intuitiveness within-subjects effects Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Mode Sphericity Assumed 2.156 1 2.156 1.776 .191 .045 Greenhouse-Geisser 2.156 1.000 2.156 1.776 .191 .045 Huynh-Feldt 2.156 1.000 2.156 1.776 .191 .045 Lower-bound 2.156 1.000 2.156 1.776 .191 .045 Error(Mode) Sphericity Assumed 46.144 38 1.214 Greenhouse-Geisser 46.144 38.000 1.214 Huynh-Feldt 46.144 38.000 1.214 137 Lower-bound 46.144 38.000 1.214 Pattern Sphericity Assumed 1099.400 4 274.850 133.303 .000 .778 Greenhouse-Geisser 1099.400 2.999 366.551 133.303 .000 .778 Huynh-Feldt 1099.400 3.285 334.685 133.303 .000 .778 Lower-bound 1099.400 1.000 1099.400 133.303 .000 .778 Error(Pattern) Sphericity Assumed 313.400 152 2.062 Greenhouse-Geisser 313.400 113.974 2.750 Huynh-Feldt 313.400 124.826 2.511 Lower-bound 313.400 38.000 8.247 Mode * Pattern Sphericity Assumed 1.831 4 .458 .729 .573 .019 Greenhouse-Geisser 1.831 3.179 .576 .729 .544 .019 Huynh-Feldt 1.831 3.503 .523 .729 .556 .019 Lower-bound 1.831 1.000 1.831 .729 .398 .019 Error(Mode*Pattern) Sphericity Assumed 95.369 152 .627 Greenhouse-Geisser 95.369 120.801 .789 Huynh-Feldt 95.369 133.116 .716 Lower-bound 95.369 38.000 2.510 138 Table C.11 Pairwise comparisons of LED cue patterns on intuitiveness scale (I) Pattern (J) Pattern Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Light Brightness Colour Gradient -1.782* .222 .000 -2.443 -1.121 Loading Progress Bar -1.577* .199 .000 -2.170 -.983 Flashing Light .795* .189 .002 .232 1.358 No Light 2.795* .255 .000 2.035 3.555 Colour Gradient Light Brightness 1.782* .222 .000 1.121 2.443 Loading Progress Bar .205 .133 1.000 -.192 .603 Flashing Light 2.577* .254 .000 1.819 3.335 No Light 4.577* .243 .000 3.854 5.300 Loading Progress Bar Light Brightness 1.577* .199 .000 .983 2.170 Colour Gradient -.205 .133 1.000 -.603 .192 Flashing Light 2.372* .241 .000 1.652 3.091 No Light 4.372* .231 .000 3.682 5.061 Flashing Light Light Brightness -.795* .189 .002 -1.358 -.232 Colour Gradient -2.577* .254 .000 -3.335 -1.819 Loading Progress Bar -2.372* .241 .000 -3.091 -1.652 No Light 2.000* .293 .000 1.126 2.874 No Light Light Brightness -2.795* .255 .000 -3.555 -2.035 Colour Gradient -4.577* .243 .000 -5.300 -3.854 Loading Progress Bar -4.372* .231 .000 -5.061 -3.682 Flashing Light -2.000* .293 .000 -2.874 -1.126 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. 139 C.8 ANOVA Tests on Interactivity Scale Table C.12 ANOVA tests of interactivity within-subjects effects Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Mode Sphericity Assumed 12.564 1 12.564 10.371 .003 .214 Greenhouse-Geisser 12.564 1.000 12.564 10.371 .003 .214 Huynh-Feldt 12.564 1.000 12.564 10.371 .003 .214 Lower-bound 12.564 1.000 12.564 10.371 .003 .214 Error(Mode) Sphericity Assumed 46.036 38 1.211 Greenhouse-Geisser 46.036 38.000 1.211 Huynh-Feldt 46.036 38.000 1.211 Lower-bound 46.036 38.000 1.211 Pattern Sphericity Assumed 861.215 4 215.304 110.642 .000 .744 Greenhouse-Geisser 861.215 3.325 258.978 110.642 .000 .744 Huynh-Feldt 861.215 3.683 233.861 110.642 .000 .744 Lower-bound 861.215 1.000 861.215 110.642 .000 .744 Error(Pattern) Sphericity Assumed 295.785 152 1.946 Greenhouse-Geisser 295.785 126.367 2.341 Huynh-Feldt 295.785 139.938 2.114 Lower-bound 295.785 38.000 7.784 Mode * Pattern Sphericity Assumed 3.872 4 .968 1.102 .358 .028 Greenhouse-Geisser 3.872 3.369 1.149 1.102 .354 .028 140 Huynh-Feldt 3.872 3.736 1.036 1.102 .357 .028 Lower-bound 3.872 1.000 3.872 1.102 .300 .028 Error(Mode*Pattern) Sphericity Assumed 133.528 152 .878 Greenhouse-Geisser 133.528 128.024 1.043 Huynh-Feldt 133.528 141.981 .940 Lower-bound 133.528 38.000 3.514 Table C.13 Pairwise comparisons of operating modes on interactivity scale (I) Mode (J) Mode Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Semi-automatic Mode Automatic Mode -.359* .111 .003 -.585 -.133 Automatic Mode Semi-automatic Mode .359* .111 .003 .133 .585 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. Table C.14 Pairwise comparisons of LED cue patterns on interactivity scale (I) Pattern (J) Pattern Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Light Brightness Colour Gradient -1.346* .217 .000 -1.994 -.698 Loading Progress Bar -1.090* .196 .000 -1.674 -.506 Flashing Light .846* .201 .002 .246 1.446 No Light 2.756* .254 .000 2.000 3.513 Colour Gradient Light Brightness 1.346* .217 .000 .698 1.994 141 Loading Progress Bar .256 .150 .960 -.191 .704 Flashing Light 2.192* .217 .000 1.545 2.839 No Light 4.103* .250 .000 3.358 4.847 Loading Progress Bar Light Brightness 1.090* .196 .000 .506 1.674 Colour Gradient -.256 .150 .960 -.704 .191 Flashing Light 1.936* .217 .000 1.290 2.582 No Light 3.846* .264 .000 3.058 4.634 Flashing Light Light Brightness -.846* .201 .002 -1.446 -.246 Colour Gradient -2.192* .217 .000 -2.839 -1.545 Loading Progress Bar -1.936* .217 .000 -2.582 -1.290 No Light 1.910* .244 .000 1.184 2.636 No Light Light Brightness -2.756* .254 .000 -3.513 -2.000 Colour Gradient -4.103* .250 .000 -4.847 -3.358 Loading Progress Bar -3.846* .264 .000 -4.634 -3.058 Flashing Light -1.910* .244 .000 -2.636 -1.184 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. C.9 ANOVA Tests on Mental Demand Scale Table C.15 ANOVA tests of mental demand within-subjects effects Measure: Mental Demand Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Mode Sphericity Assumed 75.856 1 75.856 34.753 .000 .478 Greenhouse-Geisser 75.856 1.000 75.856 34.753 .000 .478 Huynh-Feldt 75.856 1.000 75.856 34.753 .000 .478 Lower-bound 75.856 1.000 75.856 34.753 .000 .478 Error(Mode) Sphericity Assumed 82.944 38 2.183 142 Greenhouse-Geisser 82.944 38.000 2.183 Huynh-Feldt 82.944 38.000 2.183 Lower-bound 82.944 38.000 2.183 Pattern Sphericity Assumed 313.779 4 78.445 51.928 .000 .577 Greenhouse-Geisser 313.779 2.540 123.545 51.928 .000 .577 Huynh-Feldt 313.779 2.737 114.647 51.928 .000 .577 Lower-bound 313.779 1.000 313.779 51.928 .000 .577 Error(Pattern) Sphericity Assumed 229.621 152 1.511 Greenhouse-Geisser 229.621 96.512 2.379 Huynh-Feldt 229.621 104.003 2.208 Lower-bound 229.621 38.000 6.043 Mode * Pattern Sphericity Assumed 4.785 4 1.196 1.575 .184 .040 Greenhouse-Geisser 4.785 3.127 1.530 1.575 .197 .040 Huynh-Feldt 4.785 3.440 1.391 1.575 .192 .040 Lower-bound 4.785 1.000 4.785 1.575 .217 .040 Error(Mode*Pattern) Sphericity Assumed 115.415 152 .759 Greenhouse-Geisser 115.415 118.822 .971 Huynh-Feldt 115.415 130.704 .883 Lower-bound 115.415 38.000 3.037 143 Table C.16 Pairwise comparisons of operating modes on mental demand scale Measure: Mental Demand (I) Mode (J) Mode Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Semi-automatic Mode Automatic Mode .882* .150 .000 .579 1.185 Automatic Mode Semi-automatic Mode -.882* .150 .000 -1.185 -.579 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. Table C.17 Pairwise comparisons of LED cue patterns on mental demand scale Measure: Mental Demand (I) Pattern (J) Pattern Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Light Brightness Colour Gradient .936* .171 .000 .428 1.444 Loading Progress Bar .897* .157 .000 .429 1.366 Flashing Light -.410 .162 .156 -.893 .073 No Light -1.474* .178 .000 -2.005 -.944 Colour Gradient Light Brightness -.936* .171 .000 -1.444 -.428 Loading Progress Bar -.038 .117 1.000 -.386 .309 Flashing Light -1.346* .201 .000 -1.944 -.749 No Light -2.410* .253 .000 -3.165 -1.656 Loading Progress Bar Light Brightness -.897* .157 .000 -1.366 -.429 Colour Gradient .038 .117 1.000 -.309 .386 Flashing Light -1.308* .183 .000 -1.854 -.761 No Light -2.372* .263 .000 -3.157 -1.587 Flashing Light Light Brightness .410 .162 .156 -.073 .893 Colour Gradient 1.346* .201 .000 .749 1.944 144 Loading Progress Bar 1.308* .183 .000 .761 1.854 No Light -1.064* .234 .001 -1.761 -.367 No Light Light Brightness 1.474* .178 .000 .944 2.005 Colour Gradient 2.410* .253 .000 1.656 3.165 Loading Progress Bar 2.372* .263 .000 1.587 3.157 Flashing Light 1.064* .234 .001 .367 1.761 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. C.10 ANOVA Tests on Effort Demand Scale Table C.18 ANOVA tests of effort demand within-subjects effects Measure: Effort Demand Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Mode Sphericity Assumed 96.503 1 96.503 24.079 .000 .388 Greenhouse-Geisser 96.503 1.000 96.503 24.079 .000 .388 Huynh-Feldt 96.503 1.000 96.503 24.079 .000 .388 Lower-bound 96.503 1.000 96.503 24.079 .000 .388 Error(Mode) Sphericity Assumed 152.297 38 4.008 Greenhouse-Geisser 152.297 38.000 4.008 Huynh-Feldt 152.297 38.000 4.008 Lower-bound 152.297 38.000 4.008 Pattern Sphericity Assumed 263.821 4 65.955 45.163 .000 .543 Greenhouse-Geisser 263.821 1.865 141.475 45.163 .000 .543 145 Huynh-Feldt 263.821 1.957 134.789 45.163 .000 .543 Lower-bound 263.821 1.000 263.821 45.163 .000 .543 Error(Pattern) Sphericity Assumed 221.979 152 1.460 Greenhouse-Geisser 221.979 70.862 3.133 Huynh-Feldt 221.979 74.377 2.985 Lower-bound 221.979 38.000 5.842 Mode * Pattern Sphericity Assumed 11.472 4 2.868 4.009 .004 .095 Greenhouse-Geisser 11.472 3.546 3.235 4.009 .006 .095 Huynh-Feldt 11.472 3.956 2.900 4.009 .004 .095 Lower-bound 11.472 1.000 11.472 4.009 .052 .095 Error(Mode*Pattern) Sphericity Assumed 108.728 152 .715 Greenhouse-Geisser 108.728 134.766 .807 Huynh-Feldt 108.728 150.343 .723 Lower-bound 108.728 38.000 2.861 Table C.19 Pairwise comparisons of operating modes on effort demand scale Measure: Effort Demand (I) Mode (J) Mode Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Semi-automatic Mode Automatic Mode .995* .203 .000 .584 1.405 Automatic Mode Semi-automatic Mode -.995* .203 .000 -1.405 -.584 Based on estimated marginal means 146 *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. Table C.20 Pairwise comparisons of LED cue patterns on effort demand scale Measure: Effort Demand (I) Pattern (J) Pattern Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Light Brightness Colour Gradient .872* .198 .001 .283 1.461 Loading Progress Bar .744* .180 .002 .208 1.280 Flashing Light -.346 .129 .106 -.730 .038 No Light -1.397* .169 .000 -1.900 -.895 Colour Gradient Light Brightness -.872* .198 .001 -1.461 -.283 Loading Progress Bar -.128 .091 1.000 -.400 .144 Flashing Light -1.218* .198 .000 -1.807 -.629 No Light -2.269* .286 .000 -3.122 -1.416 Loading Progress Bar Light Brightness -.744* .180 .002 -1.280 -.208 Colour Gradient .128 .091 1.000 -.144 .400 Flashing Light -1.090* .179 .000 -1.623 -.557 No Light -2.141* .260 .000 -2.917 -1.365 Flashing Light Light Brightness .346 .129 .106 -.038 .730 Colour Gradient 1.218* .198 .000 .629 1.807 Loading Progress Bar 1.090* .179 .000 .557 1.623 No Light -1.051* .170 .000 -1.558 -.544 No Light Light Brightness 1.397* .169 .000 .895 1.900 Colour Gradient 2.269* .286 .000 1.416 3.122 Loading Progress Bar 2.141* .260 .000 1.365 2.917 Flashing Light 1.051* .170 .000 .544 1.558 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. 147 C.11 ANOVA Tests on Frustration Scale Table C.21 ANOVA tests of frustration within-subjects effects Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared Mode Sphericity Assumed 58.464 1 58.464 26.691 .000 .413 Greenhouse-Geisser 58.464 1.000 58.464 26.691 .000 .413 Huynh-Feldt 58.464 1.000 58.464 26.691 .000 .413 Lower-bound 58.464 1.000 58.464 26.691 .000 .413 Error(Mode) Sphericity Assumed 83.236 38 2.190 Greenhouse-Geisser 83.236 38.000 2.190 Huynh-Feldt 83.236 38.000 2.190 Lower-bound 83.236 38.000 2.190 Pattern Sphericity Assumed 306.641 4 76.660 30.364 .000 .444 Greenhouse-Geisser 306.641 1.884 162.742 30.364 .000 .444 Huynh-Feldt 306.641 1.979 154.923 30.364 .000 .444 Lower-bound 306.641 1.000 306.641 30.364 .000 .444 Error(Pattern) Sphericity Assumed 383.759 152 2.525 Greenhouse-Geisser 383.759 71.600 5.360 Huynh-Feldt 383.759 75.214 5.102 Lower-bound 383.759 38.000 10.099 Mode * Pattern Sphericity Assumed 4.856 4 1.214 1.884 .116 .047 Greenhouse-Geisser 4.856 2.903 1.673 1.884 .138 .047 148 Huynh-Feldt 4.856 3.169 1.533 1.884 .133 .047 Lower-bound 4.856 1.000 4.856 1.884 .178 .047 Error(Mode*Pattern) Sphericity Assumed 97.944 152 .644 Greenhouse-Geisser 97.944 110.317 .888 Huynh-Feldt 97.944 120.420 .813 Lower-bound 97.944 38.000 2.577 Table C.22 Pairwise comparisons of operating modes on frustration scale (I) Mode (J) Mode Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Semi-automatic Mode Automatic Mode .774* .150 .000 .471 1.078 Automatic Mode Semi-automatic Mode -.774* .150 .000 -1.078 -.471 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni. Table C.23 Pairwise comparisons of LED cue patterns on frustration scale (I) Pattern (J) Pattern Mean Difference (I-J) Std. Error Sig.b 95% Confidence Interval for Differenceb Lower Bound Upper Bound Light Brightness Colour Gradient 1.064* .196 .000 .479 1.649 Loading Progress Bar .987* .164 .000 .499 1.475 Flashing Light -.449 .152 .055 -.903 .005 149 No Light -1.282* .288 .001 -2.140 -.424 Colour Gradient Light Brightness -1.064* .196 .000 -1.649 -.479 Loading Progress Bar -.077 .114 1.000 -.417 .263 Flashing Light -1.513* .239 .000 -2.225 -.800 No Light -2.346* .384 .000 -3.489 -1.203 Loading Progress Bar Light Brightness -.987* .164 .000 -1.475 -.499 Colour Gradient .077 .114 1.000 -.263 .417 Flashing Light -1.436* .210 .000 -2.063 -.809 No Light -2.269* .360 .000 -3.342 -1.197 Flashing Light Light Brightness .449 .152 .055 -.005 .903 Colour Gradient 1.513* .239 .000 .800 2.225 Loading Progress Bar 1.436* .210 .000 .809 2.063 No Light -.833 .291 .068 -1.701 .035 No Light Light Brightness 1.282* .288 .001 .424 2.140 Colour Gradient 2.346* .384 .000 1.203 3.489 Loading Progress Bar 2.269* .360 .000 1.197 3.342 Flashing Light .833 .291 .068 -.035 1.701 Based on estimated marginal means *. The mean difference is significant at the .05 level. b. Adjustment for multiple comparisons: Bonferroni.
Thesis/Dissertation
eng
Mechanical Engineering
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Van der Loos, Hendrik F. Machiel
Design and evaluation of visual cues for human-robot collaboration in carbon fiber-reinforced polymer (CFRP) manufacturing
Text