Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Communicating emotion through a haptic link : a study of the influence of metaphor, personal space and… Smith, Jocelyn Darlene 2005

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_2006-0110.pdf [ 7.41MB ]
Metadata
JSON: 831-1.0051512.json
JSON-LD: 831-1.0051512-ld.json
RDF/XML (Pretty): 831-1.0051512-rdf.xml
RDF/JSON: 831-1.0051512-rdf.json
Turtle: 831-1.0051512-turtle.txt
N-Triples: 831-1.0051512-rdf-ntriples.txt
Original Record: 831-1.0051512-source.json
Full Text
831-1.0051512-fulltext.txt
Citation
831-1.0051512.ris

Full Text

Communicating Emotion Through a Haptic Link a study of the influence of metaphor, personal space and relationship by J o c e l y n Darlene S m i t h  B . S c , T h e U n i v e r s i t y of B r i t i s h C o l u m b i a , 2003  A THESIS S U B M I T T E D IN PARTIAL F U L F I L M E N T O F THE REQUIREMENTS FOR THE DEGREE OF Master of Science in T h e F a c u l t y of G r a d u a t e Studies (Computer Science)  The University Of British Columbia November  . 2005  © J o c e l y n Darlene S m i t h 2005  Abstract T h e w o r l d is more a n d more connected a n d yet we are often physically distant from people we care about.  Technology increasingly supports remote inter-  personal communication but has yet to integrate our sense of touch into this interaction.  Researchers i n the field of haptics (touch a n d technology) have  started exploring computer-mediated touch interaction. T h e question is how should a computer-mediated person-to-person touch interaction be designed. In this thesis, we concentrate on how the design of a haptic interaction model influences performance and subjective experience of a d y a d . Specifically, we examine the effect of using different metaphors to develop a n d e x p l a i n the haptic interaction model, creating interaction models w i t h a n d w i t h o u t a haptic display of personal space, and the type of relationship shared by the d y a d using the device on ability to communicate emotion haptically. We also explore how dyads use these interactions to communicate emotion. W e r a n a structured study, i n w h i c h participants communicated emotion w i t h a haptic device and found that participants were able to communicate some emotional content through the haptic interactions. A significant effect of the interaction metaphor on performance was found. P a r t i c i p a n t s preferred interactions w i t h a haptic indicator of personal space, a n d participants' reported metaphor preferences depended on their relationship. F i n a l l y , we found that c o m m o n actions were used to express each emotion, even though this is a new media unfamiliar to participants.  Contents Abstract  ii  Contents  iii  L i s t of Tables  vii  List of Figures  viii  Acknowledgements  I 1  2  Thesis  l x  1  Introduction  2  1.1  Motivation  2  1.2  Objectives  3  1.3  Research A p p r o a c h  3  1.4  Thesis Structure  4  Related Work  6  2.1  Media  6  2.2  Co-located C o m m u n i c a t i o n  7  2.2.1  Touch  7  2.2.2  Personal Space  8  2.2.3  Emotion  9  3  2.3  H a p t i c s Overview  2.4  C o m p u t e r M e d i a t e d Person-to-Person H a p t i c Interactions . . . .  11  Interaction D e s i g n  15  3.1  Role of V i s u a l Representations  16  3.2  Metaphors  16  3.2.1  Metaphors for the User S t u d y  19  3.2.2  Implementation Details  20  3.3  4  9  Personal Space Indicator  25  3.3.1  Initial Spatial S t i m u l i - V i b r a t i o n s and E v a l u a t i o n . . . .  26  3.3.2  F i n a l Spatial S t i m u l i - V i b r a t i o n s and E v a l u a t i o n  28  3.4  Relationship  30  3.5  Summary  31  E m o t i o n Communication Study  32  4.1  Objective  32  4.2  Hypotheses  34  4.2.1  Factors and Performance  34  4.2.2  Interaction between Factors and Performance  35  4.2.3  Subjective Experience  35  4.3  Metrics  36  4.4  Design  36  4.5  4.4.1  P h y s i c a l Setup  37  4.4.2  Equipment  40  4.4.3  Protocol  40  4.4.4  Picking Emotions  42  4.4.5  R e c r u i t i n g Subjects  42  Analysis  43  4.5.1  E m o t i o n s Conveyed Statistics  43  4.5.2  Exploration  43  5  Results  44  5.1  P a r t i c i p a n t s and S t u d y Sessions  44  5.2  O r d e r i n g Effect  45  5.3  A b i l i t y to C o m m u n i c a t e E m o t i o n s  45  5.3.1  O v e r a l l Performance  46  5.3.2  Performance by M e t a p h o r  48  5.3.3  Performance by Space Indicator  49  5.3.4  Performance by Relationship  49  5.3.5  Interaction Between E x p e r i m e n t Factors  50  5.4  5.5  5.6  6  Strategies for C o m m u n i c a t i n g E m o t i o n  51  5.4.1  Strategies for Conveying E m o t i o n  51  5.4.2  Strategies for Perceiving E m o t i o n  54  5.4.3  Strategies for Interacting  55  Subjective aspects of Interaction Experience  56  5.5.1  Preference  56  5.5.2  Connection  57  5.5.3  Comfort  57  5.5.4  Perceived A b i l i t y to Convey  58  5.5.5  Perceived A b i l i t y to Perceive  59  Performance and Strategies for Conveying  60  5.6.1  Success by Conveyer  60  5.6.2  Below Average  62  5.6.3  A b o v e Average  63  5.6.4  M o s t Successful  67  Discussion  69  6.1  Design, M o d e l , Relationship and Interaction  70  6.1.1  Metaphor  70  6.1.2  Space  71  6.1.3  72  6.2  C o m m u n i c a t i n g E m o t i o n : A c t i o n Strategies  74  6.3  Expressive C a p a c i t y of Interaction Models  76  6.3.1  M e t a p h o r and Space - E x p e r i m e n t a l Interaction Effect . .  76  6.3.2  Desired Expressiveness  77  6.4  R e a l W o r l d Influences V i r t u a l Interaction  77  6.4.1  Strategies  78  6.4.2  Relationship and M e t a p h o r  79  6.5  Success as an E m o t i o n a l C o m m u n i c a t i o n Device  79  6.6  R e m a r k s on E x p e r i m e n t Design  82  6.6.1  T r a i n i n g for D y a d H a p t i c Models  82  6.6.2  Motivation  83  6.7 7  Relationship  Summary  84  Conclusion  87  7.1  Contributions  87  7.2  Future W o r k  89  7.3  F i n a l Words  90  Bibliography  92  A  E x p e r i m e n t Script  97  A.l  Instructions  97  A.2  Interaction M e t a p h o r Script  98  B  E m o t i o n Strategy F o r m  C  F i n a l Questionaire  .  100 102  L i s t of Tables 3.1  Spatial S t u d y Results  29  4.1  Condition Ordering  37  5.1  Sent and Perceived E m o t i o n s  46  5.2  Correct A r o u s a l or Valence  48  5.3  E m o t i o n s successfully communicated by metaphor  48  5.4  E m o t i o n s successfully communicated by relationship type  5.5  E m o t i o n s successfully communicated by relationship and metaphor 51  5.6  E m o t i o n s communicated by w i t h i n subject conditions  51  5.7  Often Used A c t i o n s for each E m o t i o n  55  5.8  Relationship and M e t a p h o r / S p a c e Preference  57  5.9  Relationship and M e t a p h o r / S p a c e Connection  58  . . . .  50  5.10 Relationship and M e t a p h o r / S p a c e C o m f o r t 5.11 Relationship and M e t a p h o r / S p a c e Perceived A b i l i t y to Convey  59 .  60  5.12 Relationship and M e t a p h o r / S p a c e Perceived A b i l i t y to Perceive .  62  5.13 Successful Couple Performance across conditions  67  6.1  75  A c t i o n Use and Performance  L i s t of Figures 3.1  Scale of Intimacy of H a p t i c Interaction  17  3.2  V i s u a l representation of P i n g P o n g metaphor  21  3.3  V i s u a l Representation of the H a n d Stroke M e t a p h o r  23  3.4  H a n d Stroke metaphor: force v.s. relative velocity  24  4.1  Experiment SetUp Diagram  38  4.2  Haptic Knob  39  4.3  Experiment SetUp Photo  39  5.1  Sent vs Received E m o t i o n s  47  5.2  A c t i o n s Used to Convey E m o t i o n s  53  5.3  A c t i o n s Used to Convey E m o t i o n s T w o  54  5.4  N u m b e r Correct by Conveyer  61  5.5  Strategies of Below Average Conveyers  63  5.6  Sent vs Received E m o t i o n s : below average conveyers  64  5.7  Strategies of A b o v e Average Conveyers  65  5.8  Sent vs Received E m o t i o n s : above average conveyers  66  Acknowledgements W o w , I have finished my M S c thesis, and I had lots of help along the way. I w o u l d like to thank my supervisor K a r o n M a c L e a n . K a r o n seemed to know when to let me wander and discover, and when to reign me i n . She encouraged me to go to grad school and has been a mentor to me from the start. A l s o I would like to acknowledge the awesome influence that m y flute teacher, C a t h y Dochsteader, and my high school m a t h teacher, M s . B a i r d , have had on my life. T h a n k s to C r i s t i n a C o n a t i for being m y second reader and attending my presentation. T h a n k s to K e l l o g g B o o t h and M i c h i e l van de Panne for being on my thesis committee and to Joseph L u k for proof reading. M y family have always been there for me, and have been the sunshine i n my life from the get go. T h a n k s M o m (Sheila S m i t h ) , and D a d ( R o n S m i t h ) . M y sister has always been my closest friend, and even helped pick apart my experiment ideas u n t i l I had one that made sense. T h a n k s Rachelle for being such a great sis! I have been lucky to meet and befriend many magnificent people but w o u l d like to thanks here those who fed me and kept me sane d u r i n g the past few months: Cecile L e u n g , Stacy Langsdale, V i a n n C h a n and M i c h e a l Lee, M a r k C r o w l e y and L i l y H u e , G r e g K e m p e , M i k e K l a a s , and W a r r e n C h e u n g . T h a n k s also to my labmates. Especially to C o l i n Swindells, and M a r i o E n r i q u e z who have been friends and accomplices of mine for the past year and a half. F i n a l l y , I w o u l d like to acknowledge everyone i n my grad class. Never before have I met such a fun, intelligent and interesting group of people.  Part I Thesis  Chapter 1 Introduction 1.1  Motivation  In an increasingly connected w o r l d , we often find ourselves physically separated from those we care about.  A plethora of electronic communication devices  including the telephone, e m a i l , cell phones, instant messengers a n d blogs provide us w i t h means of connecting w i t h people who are not physically present w i t h us. However, these devices do not enable us to use our sense of touch d u r i n g remote interaction. T o u c h is an i m p o r t a n t aspect of person-to-person communication i n faceto-face situations. Research has shown that touch can have a profound effect on our m o o d and behaviour. Furthermore, touch plays an i m p o r t a n t role i n establishing and m a i n t a i n i n g intimate relationships [21]. For many years, researchers i n computer science, engineering and psychology have been w o r k i n g to better understand our sense of touch and to b u i l d , evaluate and understand systems that allow us to interact w i t h computers through our sense of touch. T h i s field of study is k n o w n as haptics. T h e m a j o r i t y of haptics research has focused on interaction between one person and a computer. However, several research projects have started to look at using touch for computer mediated c o m m u n i c a t i o n between multiple people. Some of these projects study the possible benefits of adding touch to existing interactions, while others create new touch interactions.  1.2  Objectives  T h e "computer" i n computer mediated communication means that the interaction is no longer direct. Therefore, it is possible to design and create new interactions. C o m m u n i c a t i o n s researchers tell us that people t h i n k of and use new media interactions as new communication channels w i t h their own strengths and weaknesses, rather t h a n as deficient face-to-face communication [17]. G i v e n the  flexibility  to design computer mediated haptic (touch) interactions, and  knowing that such interactions w i l l be judged by their unique strengths and weaknesses, how should an interaction designer proceed? T h e challenge is to create a compelling interaction model. In this thesis, we examine several aspects of a computer mediated dyadic (two person) haptic interaction, and t r y to assess their influence on the emotional expressiveness of the interaction and on the subjective experience of the pair engaged i n the interaction. W e explore three aspects of  computer-mediated  haptic interaction: 1. M e t a p h o r used to design and explain the interaction 2. Presence or absence of a haptic display of interpersonal space 3. Relationship shared between the people i n the d y a d engaged i n the interaction  1.3  Research Approach  O u r approach to the problem of discerning essential affordances for emotionally communicative, mediated haptic interactions was to concentrate on the interaction model. Specifically, we used an existing one-degree-of-freedom device to allow us to concentrate on the interaction on the device rather t h a n the form and function of the device itself. Moreover, we decided to concentrate on d y a d  interactions , w h i c h encompass many of our relationships and are technologically and socially easier to explore. There are many ways to design a computer mediated haptic interaction for non co-located dyads. O u r approach was to base our interactions on metaphors of haptic interaction engaged i n by dyads who are co-located i n the real world. W e wanted to find out if the metaphor used to design and explain the interaction affected the ability of dyads to communicate emotion using the haptic device. We were also interested i n the effect of the metaphor on the dyads' subjective experience. D u r i n g face-to-face interactions, people use personal space as part of their nonverbal communication. Before a dyad can engage i n real-world touch interact i o n , they must be sufficiently close together. Since our metaphors are based on face-to-face interactions, we wondered what effect providing a haptic indicator of personal space would have on performance and subjective experience. F i n a l l y , the relationship a dyad shares affects the kinds of touch interactions that are used to communicate.  We wanted to see if this was also true for  computer mediated haptic interactions. To study these three aspects of computer-mediated haptic interaction (metaphi space and relationship) we first designed several haptic interactions that varied according to metaphor and created a haptic indicator of personal space to add or remove from these interactions. T h e second phase of the research was to design and carry out an experiment to test whether metaphor, space and relationship had a n effect on ability to communicate emotion or the subjective experience of the d y a d using the interaction.  1.4  Thesis Structure  T h r o u g h o u t the remainder of this thesis we describe i n more detail our research process, results and the implications of our findings. In C h a p t e r 2, we present  an overview of related work and its relevance to the current work. In C h a p t e r 3, we discuss the design of the haptic interaction models that we created and i n C h a p t e r 4, we describe the experiment we designed to use these interaction models to test the impact of metaphor, spatial awareness and relationship on the interaction. In C h a p t e r 5, we present the result of the experiment and i n C h a p t e r 6, discuss the meanings and implications of these results. F i n a l l y , i n C h a p t e r 7, we conclude w i t h the contributions of this work and suggestions for future work.  Related Work T h i s thesis examines the effect of the haptic interaction model and the relationship shared by a d y a d on their performance and subjective experience when they are engaged i n an emotion communication task through a computer-mediated haptic interaction. W e briefly present the findings from the communications, psychology and sociology literature, which motivated and directed the current work. W e then discuss the relevant work i n the haptics literature, beginning w i t h an overview of some of the major areas of haptics research, and followed by a discussion of research i n the area of computer-mediated haptic communication.  2.1  Media  Steuer [37] encourages communications researchers to use the knowledge they have gained from s t u d y i n g existing media to make predictions about and facilitate the design of new media. Similarly, we can make use of some of this knowledge when we design and evaluate new media interactions. C o m m u n i cation events can be classified according to three factors:  the nature of the  audience, the relationship between sender(s) and receiver(s) and the m e d i u m used to communicate [17]. W e can use a similar structure to define the types of communication for which we are interested i n designing. In the current work, we are interested i n dyad interaction without an audience and i n seeing the effect of relationship on the interaction. T h e medium we are s t u d y i n g is a computer mediated haptic device but w i t h various models of interaction.  N e w communication technologies can be compared to face-to-face c o m m u n i cation or viewed as improvements to existing channels. However, users view new communication channels not as improvements on existing channels but rather as unique channels to be judged and used according to their own uses and characteristics [17]. T h u s there is value not just i n comparing communication technologies to face-to-face communication or to each other but also i n s t u d y ing the characteristics of a particular communication technology. Furthermore, since we are interested i n the use of a computer mediated haptic c o m m u n i c a t i o n device to communicate emotion, it is i m p o r t a n t to look p a r t i c u l a r l y at its suitability for this use.  2.2  Co-located Communication  In their i n t r o d u c t i o n to a compilation on nonverbal communication [39], W i e m a n n and H a r r i s o n make the following observation about nonverbal c o m m u n i cation, "Non-language messages generated by movement, unlike their language counterpart, are unique i n that they convey responsiveness on a moment-bymoment basis to the others present i n face-to-face interaction." A s technology strives to enable exchange of such real-time non-language messages, face-to-face communication often serves as a starting point. W e look to the ways people use touch and personal space i n face-to-face communication to inspire and direct the design of our haptic interaction models.  2.2.1  Touch  H a l l and K n a p p observe that " T o u c h is a crucial aspect of most h u m a n relationships. [21]" T o u c h is the first sense to develop [25] and is unique among the senses i n being reciprocal, w i t h b o t h sensing and actuation [14]. Touch can convey many meanings and the how or when of touch is largely based on culture and social-  ization [10]. Hostility, sexual interest, nurturance and dependence, affiliation [1] and the level of intimacy i n a romantic relationship [10] are some of the social messages that can be conveyed through touch. H o w touch is interpreted depends on the situation and the perceived appropriateness of the touch for the s i t u a t i o n [12]. F o r example, touch can raise anxiety i n physically or emotionally distant situations and is positively perceived if it is deemed situationally appropriate and the level of i n t i m a c y is not higher t h a n that desired by the recipient. T h e meaning of a touch is influenced by many things: part of b o d y touched or touching, length of touch, amount of pressure, movement after initial contact, the social situation i n c l u d i n g who is present, and the relationship of those i n volved i n the touching event [18]. One specific example is that the comfort level of the recipient of a touch is related to the relative i n t i m a c y of the touch and the relationship between the two. We explore the use of touch i n computer-mediated interaction, and specifically, intimacy of touch and relationship.  2.2.2  Personal Space  Interpersonal distance is one of the elements of personal expression that can only occur i n the presence of another, unlike for example, facial expressions, which can be made by an i n d i v i d u a l alone. T h e aspects of communication that require the presence of two or more individuals are often used to express attitudes and intimacy [10]. Four levels of distance were experimentally defined by H a l l [16]. These levels, in increasing order of distance, are intimate distance, personal distance, social distance and public distance. M a n y factors influence the distance at w h i c h an interaction occurs.  These factors include sex, age, culture, topic, interaction  setting, physical and personality characteristics, attitude a n d emotional orient a t i o n , and relationship [21]. Closer distances are used d u r i n g discussion of a pleasant topic t h a n d u r i n g  discussion of an unpleasant topic [21].  M o r e intimate relationships result i n  closer distances, and encounters that occur at intimate distance can be very negative if they are not intimate [10]. In general, violations of expected use of personal distances carry messages [6]. T h i s thesis examines whether the notion of personal space i n a v i r t u a l haptic space can be utilized i n emotional communication.  2.2.3  Emotion  There are two aspects of emotional expression: the physiological changes that occur simultaneously w i t h the feeling of the emotion, and the intentional communication of an emotion [10]. S t u d y i n g the intentional communication of cognitive emotion does not capture the full emotional experience but is a good first a p p r o x i m a t i o n [10]. In our experiment, we study intentional c o m m u n i c a t i o n of cognitive emotion.  2.3  Haptics Overview  Researchers have been w o r k i n g on creating devices and algorithms to facilitate h u m a n computer interaction through haptics (our sense of touch). T o date the m a j o r i t y of haptics research has focused on device design, control, rendering real world haptic sensations and the psychophysical properties of our sense of touch. A wide range of haptic devices have been developed and used. One of the most well k n o w n is the P H A N T o M [24], which is a point-contact, stylus-based, 3-degree-of-freedom ( 3 - D o F ) , force feedback device and was the first commercial haptic device. A point-contact force feedback device that operates i n a plane (2DoF) is the P a n t o g r a p h [32]. A more recent device i n this category is a low cost I D o F device designed for teaching students system dynamics and embedded control [15]. Other force feedback devices range from elaborate exoskeletons, for  example [3], to one degree of freedom force feedback knobs, for example [23]. Some examples of non-force feedback devices include passive haptic devices and tactile devices. A n example of a passive haptic device is the Tango [30], w h i c h is a physical, deformable object that senses how a user is h o l d i n g and m a n i p u l a t i n g i t . Several different types of devices have been designed to display tactile information i n c l u d i n g S T R E S S [31], w h i c h uses lateral s k i n stretch to give the illusion of vertical deformation. A greater understanding of the h u m a n haptic (touch) sense facilitates the design of devices, renderings and interactions that communicate information to humans more appropriately and effectively. H a p t i c s researchers have been exploring technology and the h u m a n haptic sense to further our knowledge about what types of haptic signals and changes i n haptic signals humans can detect. T h i s includes investigations into aspects of h u m a n perception such as the degree of pressure change required for objects of varying surface areas before the change is detectable to h u m a n subjects, and joint angle resolution [38], and the effect of spatial and t e m p o r a l differences between two vibrotactile taps on the perception of the physical distance between the two taps [9]. Sensitivity to temperature changes i n materials w i t h high t h e r m a l conductivity is an example of investigations into h u m a n perception of temperature[19]. M a c L e a n and E n r i q u e z [22] examined vibrotactile signals of v a r y i n g frequency, amplitude a n d wave shape and found that people mapped these signals to a two dimensional perceptual space. T h e effect of visual or auditory s t i m u l i on haptic perception have also been explored:  visual and audio cues have been shown to alter perceived haptic  properties. A study i n v o l v i n g different visual renderings of a physical s p r i n g , as participants compressed i t , found that the visual representation affected the perceived stiffness of the spring [36]. Sound has also been shown to influence perceived stiffness, though the effect was found not to be as strong as the effect of visual cues [11].  In summary, haptics researchers have been exploring everything related to touch and technology from device design to h u m a n haptic perception. W e are interested i n exploring how this haptic technology can be used to facilitate computer mediated person-to-person haptic interaction.  2.4  Computer Mediated  Person-to-Person  Haptic Interactions Recently, more projects are being done to begin to examine  person-to-person  computer mediated haptic interaction. T h i s research has introduced the idea of connecting people remotely through touch, looked at device design for d y a d haptic games, and employed haptics i n collaborative tasks. In this section, we discuss the research i n computer mediated person-to-person haptic interaction. M o t i v a t e d by the lack of touch interaction i n remote communication, B r a v e and Dahley designed i n T o u c h [4] [5]. T h e device consists of two sets of three rollers and communication is established through the sense of interacting w i t h a shared object that is created by b o t h sets of rollers moving as if b o t h users are acting upon t h e m . D u r i n g informal user feedback, some users thought the i n ability to convey concrete information would be a problem while others thought this was the strength of the interaction. Users suggested that this device would be most suited for intimate relationships; however, this was not systematically studied. In our research, we take some of the ideas from this work and do a more systematic, structured evaluation of the interaction and experience of the d y a d using such a haptic device.  T h i s includes looking at the effect of relationship  on the interaction and experience. A n o t h e r device built for interpersonal interaction is H a n d J i v e [13]. T h e focus of this work was on designing a device for a purely haptic two player game i n an environment where audio is inappropriate. T h e individuals involved i n the  interaction are co-located but not necessarily i n view of one another. H a n d J i v e uses two orthogonal axes so that each person controls the horizontal position of their own device, w h i c h translate into the vertical position of their partners device.  Several possible games and a possible haptic language are described  by the designers of H a n d J i v e .  T h e focus of H a n d J i v e was on device design  and play; whereas, we focus on the design of the haptic interaction model and communicating emotion. T h e " H u g Over a Distance" is a device designed to support unobtrusive, haptic interaction between intimate couples [26]. T h e prototype device consisted of a vest w i t h air pockets for receiving a v i r t u a l hug and a koala to r u b to send a v i r t u a l hug. T h e device was presented to users i n a workshop format. Users were unable to de-couple the sound of the air compressor used to inflate the pockets from the interaction, and d i d not feel that they w o u l d use it i n their d a i l y lives. However, it stimulated discussion of ideas for n e w interaction devices. A framework facilitating the use of haptic icons i n the context of instant messaging is presented i n H a p t i c I M [33].  T h e authors discuss the idea that  hapticons (iconic haptic signals) could be used as a form of non-verbal c o m m u nication cues i n the instant messaging context. T h e r e have been several projects that examine whether the presence/absence of haptics i n a v i r t u a l environment affects task performance or the sense of interacting w i t h another i n d i v i d u a l i n a collaborative task performed w i t h users located remote from each other. In one study, two people worked together to hold and move a v i r t u a l ring along a v i r t u a l wire w i t h o u t b u m p i n g into it [2]. T h e interaction was through two P H A N T o M s and two monitors connected to the same computer. A visual-only and a visual-plus-haptic condition were used. One of the two users was always the same "expert" user. T h e results showed slightly increased performance and sense of presence of the other i n the v i s u a l plus-haptic condition.  Salinas et a l .  [35] conducted another study showing  similar results involving a visual + audio condition and a visual + audio + active  haptics condition. T h e task involved using a P H A N T o M to move blocks around i n a v i r t u a l space.  Task performance and sense of presence were  somewhat  higher w i t h the active haptics. Transatlantic touch [20] is another similar study were two participants collaborated to lift a cube.  In this study, participants  were on opposite sides of the A t l a n t i c . A n increase i n sense of presence w i t h haptics was again reported. W e are interested i n examining interaction when only haptics is available for a communication task. Non-colocated collaborative application environments are often seen as lacking awareness a n d cues provided by nonverbal communication i n face-to-face situations. In a collaborative editing environment, i n which each user has their own cursor, haptics have been proposed as a way to t r y and increase awareness of where the other editor is i n the environment [29]. In this paper, several haptic interactions between cursor "avatars" are discussed.  A n observational  evaluation i n which pairs of users created U M L diagrams i n either a visual + audio or a visual + audio + haptic condition found that some users d i d make use of the haptic interactions when they were present and that there was high variability among participants i n the use of the haptic interactions. One of the haptic interactions between the two cursors is a resistance to movement of the workspace or a s m a l l v i b r a t i o n when one approaches the other.  T h e goal of  these haptic cues is to provide a haptic p r o x i m i t y sense. In this thesis, we look specifically at how the presence of a haptic p r o x i m i t y sense influences haptic interaction d u r i n g an emotion communication task. In a collaborative editing environment, i n which there is only one cursor that users share, haptic signals have been purposed as a way to provide a non-verbal mechanism for t u r n t a k i n g [7]. A n i n i t i a l study, i n w h i c h groups of four had to arrange furniture on a map according to difficult constraints, suggests that more equitable t u r n - t a k i n g may occur w i t h a haptic mechanism t h a n w i t h only a visual mechanism. O v e r a l l , participants i n the study preferred to have access to b o t h the haptic and visual mechanisms. C o m T o u c h [8] examines the use of a  simple bidirectional haptic signal combined w i t h audio, i n dyad communication. T h e idea of a haptic device that allows users to send messages through pressure and receive t h e m as a vibrotactile stimulus coupled w i t h the audio channel was tested through two experiment tasks. T h e first task was to use the device while having a conversation w i t h no view of the other.  T h e second task required  cooperation i n a survival game w i t h use of the audio channel discouraged.  In  the conversation task, participants used the device for emphasis, m i m i c r y and t u r n - t a k i n g . T h e y were not always aware of having used it i n these ways. T h e results of this research suggest that there is potential for haptic signals to play some of the roles i n remote communication that nonverbal cues provide i n faceto-face interaction. We explore the use of haptics i n emotional communication. To show the benefit of haptic interactions, many studies compare them to visual-only a n d / o r visual and audio-only interactions. O u r approach is to consider haptic-only interactions to be a unique, and potentially useful m e d i u m w o r t h s t u d y i n g i n their own right. W e are interested i n exploring how a hapticsonly device is used i n remote computer-mediated interpersonal emotional communication. A d d i n g vision a n d / o r audio creates a different interaction m e d i u m , and we are interested i n concentrating on how different haptic interaction m o d els affect the communication rather t h a n on different modalities.  Interaction D e s i g n In this thesis, we examine how the haptic interaction model a n d the relationship shared by users affects interaction between dyads given the task of communicating emotion through a haptic model. W e look at three different aspects of the interaction: i n t i m a c y of the relationship between the d y a d , the metaphor used to develop and explain the haptic interaction, and whether a haptic indicator of personal space is present i n the v i r t u a l haptic interaction. In this chapter, we describe the design of our interaction models and choice of relationship types. These models were designed to be used i n an experiment (described i n the next chapter) that manipulates metaphor, support of interpersonal spatial awareness and relationship, and measures impact on performance and subjective experience, d u r i n g an emotion communication task w i t h a haptic device. In this chapter, we first discuss the role of visual representations i n e x p l a i n ing our haptic interaction models. N e x t we briefly describe several interaction metaphors that we developed. We then discuss our rationale for choosing two of these metaphors for our experiment, and the implementation details of these two models. N e x t we present the haptic signals we designed to act as indicators of personal space, and the pilot studies we d i d to choose an indicator that people could use. T h e n we discuss the role of relationship in,the interaction. F i n a l l y , we summarize the interaction models and relationship factors that we used for the experiment.  3.1  Role of Visual Representations  O u r interaction models are designed and explained using metaphor, and this strategy makes it easier to explain the force m a p p i n g between the haptic input and output. It is not possible for a user to feel how his actions affect the output his partner feels. T h u s , to help users develop a mental model of the interaction, we also develop simple visual representations of the interaction models. B e i n g able to see what the other is doing helps a user to map the haptic s t i m u l i they feel to their own and their partners' actions. These visual representation are designed to explain the haptic m a p p i n g and are not used d u r i n g the experimental trials.  3.2  Metaphors  Initially we defined four levels of haptic metaphor i n t i m a c y that we were i n terested i n studying(Figure 3.1).  T h e least i n t i m a t e interaction between two  people involving touch is through a shared object that is touched by one person at a time, for example a soccer or p i n g pong ball. T h e next level of i n t i m a c y is through a shared object that is manipulated by b o t h people at the same time, for example a table being carried, a tug-of-war rope or a two person crosscut saw. T h e more intimate levels involve direct touch. Less intimate direct touch includes shaking hands or p a t t i n g on the shoulder. M o r e intimate direct touch includes massaging, s t r o k i n g and holding hands [18]. U s i n g this i n t i m a c y scale, we developed four interactions based on four touch metaphors that span i t : playing P i n g P o n g , c u t t i n g down a tree w i t h a crosscut saw, shaking hands a n d holding hands.  Ping Pong  T h e P i n g P o n g interaction metaphor is of two people p l a y i n g a  game w i t h a ball and two paddles. T h e haptic knobs become the paddles a n d the m o t i o n of a v i r t u a l b a l l determines the force on the paddles. E a c h person  Metaphor Intimacy Type of Touch  With a free shared object 1  Through a stiff shared object  Direct touch  Direct intimate' touch  F i g u r e 3.1: Scale of Intimacy of H a p t i c Interaction  controls the horizontal position of a P i n g P o n g paddle a n d a ball moves back and forth between the paddles. H i t t i n g the ball speeds it up a n d c r a d l i n g i t slows it down. A v i r t u a l net separates the two players. A player r u n n i n g into the net feels a strong force as i f h i t t i n g a wall.  Crosscut Saw  T h e crosscut saw interaction models a s i t u a t i o n where two  people interact simultaneously w i t h a shared object. T h e interaction metaphor is that of c u t t i n g down a tree using a large two person crosscut saw. T h e force feedback depends on the current user actions as well as the mode. T h e r e are three basic modes: 1. T h e saw is o n the ground 2. T h e saw is being h e l d / m a n i p u l a t e d by b o t h people 3. T h e saw is being picked up or p u t down. T h i s t h i r d mode actually includes multiple states involving a l l combinations of one or more of the people picking up or p u t t i n g down the saw. W h e n b o t h people are holding the saw, they feel the m o t i o n of the other through the saw. If they move the saw i n front of a v i r t u a l tree a n d move i n the  same direction together, then they saw the v i r t u a l tree. A rough v i b r a t i o n is felt when sawing. After sufficient sawing, the tree falls and a violent v i b r a t i o n is displayed on the knob as if one is feeling the ground shake as the tree hit i t .  Shaking hands  In this interaction, the metaphor of shaking hands is used.  T h e r e are three modes d u r i n g the interaction. 1. T h e approach. T h e hands are not i n contact. 2. T h e contact. T h e i n i t i a l contact stage. T h e i n i t i a l contact w i l l be influenced by the approach. 3. T h e shake. T h i s is the actual handshake. A connection is modeled between the two hands. T h e position of each motor is mapped to the m o t i o n of the h a n d . W h e n the hands are apart (approach stage), then the motor position represents the horizontal position of that h a n d i n the space; thus moving the motor towards of the other w i l l b r i n g the corresponding h a n d closer to its partner.  When  the hands are i n contact(shake stage), then motor position is m a p p e d to the rotational position of the h a n d ; moving the motor side-to-side then corresponds to shaking the h a n d up and down.  H a n d Stroke  T h i s interaction started out based on an interlocked h a n d h o l d ,  w i t h a t t r a c t i o n points corresponding to when two hands are positioned w i t h fingers intertwined. M o v i n g between these points w o u l d feel like sliding across a surface w i t h friction. People using just the sliding across mode of this interaction found it compelling and we decided to simplify the interaction by d r o p p i n g the intertwined mode, because it seemed to break up the connect that people felt w i t h the sliding action.  Furthermore, stroking is the most intimate form of  face-to-face touch [27], and thus the metaphor of two hands stroking is a more intimate metaphor t h a n that of two hands holding each other.  3.2.1  Metaphors for the User Study  After designing these four interactions, we decided to concentrate on the P i n g P o n g and H a n d Stroke interactions for our user study (described i n the next chapter). We decided to use only two interactions because we expected there might be considerable variation among users, and thus wanted a w i t h i n subjects design for the metaphor factor.  It was not realistic to test more t h a n  two metaphors i n a single session: using two metaphors allowed sufficient time for t r a i n i n g , an appropriate number of trials for each condition and for p a r t i c i pants to answer questions about their interaction experience, d u r i n g a two-hour session. T h e P i n g P o n g and H a n d Stroke metaphors were chosen for several reasons. T h e p r i m a r y reason was their location at opposites ends of our i n t i m a c y scale, thus providing the greatest difference i n intimacy. A n o t h e r reason was because the relatively straightforward m a p p i n g from the metaphor to the interaction model allowed users to quickly understand the haptic model through the metaphor.  T h e crosscut saw and handshake metaphors b o t h involved several  modes, and it was not always clear to pilot subjects w h i c h mode they were i n . It may be possible to create haptic signals that would provide a better indication of mode, but we d i d not find a way to do this without creating more complex interaction models , w h i c h could make it more difficult for users to understand the mapping between the haptic metaphor and the haptic interaction. A n o t h e r option would have been to simplify the crosscut saw and handshake interactions; for example, by restricting users to always hold the saw i n the crosscut model.  However, we felt this might create interactions that were too close to  being simple p u s h / p u l l interactions, which Fogg et A l found led users to fight for control [13].  3.2.2  Implementation Details  In the following sections, we describe the haptic and visual implementation of the two interaction models that we chose to develop further and use for our user study.  P i n g P o n g Implementation  T h i s haptic interaction is based on a physical  rendering of a model of a ball i n a horizontal plane. T h e ball has a mass and an i n i t i a l velocity. W h e n it is not i n contact w i t h either paddle it continues moving i n the space w i t h only a little friction applied to its m o t i o n . T h e p a d dle is modelled as a spring w i t h an anchor point that moves w i t h the user's knob. T h e spring constant determines how quickly the b a l l changes direction. M o v i n g the paddle after the ball has come i n contact w i t h it changes the position of the spring and thus influences how fast the b a l l leaves the paddle. For example, "swinging" your "paddle" towards the other player, after the ball has contacted, it sends the ball faster i n their direction. A faster ball w i l l hit a paddle w i t h a greater force. In the middle of the v i r t u a l space, separating the two players is a net, w h i c h is modelled as a wall. If a user crosses the net, then a constant force w i l l push them back towards their side of the net. T h e visual representation of the P i n g P o n g interaction (Figure 3.2) consists of two rectangles whose positions correspond to the position of the paddles as determined by the positions of the two knobs. In the middle of the screen, a line representing the net divides the space. A circle representing the ball moves back and forth across the screen as the ball moves around i n the space. T h e force on the ball is used to update the ball position. W h e n the ball is moving through the space between the paddles, only a small amount of force " f r i c t i o n " is applied to the m o t i o n of the ball slowing it down.  If the ball is  i n contact w i t h a paddle, the force on the b a l l is determine by how far " i n t o " the paddle the b a l l has travelled(Equation 3.1).  T h e paddle is modelled as a  point w i t h a small spring attached. T h e b a l l makes contact w i t h the paddle by  Figure 3.2: Screen shot of visual representation of P i n g P o n g metaphor  h i t t i n g the spring a n d leaves the paddle when i t leave contact w i t h the spring at rest position.  i f Xb <  —b±b  fb = ' k((x i p  k((x  p2  xi p  and  — I) — Xb) — b i b  i f Xb > x i  + I) — Xb) — b±b  if Xb < x 2 + I.  fb is the force on the ball x i is the position of the paddle i p  Xb is the position of the ball ±b is the current velocity of the ball Xb is the current acceleration of the ball  p  P  — I,  Xb >  x 2, P  (3-1)  k is the spring constant I is the spring length b is the d a m p i n g constant  T h e force on the ball is used to update the ball's acceleration (xb), velocity (ib)  and position (xb) d u r i n g the i t h time step of d u r a t i o n T. Xbi =  ±b  t  Xb  (3.2)  fbi/rrib  =  Xb T  +  Xb^!  (3.3)  =  Xb,T  +  Xi-i  (3.4)  t  T h u s , if the ball is i n contact w i t h a paddle d u r i n g a given time step, then there is a force on the corresponding knob. T h i s force is the opposite of the force applied to the ball plus a small constant, E q u a t i o n 3.5. T h e constant is used to ensure that a slow moving ball w i l l still be felt when it comes into contact w i t h the paddle. In a d d i t i o n , if the paddle has moved into/across the net, then a strong force w i l l push the paddle back towards its side of the net.  fp2  -fb  + c  if Xb >  x i -I  and  xi >  Pw,  -fb  + c + w  if Xb >  x i -I  and  xi <  Pw,  0  if Xb <  x i -I  and  xi >  Pw,  w  if Xb <  x i -I  and  xi <  Pw  p  p  p  p  p  p  p  p  -fb  + c  if Xb <  X2 +  1 and  x2 <  Pw,  -fb  + C+ W  if Xb <  X2 +  1 and  x2 >  Pw,  0  if Xb >  X2 +  1 and  x2 <  Pw,  w  if Zfc >  X2 +  1 and  x2 >  Pw  p  p  P  P  = p  p  P  P  (3.5)  (3.6)  c is a small constant added so that a slow moving ball w i l l still be felt w is the net constant p  w  is the position of the net  H a n d Stroke Implementation  If the v i r t u a l hands are i n contact then the  force felt on the knobs is determined by the relative velocity of the knobs and the area of the v i r t u a l hands that is overlapping. Metaphorically, slow movements are like the two hands being pushed more against each other and fast movements are like the hands gently and quickly brushing across each other. T h e visual representation of the H a n d Stroke interaction consisted of two rectangles whose position corresponds to the position of the hands i n the space as determined by the position of the paddles(Figure 3.3).  Figure 3.3: Screen shot of visual representation of the H a n d Stroke metaphor; (a) "hands" not i n contact, (b) "hands" i n contact  Figure 3.4 and E q u a t i o n 3.7 shows how the force varies according to the relative velocity of the haptic knobs. T o increase the richness of the interaction model at slower velocities a strong force is used at low to medium relative velocities. A s the relative velocity becomes high, the amount of force gradually decreases. A l s o as the area of contact decreases, the force decreases.  if x < Slow Velocity,  six / i f c  =  i  if Slow Velocity < x < M e d i u m V e l o c i t y ,  fbig  fbig  — S2X  ( - ) 3  7  i f M e d i u m Velocity < x < V e r y Fast Velocity.  fk! = the force based o n the relative velocities of the two knobs x = the relative velocity of the knobs fbig  = a constant force of large magnitude  51  = slope up determined to make a smooth t r a n s i t i o n to  52  = slope down determined to make a continuous t r a n s i t i o n from  fbi  g  fbi  g  to  f aii sm  T h e area of the contact between the hands also affects the force. A s the area of contact decreases, so does the force, E q u a t i o n 3.8. f k  1  ~ c * y  2  if c * y  2  and  <  f  >  /fci or  kl  y <  fk  3  0 fk  2  = the force output to the knobs  if c * y  2  y  > k.  k,  (3.8)  c = a constant y = relative distance between the v i r t u a l hands (0 = directly on top of one another) k = distance apart at which the hands are no longer touching  3.3  Personal Space Indicator  T h e models we created are based on metaphors of real w o r l d touch interactions that involve a physical space. W i t h these models we create a v i r t u a l interaction space. T h e basic models make use of distance i n this interaction space but do not explicitly give users any indication of the position of their partner relative to their own position i n the v i r t u a l space. Since the visual representations were only used for t r a i n i n g , everything i n c l u d i n g distance i n the v i r t u a l space h a d to be perceived through haptics. D u r i n g face-to-face interactions, peoples' use of personal space correlates w i t h the intimacy of the interaction and emotional orientation, among other things [21]. Furthermore, when using our preliminary interaction models w i t h b o t h the visual and haptic representation present, people seemed to make use of the concept of relative distance that the visual provided.  T h u s , we were  interested to see if a haptic display of personal space would affect either how well emotion could be communicated or the interaction experience of users, when there is no graphical feedback present. W e decided to use two values for the factor haptic personal space indicator: haptic display of personal space present or absent. T h i s required first designing a haptic means of signalling relative distance between two people i n the v i r t u a l space. W e decided to use a haptic v i b r a t i o n modulated by distance to indicate interpersonal distance and thus developed several possible haptic vibrations to indicate how far apart the two people involved i n a haptic interaction were from each other i n the v i r t u a l space. We d i d two pilot studies to test these candidate  vibrations and to identify the one that people could best use for a sense of distance i n the v i r t u a l space.  3.3.1  Initial Spatial Stimuli - Vibrations and Evaluation  In a pilot study w i t h four subjects, we tested four haptic vibrations for i n d i c a t i n g interpersonal distance i n a haptic space.  These test were done i n a v i r t u a l  haptic space that d i d not contain any other signals, but these vibrations were all designed to be subtle so that they w o u l d not overpower the interaction models that we w o u l d later add t h e m to.  Vibrations  W e created four haptic vibrations that depended on the spatial  distance between two objects i n the v i r t u a l space. A l l were sin waves that were continuously displayed and the distance affected the amplitude a n d / o r frequency of the wave. 1. Frequency increases linearly w i t h distance 2. A m p l i t u d e decreases and Frequency increases w i t h distance 3. A m p l i t u d e decreases w i t h distance 4. A m p l i t u d e decreases quadratically w i t h distance.  E x p e r i m e n t Task  In this study, the subjects held on to the haptic knob  as a haptic v i b r a t i o n was displayed according to the v i b r a t i o n being modelled and the position of a simulated second person moving towards or away from t h e m i n the space.  F o r each t r i a l , subjects were asked to indicate, w i t h the  mouse, where i n the space (along a horizontal axis) they thought the other had started and stopped and to indicate, by selecting a labelled key, what speed they thought the other was moving at: slow, m e d i u m , fast. Subjects were not t o l d the mappings as we hoped that one or more of our mappings might be intuitive to the subjects.  Subjects  Four subjects d i d 20 trials for each of the four conditions.  subjects were a l l computer science students:  three males, one female.  The The  subjects were a l l 20-29 years o l d .  Results a n d Discussion  O v e r a l l , subjects were not able to determine the  absolute or relative start, or end positions and the results were almost exactly what would be expected if subjects were p i c k i n g positons and speeds at r a n d o m . Furthermore, there was no pattern to suggest that subjects had understood our vibrations but mapped t h e m i n the reverse direction, for example m a p p i n g an increase i n amplitude to greater rather t h a n smaller distance. T h e only except i o n to these r a n d o m results was w i t h the v i b r a t i o n w i t h amplitude decreasing w i t h greater distance. In this condition, one subject was able to determine d i rection (i.e. selected end position was on the correct side of the selected start position) i n 9 0 % of the trials. Discussions w i t h subjects revealed that they were unable to successfully make a m a p p i n g of what was going on. Furthermore, when asked about the mappings every subject seemed to have been looking for something different. T h e various models that subjects were looking for or formed d u r i n g the experiment were quite varied. These models included a model similar to that of a race car approaching and receding, a m a p p i n g of intensity to velocity a n d length to distance, acceleration, and a step function. Several subjects seemed to assume that there were separate indicators of distance and speed.  T h i s may be an  artifact of the tasks that subjects were asked to do w h i c h involved separately locating the starting and ending locations of a simulated other a n d i n d i c a t i n g the speed, or it may suggest that is appropriate to separate the velocity and position cues.  Implications  T h e results of this p r e l i m i n a r y study led to two conclusions.  F i r s t , there does not appear to be a c o m m o n intuitive m a p p i n g that subjects  were expecting. It is possible that there is a haptic signal that i n t u i t i v e l y maps to distance but we d i d not find it either among the four we developed or by asking subjects directly what they would expect as a haptic signal to indicate distance. T h i s suggests that some i n i t i a l t r a i n i n g to help subjects understand our v i b r a t i o n mappings is necessary. G i v e n that several subjects seemed to be assume that there were separate indicators of distance and speed, we may want to t r y haptic vibrations that separate the velocity and position cues.  3.3.2  Final Spatial Stimuli - Vibrations and Evaluation  After our experience w i t h the first study we designed and r a n a second small study to evaluate potential haptic indicators of s p a t i a l awareness. Some changes were made to the stimuli as well as to the study design.  Vibrations  Three of the s t i m u l i used were based on the comments from the  subjects i n the first study, one w i t h discretized intervals, one w i t h separate velocity and position cues, and one w i t h separate velocity and position cues and discretized distance intervals. We tried the discrete intervals because one subject i n the first pilot had indicated that he was looking for a step function. T h e fourth stimulus was the most promising from the first study. 1. sin frequency decreases at discrete intervals w i t h distance 2. sin frequency increases w i t h velocity of partner and amplitude decreases linearly w i t h distance 3. sin frequency decreases at discrete intervals w i t h distance and amplitude increases w i t h relative velocity 4. sin amplitude decreases linearly w i t h distance  E x p e r i m e n t Task  Similarly, to the first pilot the subjects held on to the  haptic knob as a haptic v i b r a t i o n was displayed according to the v i b r a t i o n be-  ing modelled and the position of a simulated second person moving towards or away from them i n the space. For each t r i a l , subjects were asked to i n d i cate, by selecting a labelled key, where i n the space (near, mid-distance, far along a horizontal axis) they thought the other had started and stopped and to indicate, by selecting a labelled key, what speed they thought the other was moving at: slow, m e d i u m , fast. T h i s time there were five t r a i n i n g trials for each condition d u r i n g which the subjects saw a visual representation of the other moving towards or away from t h e m i n the space at the same time as the haptic stimulus was played. For each condition, after the five t r a i n i n g trials w i t h the graphic representation on, there were 20 test trials d u r i n g w h i c h only the haptic stimulus was played.  Results a n d Discussion  T h e results of this study showed that after t r a i n i n g ,  subjects were able to reliably use the two haptic vibrations where amplitude depended on distance, vibrations (2) and (4), to determine speed, direction, start a n d end position (see Table 3.1).  Table 3.1:  (1)  (2)  (3)  (4)  chance  speed  51%  74%  50%  73%  33%  direction  89%  85%  80%  90%  50%  start position  75%  75%  59%  80%  33%  end position  66%  78%  56%  79%  33%  T h e mean percentage of correct trials for each space v i b r a t i o n con-  dition.  Implications  W i t h t r a i n i n g , subjects were able to understand and use the  space indicators where the amplitude varied according to distance. W e decided to use the v i b r a t i o n w i t h only the amplitude v a r y i n g (4) to indicate space i n our metaphor implementations. O v e r a l l , subjects were the most accurate using  this model, and also it is simpler t h a n the v i b r a t i o n w i t h b o t h the amplitude and frequency varying.  3.4  Relationship  In the social sciences and communication literature, it is recognized that the relationship between individuals is an i m p o r t a n t part of what defines a c o m m u nication event [17].  In particular, relationship influences touch protocols and  the meanings associated w i t h a touch. T h e gender of those involved i n interpersonal touch is also found to influence how the touch is interpreted and received. A n e c d o t a l reports suggest that the same may be true of  computer-mediated  touch. One of the first computer-mediated person-to-person touch interactions involved two sets of rollers. E a c h user moved a hand across her set of rollers and felt the motion of her partner through the motion of the rollers. W h e n people interacted w i t h the device, they thought that it would be most appropriate for intimate relationships [4] and not as appropriate for less intimate relationships. T h i s preference suggests that relationship may influence  appropriateness  of computer-mediated haptic interaction models as it does i n face-to-face touch interactions. T o begin a more structured investigation of the effect of relationship on computer-mediated touch interaction, we varied the relationship of those interacting. W e wanted dyads whose relationship to each other varied i n intimacy. Therefore, we chose to use dyads who were either strangers or who were romantic partners. T h e intimacy levels of these dyads are d i s t i n c t l y different and thus we believe that their interactions are most likely to show relationship differences. In a l l cases, the pairs were cross gender - one male, one female - i n order to avoid any differences i n same/opposite gender interactions.  3.5  Summary  In this chapter, we described the design of the values of the factors we developed for our user study. These factors are the metaphor used to design and explain the haptic interaction model, a haptic display of interpersonal space, and the relationship of the users. T h e metaphors we considered for our user study ranged from game-like i n teractions to very intimate interpersonal touch interactions. W e chose the i n teraction models based on the least intimate metaphor ( P i n g Pong) and the most intimate metaphor ( H a n d Stroke). We designed several haptic displays of interpersonal space and based on the results of pilot studies chose a sin wave w i t h amplitude v a r y i n g according to distance. In our study, half the interaction models (one w i t h each metaphor model) had this interpersonal space indicator and the other half (one w i t h each metaphor model) do not. W e chose two relationship types (strangers and couples) based on the difference i n i n t i m a c y of these relationship types.  Emotion Communication Study 4.1  Objective  T h e objective of this study was to examine the effect of the design of an interaction model on computer mediated communication t h r o u g h a haptic link. In particular, we were interested i n the effect of the interaction design on peoples' ability to intentionally communicate cognitive emotion using the device, their patterns of c o m m u n i c a t i o n and subjective experience of the interaction. Since our focus was on the design of the v i r t u a l interaction model rather t h a n on the device itself, a simple pre-existing device was used. T h i s study also looked at the effects of the relationship between the people using the device on their ability to communicate emotion, c o m m u n i c a t i o n patterns and subjective experience using the haptic device. W e looked at the effect of relationship because we believe that the relationship between users w i l l influence how the interaction model for such devices should be designed. T h e haptic interaction model, i n the context of computer-mediated-personto-person haptic interaction, is how the computer maps the input by the users into the haptic devices to device output to those users. It should be noted that b o t h users provide input and receive output forces. T h i s m a p p i n g is entirely up to the interaction designer and may range from a direct l i n k i n g of force feedback  devices to a complex function based on the state of b o t h devices i n the past and present. T h e r e were three controlled experimental factors i n our study design: two relating to the interaction model and one relating to the participants. 1. Interaction M o d e l M e t a p h o r - 2 levels 2. H a p t i c D i s p l a y of Interpersonal Space - 2 levels 3. Relationship of P a i r s - 2 levels T h e first factor is the metaphor used to design the interaction model. T w o metaphors suggesting different levels of touch i n t i m a c y were used:  a game  metaphor ( P i n g Pong) and an intimate touch metaphor ( H a n d Stroke).  The  interpersonal space factor had two values: a haptic i n d i c a t i o n of interpersonal space i n the v i r t u a l space and no haptic i n d i c a t i o n of interpersonal space. T h e levels of relationship were stranger pairs and romantic couple pairs. These three factors are described i n detail i n previous chapter (Chapter 3). T h i s study looked at intentional communication of cognitive emotion. Specifically, i n this study users were asked to convey various emotions using the haptic interactions but there was no attempt to make the users feel the emotion that they were conveying.  Since users were intentionally conveying emotions that  they were not feeling, the methods they used to convey emotion were a cognitive a p p r o x i m a t i o n of how they might convey actual felt emotion. T h u s our results do not say a n y t h i n g about how well felt emotion could be communicated without intentional action; however, Collier suggests that l o o k i n g at emotions as being intentionally conveyed is a good starting point for e x a m i n i n g emotional communication [10].  4.2  Hypotheses  In this section, we present three sets of hypotheses that we had about the results of this experiment. T h e first set had to do w i t h the effect of our p r i m a r y factors on performance.  T h e second set had to do w i t h interaction between  these factors. T h e t h i r d set had to do w i t h the reported subjective experience of the participants.  4.2.1  Factors and Performance  O u r p r i m a r y hypotheses had to do w i t h the ability to convey emotion i n the different conditions: 1. M e t a p h o r : T h e metaphor used influences the ability of pairs to c o m m u nicate emotion through a v i r t u a l haptic space.  Specifically, pairs w o u l d  be able to successfully communicate emotion i n more trials w i t h the h a n d stroke metaphor w i l l t h a n w i t h the ping pong metaphor. 2. Space: T h e level of haptic support for awareness of interpersonal distance influences the ability of pairs to communicate emotion. Specifically, pairs would be able to successfully communicate emotion i n more trials w i t h a haptic indicator of personal space t h a n w i t h o u t a haptic indicator of personal space. 3. Relationship: T h e type of relationship the pair shares influences their ability to communicate emotion. Specifically, romantic partners w o u l d be . successful at communicating emotion i n a greater number of trials t h a n strangers. W e hypothesized that pairs would be more successful at c o m m u n i c a t i n g emotion w i t h the more intimate metaphor because it involves direct touch i n the v i r t u a l space. T h u s we thought it w o u l d be easier for users to control a n d understand how their actions affected what their partner felt.  W e hypothesized that pairs would be more successful at c o m m u n i c a t i n g emotion w i t h a haptic indicator of personal space because it would give them more ways to differentiate the emotions. W e also thought that it w o u l d possibly enable people to b r i n g spatial methods used i n face-to-face c o m m u n i c a t i o n into the v i r t u a l haptic space. F i n a l l y , we hypothesized that couples w o u l d be more successful at c o m m u nicating emotion through a haptic device t h a n strangers since they have more knowledge of their partner's communication patterns. W e believed that couples would be able to use their knowledge of how their partner communicates emotion face-to-face and through other media to understand their partner's strategies i n this new media.  4.2.2  Interaction between Factors and Performance  W e had one hypothesis about interactions between factors. 1. There w i l l be an interaction between relationship and metaphor. Specifically, there w i l l be a stronger effect for metaphor for romantic partners then for strangers. It was our hypothesis that couple w o u l d be more comfortable w i t h the H a n d Stroke metaphor and so would be better able to make use of this metaphor.  4.2.3  Subjective Experience  T h e final set of hypotheses involved the p a r t i c i p a n t s ' self reported subjective experience of the interactions. P a r t i c i p a n t s were asked w h i c h interaction they preferred, felt most connected to their partner t h r o u g h , felt most comfortable w i t h , found the easiest to use to convey emotion and found the easiest to use to perceive emotion. For each of these questions we had the following hypotheses about w h i c h interaction users would choose.  1. R o m a n t i c partners would choose the more intimate metaphor, h a n d stroke. 2. Strangers w o u l d choose the more game-like metaphor, p i n g pong. 3. B o t h romantic partners and strangers w o u l d choose interactions w i t h a haptic indicator of space. We made the hypotheses about metaphor because we thought the more game-like metaphor was more appropriate for interaction between strangers and the more intimate metaphor was more appropriate for interaction w i t h i n a couple. W e made the hypothesis that everyone would select interaction models w i t h a haptic indicator of space because it would provide t h e m w i t h a better sense of their partners intentions.  4.3  Metrics  T h e hypotheses about performance were tested based on the number of trials i n w h i c h pairs successfully communicated emotion i n each condition. T h e h y potheses about subjective experience were tested based on p a r t i c i p a n t s ' answers to a questionnaire given at the end of the experiment ( A p p e n d i x C ) .  4.4  Design  In this study, we used a m i x e d design to test our hypotheses.  To test the  hypotheses about the effect of metaphor and personal space on ability to c o m municate emotion, a within-subjects design was used.  E a c h pair of subjects  used each metaphor and space combination. T h u s each pair used four interaction conditions. Since the relationship between the i n d i v i d u a l s i n each d y a d is predetermined, the hypotheses relating to relationship where tested between subjects.  A B a l a n c e d L a t i n Squares design (Table 4.1) was used to l i m i t order effects while using a manageable number of subjects. Ordering 1  Ping Pong,  Ping Pong,  H a n d Stroke,  H a n d Stroke,  N o space  Space  N o Space  Space  Ping Pong  H a n d Stroke  Ping Pong  H a n d Stroke  Space  Space  N o Space  N o Space  H a n d Stroke  H a n d Stroke  Ping Pong  Ping Pong  Space  N o Space  Space  N o Space  H a n d Stroke  Ping Pong  H a n d Stroke  Ping Pong  N o Space  N o Space  Space  Space  2  3  4  Table 4.1: C o n d i t i o n Orderings: T h e four orderings used i n our balanced L a t i n square design. E a c h ordering was used for two stranger pairs and two romantic partners pairs  4.4.1  Physical Setup  D u r i n g the experiment the two subjects were located i n the same r o o m ; however, they were unable to see each other or interact physically(Figures 4.1, 4.3). T h i s was achieved by using a physical p a r t i t i o n . E a c h subject had a haptic device, F i g u r e 4.2, w i t h w h i c h they interacted. T h e two subjects were able to see the same monitor. T h i s monitor was used d u r i n g t r a i n i n g and to give directions d u r i n g the experiment.  T h e monitor  was used to communicate procedural information the experiment trials, but no metaphor or model information.  Figure 4.1: T h e participants sat on either side of a p a r t i t i o n and communicated emotion through a haptic device.  Figure 4.3:  A picture of two users using a haptic interaction model i n our  experiment set up.  4.4.2  Equipment  T h e haptic devices used i n this experiment were two single degree of freedom force feedback devices (one for each member of the p a i r ) . T h e input to the computer from the motors is the motor position. T h e output from the computer to the motors was a voltage, which was conveyed as a force. T h e motors used were M a x o n R E 025s w i t h 1024 c o u n t / r e v o l u t i o n optical encoders, H E D M - 5 5 0 0 . E a c h motor is configured i n direct drive and has a circular polycarbonate handle connected to its shaft. A single computer r u n n i n g W i n d o w s X P was used to r u n the haptic interactions. T h e target update rate was 2 k H z ; however, since W i n d o w s X P is not a realtime operating system there was some variation i n the signal update rate. For these interactions the noise introduced by this variation was not a problem.  4.4.3  Protocol  In this section, the experiment protocol is described. P a r t i c i p a n t s were taken into the experiment r o o m one at a time.  In the  strangers condition, one participant is instructed to come 15 minutes before the other to prevent the participants meeting i n the experiment r o o m before the experiment. T h e same instructions introducing the experiment were read by the experimenter to each pair of subjects after they had b o t h arrived ( A p p e n d i x A ) . For each of the four conditions the following protocol was followed 1. T h e condition was described using a common script. 2. T r a i n i n g ( V i s u a l and H a p t i c Interaction M o d e l s On) (a) T h e haptic interaction and the associated visual representation of the interaction was switched on and participants tried it out. (b) P a r t i c i p a n t s were t o l d the four emotions that they w o u l d be using the interaction model to convey and were instructed to t r y their ideas  for communicating the four emotions. (c) P a r t i c i p a n t One is given a list of five emotions (d) P a r t i c i p a n t One conveys next emotion on list (for up to 16 sec.) (e) P a r t i c i p a n t T w o indicates which emotion they believe One is conveying (prompted after 16 sec.) (f) Repeat (d-e) for each of the five emotions on the list. (g) Repeat (c-f) but w i t h P a r t i c i p a n t T w o as the conveyer a n d P a r t i c i pant One as the receiver 3. Trials (Haptic Interaction M o d e l O n - no visual model) (a) P a r t i c i p a n t One given a list of 10 emotions (b) P a r t i c i p a n t One conveys next emotion on list (for up to 16 sec.) (c) P a r t i c i p a n t T w o indicates which emotion they believe One is conveying (prompted after 16 sec.) (d) Repeat (b-c) for each of the 10 emotions on the list. (e) Repeat (a-d) but w i t h P a r t i c i p a n t T w o as the conveyer and P a r t i c i pant One as the decoder 4. P a r t i c i p a n t s filled out a form i n d i c a t i n g their strategies for conveying each of the four emotions(Appendix B ) . A t the end of the experiment, participants answered a questionnaire about their experience of the different conditions ( A p p e n d i x C ) . O n the questionnaire, we asked participants which interaction model they preferred, felt most connected to their partner t h r o u g h , felt most comfortable w i t h , found the easiest to use to convey emotion and found the easiest to use to perceive emotion. W e also asked them what they thought was good about each interaction model and what they w o u l d change about each interaction model.  4.4.4  Picking Emotions  T h e four emotion words used i n this study were choosen to cover the emotion space and to be distinct from one another i n this space. T h e emotions used were A n g r y , Delighted, Relaxed and U n h a p p y . These emotions were picked based on their placement i n a two dimensional affect g r i d [34]. T h e affect g r i d is a tool developed for recording subjective emotion and is based on research showing that emotions can be represented i n a two dimensional affect space. T h e grid is based on two orthogonal dimensions of valence (pleasure/displeasure) and arousal (sleepiness/arousal).  E a c h quadrant of the grid is represented by an  emotion word i n our study. Once the four emotion words were chosen, lists of these words were created for participants to convey.  Lists of 20 emotion words (5 repetitions of each  emotion) were created w i t h the words i n r a n d o m order. T h e lists were d i v i d e d i n half. T h e first half was conveyed by one participant and the second half was conveyed by the other participant. If a l l instances of an emotion were i n one half of the list, some adjustment was done so that each participant w o u l d convey each emotion at least once for each condition.  4.4.5  Recruiting Subjects  Subjects were recruited using the reservax hci@ubc web site and t h r o u g h posters posted on the University of B r i t i s h C o l u m b i a campus. For the pairs to be considered for the romantic pairs relationship, we asked that they be i n a hetersexual, romantic relationship w i t h each other for at least six months. F o r recruiting for the stranger pairs, we asked people not to intentionally sign up at the same time as someone they knew. W h e n subjects arrived we made sure that they d i d not know the other.  4.5  Analysis  4.5.1  Emotions Conveyed Statistics  In order to test our hypotheses about performance, a repeated measures A N O V A was performed on number of trials i n w h i c h emotion was successfully c o m m u nicated for each condition using S P S S . T h e participants' responses to the subjective experience questions that asked t h e m to pick an interaction model were analysed using \  4.5.2  2  tests.  Exploration  Further exploration of the d a t a provided interesting insight into how emotions were conveyed. One avenue of exploration was to look at w h i c h emotions were mistaken for one another. F o r example, it was possible that w i t h this device conveying arousal would be easier t h a n conveying valence. In this case, a table of sent versus perceived emotion w o u l d show that, i n general, when A n g r y was conveyed either A n g r y or Delighted was preceived. A n o t h e r avenue of exploration was to look at the d a t a , w h i c h we collected during the experiment, about participants' reported strategies for conveying emotion. We looked for c o m m o n strategies for each emotion.  Results In this chapter, we discuss the study and the study results.  T h e first sec-  t i o n summarizes the demographic information about participants and statistics about the sessions. N e x t we talk about how we checked for an effect of ordering. T h e remaining sections present the quantitative results of the study. T h e first deals w i t h how successfully participants could communicate emotion i n the various experimental conditions. T h e second looks at the strategies participants reported using to communicate the four emotions i n the study. In the t h i r d , we present some results related to participants' subjective reports of their interact i o n experience. T h e results reported i n these three sections are a l l aggregated results. In the final section, we divide the results into groups to have a closer look.  5.1  Participants and Study Sessions  There were a t o t a l of 16 pairs (32 individuals) that participated i n this study. H a l f (eight) of these pairs consisted of two individuals who d i d not know each other. W e refer to these pairs as "strangers". T h e other half (eight) of these pairs consisted of individuals who had been i n a romantic relationship together for at least six months. W e refer to these pairs as "couples". " P a r t n e r " and " p a i r " are used to refer to any or a l l pairs regardless of relationship. " I n d i v i d u a l " is used to refer to one person regardless of h i s / h e r relationship to h i s / h e r partner i n this study. T h e participants were between 17 and 49 years old w i t h a mean  age of 24. M o s t were university students from various areas of study, one was a high school student and the rest were employed i n different fields. T h e study sessions t y p i c a l l y lasted about two hours. T h e i n d i v i d u a l s i n the stranger pairs were instructed to arrive 15 minutes apart and d i d not meet each other beforehand. However, i n two cases the two i n d i v i d u a l s accidentally saw each other before the experiment.  Couples arrived together.  In b o t h cases,  pairs d i d not have an opportunity to discuss the interactions u n t i l the end of the study.  5.2  Ordering Effect  In the study, a B a l a n c e d L a t i n Squares design was used to mitigate any effect of condition order. T h e r e were four orderings of conditions each of w h i c h was used by four pairs (two stranger pairs and two couple pairs). Before doing our experimental analysis, we checked for an effect of order by adding order as a factor. We r a n a repeated measures A N O V A on the d a t a w i t h order as a factor and found that there was not an order effect. T h u s , for the rest of the analysis we d i d not consider order.  5.3  Ability to Communicate Emotions  In this experiment, we asked dyads to communicate emotion using a knob and a set of haptically rendered models. In this section, we look at how m a n y emotions were successfully communicated under different conditions. A l s o , we examine the patterns i n the communication and miscommunication of the four emotions: A n g r y , Delighted, R e l a x e d , U n h a p p y . Specifically, for each emotion we ask how often was it correctly perceived, and how often was it incorrectly perceived as one of the other three emotions. F u r t h e r , we look at what patterns can be found i n these miscommunications.  5.3.1  Overall Performance  P a r t i c i p a n t s successfully communicated emotion i n 54% of trials. If participants had selected from the four possible emotions at r a n d o m then the expected mean success rate would be 2 5 % . T h u s while overall participants were not able to successfully communicate emotion w i t h high accuracy they were able to successfully communicate some emotional information. L o o k i n g at Table 5.1 or F i g u r e 5.1, it is possible to see that each of the four emotions was perceived as the intended emotion more often t h a n it was perceived as any other emotion.  T h e results of the Pearson C H I - s q u a r e test  indicate that there is a statistically significant relationship (p < 0.01) between the sent a n d the received emotion. Sent E m o t i o n Perceived E m o t i o n  Angry  Delighted  Relaxed  Unhappy  Angry  62.2%  9.4%  2.5%  6.2%  Delighted  23.1%  49.2%  14.7%  15.3%  Relaxed  6.6%  24.8%  56.9%  30.8%  Unhappy  8.4%  16.3%  25.9%  47.6%  Table 5.1:  E a c h c o l u m n represents the emotion that the conveyer was t r y i n g  to convey a n d each row is the emotion that the perceiver thought was being conveyed. T h e values are the percentage of trials (out of 320 for each column) that the row emotion was received when the c o l u m n emotion was sent.  The  correct responses lie along the diagonal.  It is also possible to see that of the four emotions, A n g r y was most often identified correctly (62% of the time). R e l a x e d was next a n d was correctly identified 5 7 % of the time. U n h a p p y a n d Delighted were correctly identified i n slightly less t h a n half the trials. These results correspond w i t h participant comments at the end of the experiment. Several participants commented that A n g r y was  100.00%  75.00%  Angrv  Delighted  Relaxed  Unhappy  Sent Emotion  F i g u r e 5.1: T h i s graph shows how often each emotion the conveyers sent were perceived as the intended emotion a n d as each of the other emotions.  the easiest to convey a n d perceive a n d several indicated that U n h a p p y was the most difficult convey a n d perceive. D u r i n g post-experiment discussion several pairs also discovered that they were using opposite strategies for U n h a p p y and R e l a x e d while others said R e l a x e d was easy. W h e n the a n emotion was misidentified, it c a n been seen that for A n g r y , U n h a p p y a n d Relaxed the most common mislabel is the emotion w i t h the same arousal level: Delighted, R e l a x e d a n d U n h a p p y respectively. For example, 6 1 % of the time that A n g r y was perceived as something other t h a n A n g r y i t was identified as Delighted, w h i c h is the other high arousal emotion (33% w o u l d correspond to chance). It is interesting t o note, however, that Delighted is very rarely misidentified as A n g r y . F o r each emotion the perceived emotion has either the same arousal a n d / o r valence at a rate higher t h a n chance (75%) as c a n be seen i n Table 5.2.  Perceived E m o t i o n  Angry  Delighted  Relaxed  Unhappy  Chance  same arousal  85.3%  58.6%  82.8%  78.4%  50%  same valence  68.9%  74%  71.6%  53.8%  50%  94%  83%  98%  85%  75%  same arousal or valence  Table 5.2:  T h e percentage for each sent emotion where the received emotion  had arousal/valence i n the same direction as the target emotion, i n c l u d i n g when the target emotion was received.  5.3.2  Performance by Metaphor  There were two values of metaphor used i n this experiment ( P i n g P o n g , H a n d Stroke). W e hypothesized that the metaphor used influences the ability of pairs to communicate emotion through a v i r t u a l haptic space. Specifically, participants w i l l be able to successfully communicate emotion more frequently using the H a n d Stroke metaphor t h a n the P i n g P o n g metaphor. Table 5.3 shows that a higher number/percentage of emotions were successfully communicated w i t h the H a n d Stroke metaphor t h a n w i t h the P i n g P o n g metaphor. Metaphor  M e a n # correct out of 40 trials ± S . E .  M e a n Percent ± S . E .  Ping Pong  19.5 ± 1 . 4  48.3% ± 3.5%  H a n d Stroke  23.8 ± 2 . 1  59.5% ± 5 . 3 %  Table 5.3:  M e a n number and percent of trials d u r i n g w h i c h emotions were  successfully communicated. ( E a c h cell represents an average of 16 pairs using the given metaphor, thus it is composed of 16 * 40 = 640 observations)  O n average participants successfully communicated emotion for four more trials (11% performance difference) using the H a n d Stroke metaphor t h a n w i t h  the P i n g P o n g metaphor; this difference is statistically significant at the p < 0.05 level (p = 0.036).  5.3.3  Performance by Space Indicator  In our experiment, half the trials utilized a haptic indicator of personal space and half d i d not. W e hypothesized that the presence of a haptic indicator of personal space w o u l d affect the ability of pairs to communicate emotion; specifically, that pairs w o u l d have a higher success rate w i t h a haptic indicator of personal space. T h e d a t a do not support this hypothesis for the indicator of personal space used i n our study though other representations of space may generate different results. T h e means for the number of correctly identified emotions w i t h the two space conditions are the same (54%). O v e r a l l , adding the indicator of space we developed to our interaction models d i d not affect the ability of participants to communicate emotion and we cannot reject the null hypothesis that a n indicator of personal space does not affect ability to communicate emotion. However, an interaction effect between space and metaphor, w h i c h we discuss later i n this chapter.  5.3.4  Performance by Relationship  H a l f the pairs i n our study were strangers to each other and half were couples. W e hypothesized that the type of relationship the pair shares w o u l d affect their ability to communicate emotion.  Specifically, we hypothesized that couples  w o u l d be more successful at communicating emotion t h a n strangers. Table 5.4 shows a trend for couples to be more successful at communicating emotion, w i t h a performance difference of 1 1 % .  However, we are unable to  reject the null hypothesis that relationship does not affect performance at the p = 0.05 level since p = 0.124.  Relationship  M e a n # correct out of 80 trials  M e a n Percent ± S . E .  Strangers  38.6 ± 2 . 4  4 8 . 3 % ± 3%  Couples  47.9 ± 5 . 0  60.0% ± 6.5%  Table 5.4:  M e a n number and percent of trials d u r i n g w h i c h emotion was suc-  cessfully communicated. (Each cell represents an average of 8 pairs of the given relationship type. T h u s it is composed of 8 * 80 = 640 observations.  5.3.5  Interaction Between Experiment Factors  W e also looked for any interaction effects among our m a i n factors of metaphor, presence of a personal space indicator and relationship type.  Metaphor and Relationship  We had hypothesized that there w o u l d be a n  interaction effect between relationship and metaphor. Specifically, we h y p o t h esized that there w o u l d be a stronger effect of metaphor for romantic partners t h a n for strangers. T h e reason for this hypothesis was that we believed t h a t couples w o u l d feel more comfortable w i t h the more intimate H a n d Stroke metaphor t h a n strangers and thus w o u l d be better able to make use of i t . T h e difference i n the means between the metaphors for strangers is 4% and for couples it is 17%. However, there is not a statistically significant interaction(p = 0.182).  M e t a p h o r a n d Space  A n interaction effect was found between metaphor  and space. Specifically, adding the indicator of personal space to the P i n g P o n g metaphor interaction improved performance (9% better with) but adding the indicator of personal space to the H a n d Stroke metaphor interaction lowered performance (7.5% lower). T h i s interaction effect is statistically significant at p < = 0.05 (p = 0.006).  Relationship  Metaphor  M e a n # correct out of 40 trials ± S . E .  M e a n Percent  Strangers  Ping Pong  18.5 ± 1 . 9  4 6 . 3 % ± 4.8%  Strangers  H a n d Stroke  20.1 ± 1 . 0  50.3% ± 2.5%  Couples  Ping Pong  20.5 ± 2 . 0  51.3% ± 5 . 0 %  Couples  H a n d Stroke  27.4 ± 3.7  68.5% ± 9.3%  Table 5.5:  M e a n number and percent of emotions that were successfully c o m -  municated as a function of relationship type and metaphor. E a c h cell represents an average of 8 pairs. T h u s there were a t o t a l of 8 * 40 = 320 trials for each combination of relationship type and metaphor Metaphor  Personal Space  M e a n # correct out of 20 trials ± S . E .  M e a n Percent  Ping Pong  Indicator Off  8.9 ± 0 . 8  4 4 % ± 4%  Ping Pong  Indicator O n  10.6 ± 0 . 8  5 3 % ± 4%  H a n d Stroke  Indicator Off  12.6 ± 1.1  6 2 % ± 5.5%  H a n d Stroke  Indicator O n  11.1 ± 1 . 1  55.5% ± 5.5%  Table 5.6:  M e a n number and percent of trials i n w h i c h emotions were suc-  cessfully communicated. E a c h cell represents an average of 16 pairs, thus it is composed of 16 * 20 = 320 observations  5.4  Strategies for Communicating Emotion  In this section, we present p a r t i c i p a n t s ' reported strategies of conveying and perceiving emotion.  5.4.1  Strategies for Conveying Emotion  After each condition, participants were asked to fill i n a form ( A p p e n d i x B ) indicating w h i c h actions they used to communicate each emotion.  T h e form  contained eight actions (listed below) and r o o m for participants to add i n u p to  three a d d i t i o n a l actions. • hold still • move slowly • move quickly • hit gently • hit h a r d • move close • move far away • repeat an action W e are interested i n using the participants' reported actions to see if there are any commonalities i n strategies, where a "strategy" is defined as the subset of actions that participants reported using for conveying the four emotions i n the various experimental conditions. O v e r a l l metaphor, space and relationship type d i d not have a significant impact on the number of participants who reported using each action for a particular emotion. We also looked at aggregated results and present those here.We combine the strategies for each of the 32 participants where each participant reported four strategies for each emotion (one for each combination of metaphor and presence or absence of personal space indicator) for a t o t a l of 128 reported strategies for each emotion. Figures 5.2 and 5.3 show two views of the participants aggregated reported strategies for conveying each emotion.  F i g u r e 5.2 shows an action profile for  each emotion; where as, F i g u r e 5.3 gives an emotion profile for each action. For each metaphor and space condition, for each action participants either reported using the action or not using it for each emotion.  T h u s b o t h these  figures  show sums of the number of times an action was reported as being used for an emotion.  100.0%  I Hold Still •ft Move Slow ly Move Quickly * Hit Gently & Hit Hard Move Close '/ Move Far Repeat  Angry  Delighted  Unhappy  Figure 5.2: T h e percentage of times the actions were reported as being used organized by emotion, aggregated across a l l subjects and for a l l combinations of metaphor and personal space indicator.  For each emotion there is at least one action that is frequently (> 50%) used to convey it and not often used to convey any other emotion. Table 5.7 lists the frequently used / potentially distinguishing actions for each emotion. Move Quickly is almost always used for A n g r y and frequently used for D e lighted but rarely used for R e l a x e d or U n h a p p y and thus differentiates A n g r y and Delighted from R e l a x e d and U n h a p p y . Hit Hard is almost always used for A n g r y and rarely used for Delighted, thus differentiating A n g r y from Delighted. T h e most frequently reported action for Relaxed is Move Slowly.  T h i s action  is rarely (< 25%) used for A n g r y and Delighted but is sometimes (about 44%) used for U n h a p p y . However, Move Far away is frequently used for U n h a p p y (about 62%) and rarely used for Relaxed (< 21%).  Strategies for Conveying Emotion  100.0% a  «  90.0%  hold still  move slowly  move quickly  hit gently  hit hard  move close  move far away  repeat an action  Action  Figure 5.3:  T h e percentage of times a n actions was reported as being used  for each emotion organized by action, aggregated across a l l subjects a n d for a l l combinations of metaphor a n d personal space indicator.  5.4.2  Strategies for Perceiving Emotion  In the questionnaire at the end of the study, participants were asked about their strategy for perceiving emotion.  T h e haptic model is d y n a m i c a n d the force  feedback depends o n b o t h participants, thus when perceiving a participant could choose t o stay still a n d feel what the other was doing or could actively engage i n the interaction to determine the emotion. In the final questionnaire, participants were asked i f they moved or remained still i n order to feel the emotion being conveyed.  T h e m a j o r i t y of participants (26/32 or 80%) indicated that they  remained s t i l l . 1 9 % of these participants also indicated that it depended o n the interaction or the emotion they thought they were receiving. T h e 6 participants that d i d not report staying still indicated that it depended o n the interaction  Emotion Angry  Delighted  Relaxed  Unhappy  Action  % used for this emotion  Move Quickly  86%  Hit Hard  84%  Move Quickly  65%  M o v e Close  63%  Hit Gently  50%  M o v e Slowly  70%  Hit Gently  55%  Move Far Away  63%  Table 5.7: T h e actions reported as being used i n > 5 0 % of the strategy reports (all participants a n d conditions, 32 X 4) for each emotion.  a n d / o r the emotion they thought they were receiving. T h e relationship of the participant to h i s / h e r partner d i d not significantly affect the response to this question.  5.4.3  Strategies for Interacting  In the final questionnaire, participants were also asked about how their partner's actions influenced their actions. W h e n asked if as the perceiver they found that they w o u l d express a perceived emotion back to the conveyer, 6 3 % (20/32) of the participants indicated that they w o u l d m i r r o r the emotion that they thought was being conveyed. W h e n participants were asked i f they changed their strategy to be closer to the strategy they perceived their partner to be using, 7 2 % (23/32) said yes. A g a i n these responses d i d not depend significantly o n relationship.  5.5  Subjective aspects of Interaction Experience  C o m m u n i c a t i n g emotion is a very social task and as such we are interested not only i n participants' performance and communication strategies but also i n their experience of the interaction. In the final questionnaire, we asked participants to choose which interaction they preferred, felt most connected to their partner through, felt most comfortable w i t h , found the easiest to use to convey emotion and found the easiest to use to perceive emotion.  W e hypothesized that for  each of these questions the couples w o u l d select the more intimate metaphor interaction ( H a n d Stroke) and the strangers would select the more game like i n teraction metaphor ( P i n g Pong). We also hypothesized that participants would choose interactions w i t h an indicator of space regardless of relationship type.  5.5.1  Preference  A s hypothesized, strangers tend to prefer the P i n g P o n g interaction whereas couples prefer the H a n d Stroke interaction Table 5.8. T h e association between relationship and metaphor is statistically significant (x  2  = 6.35, p < 0.012).  T h e strength of this association is weakly positive(p/ii = 0.445, p < 0.012). Preference for presence of a personal space indictor d i d not depend on relationship or preferred metaphor and 8 1 % (26/32) participants preferred a n indicator of personal space (x  2  = 34, p < 0.001).  T h i s x value is obtained  when the expected frequencies for preference for space o n , off or same (no space preference) are the same. It should be noted that this preference for a personal space indicator may only apply to the preferred metaphor.  Relationship  Space Indicator  H a n d Stroke  Off  2  2  On  8  4  10  6  Off  0  1  On  3  11  Either  0  1  Total  3  13  13  19  Strangers  Total  Couples  Total  Table 5.8:  Ping Pong  T h e number of participants that preferred each interaction according  to metaphor and presence of a indicator of personal space, (out of a t o t a l of 32 participants) (one participant indicated that the presence or absence of a personal space indictor d i d not make a difference - i.e.  Either  on or off was  equally preferred by this users)  5.5.2  Connection  T h e feeling of connection was not associated w i t h the type of relationship(Table 5.9). M o s t participants (84%)  choose the H a n d Stroke metaphor as the one w i t h  w h i c h they felt most connected and most participants (81%) choose presence of an indicator of personal space as the space condition by w h i c h they felt most connected. These connection results are statistically significant w i t h x  2  p < 0.001 for metaphor and x  5.5.3  2  = 15,  = 34, p < 0.001 for space.  Comfort  A s hypothesized, strangers tended to feel more comfortable using the P i n g P o n g interaction whereas couples tended to feel more comfortable w i t h the H a n d Stroke interaction (see Table  5.10).  T h i s interaction is statistically  Relationship  Strangers  Couples  Space Indicator  H a n d Stroke  Off  1  2  On  3  10  Total  4  12  Off  0  2  On  1  12  Either  0  1  Total  1  15  5  27  Total  Table 5.9:  Ping Pong  T h e number of participants that felt the most connected using each  interaction according to metaphor and presence of a indicator of personal space, (out of a t o t a l of 32 participants)  significant(x  2  = 8.5, p < 0.003) and the association between relationship and  chosen most comfortable metaphor is a weak positive association(p/ii = 0.516, p < 0.003). Regardless of relationship and chosen most comfortable metaphor, people generally felt more comfortable w i t h an indicator of personal space present (78%, X  2  = 30, p < 0.001).  5.5.4  Perceived Ability to Convey  A s hypothesized, couples indicated that it was easiest to convey emotion w i t h the H a n d Stroke metaphor w i t h an haptic indicator of personal space present (see Table 5.11). However, strangers were more evenly divided on this question w i t h just less t h a n half saying it was easiest to convey emotion w i t h the P i n g P o n g metaphor and just over half saying that it was easiest to convey emotion w i t h the H a n d Stroke metaphor. However, like couples most strangers indicated that they found it easiest to convey emotion when there was a haptic indicator  Relationship  Strangers  Space Indicator  2  2  On  8  4  10  6  Off  0  2  On  2  11  Either  0  1  Total  2  14  12  20  Total  Table 5.10:  H a n d Stroke  Off  Total  Couples  Ping Pong  T h e number of participants that felt the most comfortable using  each interaction according to metaphor and presence of a indicator of personal space, (out of a t o t a l of 32 participants)  of space w i t h their chosen metaphor. T h u s we find support for our hypothesis that couples w i l l find it easiest to convey emotion w i t h the H a n d Stroke metaphor but not for our hypothesis that stranger w i l l find it easiest to convey emotion w i t h the P i n g P o n g metaphor. O v e r a l l the H a n d Stroke is more frequently perceived as being easier to convey emotion w i t h and this is statistically significant (x  2  = 15, p < 0.001).  B o t h groups indicated that they found it easiest to convey emotion using their chosen metaphor w i t h a haptic indicator of personal space(81%, x  2  = 33, p <  0.001).  5.5.5  Perceived Ability to Perceive  O v e r a l l , participants found it easier to perceive emotion using the hand stroke metaphor w i t h an indicator of personal space. These results are statistically significant w i t h x space.  2  = 8, p < 0.005 for metaphor and x  2  = 30, p < 0.001 for  Relationship  Strangers  Couples  Space Indicator  Ping Pong  H a n d Stroke  Off  0  3  On  6  6  Same  1  0  Total  7  9  Off  0  1  On  1  13  Either  0  1  Total  1  15  8  24  Total  Table 5.11: T h e number of participants that felt the best able to convey emotion using each interaction according to metaphor and presence of a indicator of personal space, (out of a total of 32 participants)  5.6  Performance and Strategies for Conveying  In this section, we examine the relationship between strategy for and success at conveying emotion. We divide the d a t a for each pair according to direction, which partner was the conveyer. W e then group the conveyers, into three groups, based on how m a n y emotions were successfully communicated when they were conveying. F i n a l l y we look at the communication patterns w i t h i n these three groups.  5.6.1  Success by Conveyer  Based on the natural distributions of success rate by conveyer (Figure 5.4) we d i v i d e d the conveyer d a t a into three groups. T w o groups contain the m a j o r i t y of conveyers. T h e first, w h i c h corresponds to the region around the first mode i n the F i g u r e 5.4, contains conveyers who successfully conveyed emotion i n 11 —  N u m b e r o f trials with Correctly Identified Emotion (/40)  F i g u r e 5.4: T h i s histogram shows how many conveyers were able to successfully convey a particular number of emotions out of 40 trials.  Relationship  Space Indicator  Strangers  Couples  Total  Table 5.12:  Ping Pong  H a n d Stroke  Off  1  4  On  5  6  Total  6  10  Off  0  On  2  12  Either  0  1  Total  2  14  8  24  .  1  T h e number of participants that felt the best able to perceive emo-  t i o n using each interaction according to metaphor and presence of a indicator of personal space, (out of a t o t a l of 32 participants)  21 trials.  T h i s is the group that successfully conveyed emotion i n a greater  number of trials t h a n chance but less t h a n the overall average.  T h e second  group, w h i c h corresponds to the second mode i n the graph, contains conveyers who successfully conveyed emotion i n 22 — 29 trials. T h i s group was successful conveying i n a greater t h a n average number of trials but i n less t h a n 7 5 % of the trials. T h e remaining group contains the three conveyers that do not fit i n either of these groups. These three conveyers successfully conveyed emotion i n at least 7 5 % of the trials. It should be noted that while we divide by conveyer, success at c o m m u n i c a t i n g the emotions also depends on the a b i l i t y of the perceiver, w h i c h remains a constant for each conveyer i n this study.  5.6.2  Below Average  W e combine the strategies of a l l the conveyers i n the below average group (17 participants) for conveying emotions (Figure  5.5).  C o m p a r i n g the strategies  of this group w i t h the strategies of a l l the conveyers (Figure  5.2) we see that  the action use profiles look similar w i t h only a few differences.  T h e biggest  difference is that this group uses move far away less often for U n h a p p y . W e also see that move close is less frequently used for Delighted and Relaxed.  II hold still ft move slowly move quickly hit gently iv hit hard move close 0 move far reoeat an action  Angry  Delighted  Unhappy  Figure 5.5: T h i s graph shows the strategies that below average conveyers reported using for each emotion.  L o o k i n g at the sent v.s. the received emotions for the below average conveyers (see figure 5.8) we see that there is more confusion overall. A n g e r is misidentified 7% more often a n d 6% is from it being mistaken as Delighted. T h e other emotions are a l l misidentified more than 10% more frequently. B o t h Delighted a n d R e l a x e d are much more frequently misidentified as U n h a p p y a n d U n h a p p y is identified as R e l a x e d more often t h a n as U n h a p p y .  5.6.3  Above Average  We combine the strategies of a l l the conveyers i n the above average group for conveying emotions(Figure 5.7). C o m p a r i n g the strategies of this group w i t h  100.0%  r  75.0%  111 Angry •J Delighted Relaxed - Unhaoov  50.0%  SsSj  25.0%  0.0%  1 • IIII Angry  1  Delighted  Relaxed  (  | Unhappy  Sent Emotion  F i g u r e 5.6: T h i s graph shows how often each emotion the below average conveyers sent were perceived as the intended emotion a n d as each of the other emotions.  the strategies of a l l the conveyers(Figure 5.3) we see that generally the actions are used i n the same ways. T h e one obvious difference; however, is that move far away, w h i c h is the differentiating action for U n h a p p y we identified earlier, is used considerably more often by the above average group. 100.0% -t—  — |  Ml hold still move slowly move quickly v hit gentry  I  1  Angry  » hit hard move close  i  ^ move far away repeat an action  11 Delighted  Relaxed  Unhappy  Figure 5.7: T h i s graph shows the strategies that above average conveyers reported using.  L o o k i n g at the sent v.s. the received emotions for this group(Figure 5.8) we see that A n g r y and R e l a x e d were b o t h correctly identified 7 0 % of the time. T h i s is a n increase over the overall average of 9% and 14% respectively. U n h a p p y was also correctly identified at a rate about 9% higher t h a n the overall average. There was only a 2% increase i n the number of times that Delighted was correctly identified. A large p o r t i o n of the difference between the overall success and the success of this group comes from U n h a p p y and Relaxed being less frequently confused.  ! 0  Ml Angry :•: Delighted Relaxed = UnhaDDV  Angry  Delighted  Relaxed  Unhappy  Sent Emotion  Figure 5.8: T h i s graph shows how often each emotion the above average conveyers sent were perceived as the intended emotion a n d as each of the other emotions.  5.6.4  Most Successful  There were three conveyers who were able to successfully convey emotion i n more t h a n 7 5 % of trials. T w o of these individuals were i n the same pair, and overall this pair was able to successfully communicate the emotions i n 8 9 % of trials. Instead of looking at the three conveyers i n this group, here we take a closer look at the performance and reported strategies and experience of this high performance pair. T h i s pair was a couple. L o o k i n g at how they d i d across conditions Table 5.13 we can see that this couple was always able to successfully communicate i n the H a n d Stroke metaphor trials. L o o k i n g specifically at the miscommunications w i t h the P i n g P o n g metaphor (results not shown) it is possible to see that without an indicator of space the miscommunications are a l l confusions between the two high arousal emotions:  A n g r y and Delighted.  Furthermore, i n one  direction these emotions were consistently switched, w h i c h suggests the two individuals i n this pair could differentiate the signals they were using but that one partner mapped the other's A n g r y to Delighted and Delighted to A n g r y . W i t h space, the mistakes are a bit more spread out though again they are again mostly i n one direction.  Table 5.13:  Metaphor  Space Indicator  Ping Pong  Off  80%  Ping Pong  On  75%  H a n d Stroke  Off  100%  H a n d Stroke  On  100%  Success R a t e  T h e "most successful" couple was able to successfully communicate  the four emotions every time w i t h the H a n d Stroke metaphor. In general, this couple had a strategy for each emotion that was different from every other emotion. T h e strategy profile were similar though not always  the same a n d were relatively consistent across conditions; although R e l a x e d and Delighted were sometimes different. D u r i n g the interview after the study this pair described their strategies as being the same across conditions for each emotion except for Delighted, w h i c h varied depending on the interaction. For all of the interaction experience questions on the final questionnaire (preference, connection, comfort, ability to convey, ability to perceive) b o t h the participants of this pair selected the H a n d Stroke metaphor w i t h space.  Chapter 6  Discussion T h e purpose of this thesis is to explore how the design of a haptic interaction for dyads and dyads' relationship affects performance i n a communicating emotions task, as well as i n d i v i d u a l ' s subjective experience. In our work, we decided to focus on (a) the metaphor used to develop and explain the v i r t u a l space of the interaction; (b) presence or absence of a haptic indicator of distance from the other i n the v i r t u a l space; and (c) the intimacy of the relationship. In the previous chapter, we presented the results from our experiment. In this chapter, we discuss the possible meanings and implications of these results. We first look i n Section 6.1 at each of the control variables: metaphor, space and relationship, and their impact on the assigned task of communicating emotions. T h e remaining sections explore implications that come out of looking at various results across conditions a n d / o r relating to the strategies used to communicate emotion. In Section 6.2, we explore the relationship between emotion, communication strategy and task performance.  In Section 6.3, we discuss the  notion of a haptic interaction model having a expressive capacity as one of its defining characteristics. N e x t we discuss how participants used real world knowledge i n the v i r t u a l space(Section 6.4). O u r experimental explorations were driven by the hypothesis that a haptic interaction model could provide a useful emotional communication channel for distributed dyads. In Section 6.5, we evaluate the validity of this assumption based on the results of our experiment. F i n a l l y , i n Section 6.6, we discuss some considerations that should be taken into account when interpreting the results of our experiment or designing similar  experiments i n the future.  6.1  How Design of an Interaction M o d e l and Relationship Influence Interaction  In this section, we discuss the implications of results (performance and subjective responses) due to the three variables of metaphor, personal space indictor and relationship.  6.1.1  Metaphor  Based on our experiment results, metaphor is a significant factor i n b o t h performance and subjective experience. We chose the two metaphors for our experiment based on their relative distance from each other i n terms of i n t i m a c y of haptic interaction. P i n g P o n g is a low intimacy interaction and H a n d Stroke is a high i n t i m a c y interaction. One other related difference between these two interaction metaphors is the level of indirection: the actions of one's partner can be felt indirectly i n the P i n g P o n g interaction and directly i n the H a n d Stroke interaction. For some of the differences found between these two metaphors it is not possible to say if it is the intimacy, the indirection or another factor that is the key factor, but results suggest that it may be a combination of i n t i m a c y and indirection, as w i l l be discussed below. T h e H a n d Stroke metaphor appears to better facilitate b o t h actual and perceived task performance.  T h e results suggest that this may be due b o t h  to the directness and to the intimate nature of this metaphor. If it was only the i n t i m a c y of the interaction affecting the performance, then we w o u l d expect strangers' actual and perceived performance to be better w i t h the less intimate P i n g P o n g metaphor; however, their actual and perceived performance is better w i t h the H a n d Stroke metaphor. If it was only the direct interaction style of the  metaphor that lead to this higher performance, we w o u l d expect the performance difference between the metaphors to be the same regardless of the relationship of the dyad. However, this is not the case as the performance increase for the H a n d Stroke metaphor is larger for couples t h a n for strangers (though this result is not statistically significant). A s well, the H a n d Stroke metaphor is perceived overall as being easier to convey a n d perceive emotion w i t h , but this perception is less c o m m o n among strangers t h a n couples (though this result is also not statistically significant). T h e metaphor also affects other subjective aspects of the interaction, such as preference, feeling of connection a n d comfort.  O v e r a l l the H a n d Stroke  metaphor is preferred a n d creates a greater sense of connection. T h i s metaphor is also found to be more comfortable for couples. These findings are likely a result of the increased i n t i m a c y of the interaction a n d a greater perception of success using this interaction. T h e greater sense of connection is found i n b o t h strangers a n d couples. W h e n split by relationship, couples prefer a n d are more comfortable w i t h this metaphor but strangers prefer a n d are more comfortable w i t h the P i n g P o n g metaphor. T h i s difference depending on relationship suggests that it is the i n t i m a c y of the metaphor influencing this aspect of the user experience.  6.1.2  Space  D u r i n g the experiments two variants of each metaphor were used: one w i t h an additional haptic v i b r a t i o n to indicate personal space a n d one w i t h o u t such a v i b r a t i o n . L o o k i n g at the overall performance results, this a d d i t i o n a l v i b r a t i o n d i d not make a difference i n performance; however, l o o k i n g more closely we see that the story is more complex. Specifically, there is an interaction effect between metaphor a n d space. W h e n used i n combination w i t h the P i n g P o n g metaphor, the a d d i t i o n a l space v i b r a t i o n increase performance but it decreases  performance when added to the H a n d Stroke metaphor. We examine this further i n Section 6.3. Despite lack of improvement i n overall performance results there is a strong argument for including such a space indicator based on the reported subjective experience. Specifically, a large a n d statistically significant m a j o r i t y of p a r t i c i pants preferred, felt more connected, a n d felt more comfortable w i t h a spatial indicator added to the metaphor that they most preferred, felt most connected w i t h a n d felt most comfortable w i t h respectively. P a r t i c i p a n t s also indicated that it was easier to convey a n d perceive emotion w i t h a spatial indicator but this could be positive or negative depending on the relative importance of perceived versus actual performance. Specifically, a higher perceived performance t h a n actual performance is likely at first to encourage continued use of the i n teraction a n d potentially leading to increased performance.  However, it may  also lead to problematic miscommunications that result i n the interaction being abandoned entirely. T h u s overall subjective results suggest that an a d d i t i o n a l haptic indicator of person space is desirable; however, performance results suggest that the spatial v i b r a t i o n used i n this study was not successful at increasing performance. It is possible that a different spatial signal would result i n better performance results. A n o t h e r possibility is that dyads could make effective use of such an indicator only i n situations that provide context or other feedback cues.  6.1.3  Relationship  T h e relationship shared by dyads interacting through a haptic device affects the interaction. In terms of performance, there were no statistically significant performance differences, but couples d i d better t h a n strangers overall. In terms of subjective experience, however, differences are significant: strangers prefer and are more comfortable w i t h the P i n g P o n g metaphor, whereas, couples prefer  and are more comfortable w i t h the H a n d Stroke metaphor. Observations d u r i n g the experiment and participants' w r i t t e n comments suggest that intimate metaphors such as the H a n d Stroke metaphor may cause discomfort if used by people who are not i n an intimate relationship. Several participants i n stranger pairs indicated on their final questionnaires that the H a n d Stroke made them uncomfortable. Furthermore, after the experiment i n at least half of the stranger pairs, one or more of the participants d i d not want to meet the person they had been interacting w i t h , though this may also be a result of other aspects of the study. In one case where participants were happy to meet, they joked about the sensation of the H a n d Stroke metaphor before moving on to discussing the interaction. Similarly, some couples  commented  that they felt like they were touching when using the H a n d Stroke metaphor. Couples have an apparent advantage over strangers when developing their strategy for communicating i n that they know their partner's real world c o m m u n i c a t i o n strategies. However, for at least one couple this actually became a hindrance. D u r i n g the discussion at the end of their session they discovered that they had been using different strategies. Specifically, he had used what he thought were the most obvious mappings based on common means of expressing emotion and she had tried to use knowledge of how he w o u l d act i n face-to-face situations, which was i n some cases different from the common means. T h u s , to successfully communicate i n this experiment, which intentionally d i d not perm i t collaboration i n preparation of a shared strategy, b o t h partners of a couples independently had to decide how much of their knowledge of each other's faceto-face interaction styles they should port to the haptic interaction.  6.2  Communicating Emotion: A c t i o n Strategies  Here, we discuss the interactions between strategy and success at conveying emotion.  R e c a l l that i n Section 5.4.1, we identified unique actions that were  commonly used for each emotion. T h e existence of c o m m o n interaction strategies, even i n absence of collaborative strategy p l a n n i n g , suggests that there may be a n a t u r a l action-to-emotion m a p p i n g that a m a j o r i t y of participants were using. W e hereafter t e r m the set of the actions commonly used to express an emotion as its action fingerprint. E a c h emotion's action fingerprint contains either a unique action or set of actions that are not c o m m o n l y used to convey the other emotions.  T h u s the aggregated strategies for each emotion suggest  that if these unique finger print actions are distinguishable i n the interaction, then using the finger print actions for each emotion w o u l d make it possible for the four emotions to be successfully communicated. L o o k i n g at the use of an emotion's finger print actions and the success rate at identifying that emotion we can see a correspondence(Table 6.1). Specifically, the more consistently an emotions action fingerprint strategy was used the more frequently this emotion was successfully communicated. In Section 5.6.1, we divided the d a t a by conveyer according to success rate of communication.  In the "below average" group, A n g r y is misidentified as  Delighted more often t h a n the overall average. T h i s is probably because move close, w h i c h is the m a i n difference between the A n g r y and Delighted action finger prints, is less frequently used for Delighted by this group. T h i s group used move far away less frequently to express U n h a p p y and b o t h Delighted and R e laxed are more frequently misidentified as U n h a p p y by this group. F u r t h e r m o r e , U n h a p p y is identified as R e l a x e d more frequently t h a n it is correctly identified. T h i s reflects the observation that there is no c o m m o n identifying characteristic action used for U n h a p p y by this group. In the above average group, the oppo-  Emotion  Angry  Delighted  Relaxed  Unhappy  Table 6.1:  Action  % C o n d i t i o n s People  % of Trials  R e p o r t e d U s i n g this  This Emotion  Fingerprint  A c t i o n for this E m o t i o n  Successfully C o m m u n i c a t e d  Move Quickly  86%  62%  Hit Hard  84%  Move Quickly  65%  M o v e Close  63%  Hit Gently  50%  M o v e Slowly  70%  Hit Gently  55%  Move Far Away  63%  49%  57%  48%  Use of finger print strategies for each emotion and rate at w h i c h  each emotion was successfully communicated.  site trends can be seen. M o s t obvious is the increase i n the use of move far away for U n h a p p y and the decrease i n the confusion between R e l a x e d a n d Unhappy. T h u s default strategies emerged for each emotion; and using these unique and distinctive strategies improved performance. In other words, using common strategies i n this new interaction m e d i u m , as using common strategies w i t h new acquaintances i n face-to-face interaction, means you are more likely to be understood. Were people to use this interaction m e d i u m extensively, we might expect people and pairs to develop their own unique strategies. T h i s is similar to the observation that i n face-to-face interaction people develop unique nonverbal interaction patterns, which replace conventional strategies as the relationship between them develops [21].  6.3  Expressive Capacity of Interaction Models  It is possible to t h i n k of a haptic interaction model as having a certain level of expressive capacity. T h e expressive capacity of an interaction relates to how much information it is possible to uniquely convey and differentiate w i t h that interaction. O u r results suggest that having an expressive capacity that matches the task leads to better  6.3.1  performance.  Metaphor and Space - Experimental  Interaction  Effect A d d i n g a haptic indicator of personal space to the P i n g P o n g interaction i n creased the average number of emotions successfully communicated;  however,  adding a haptic indicator of personal space to the H a n d Stroke interaction decreased the average number of emotions successfully communicated.  Adding  space to P i n g P o n g supplies information that is missing without a space i n d i cator. Specifically, without the space indicator it is not possible to determine if the other is close or far i n the v i r t u a l space.  O n the other h a n d , adding  an indicator of space to hand stroke adds some additional information about the haptic context but if hands stay touching then the space indicator is not providing a n y t h i n g new. W i t h o u t the space indicator i n hand stroke it is still possible to indicate close and far by touching / not touching so adding space adds granularity and redundancy i n distance information rather t h a n providing something entirely new. T h e results and participants' comments suggest that w i t h o u t a spatial i n d i cator it is difficult to find means to differentiate the four emotions w i t h the P i n g P o n g metaphor; whereas, for the H a n d Stroke metaphor, adding a spatial i n d i cator increases the expressiveness of the interaction beyond what is required of the task. People who could communicate well w i t h H a n d Stroke without space d i d not always know how to use additional information and one participant  commented she needed to t h i n k w i t h space as opposed to just feeling, without it. T h i s suggests that performance was best when the expressive capacity of the interaction was such that it was just enough for the task. E x t r a expressive capacity introduced added confusion.  6.3.2  Desired Expressiveness  E v e n though the performance is highest i n the H a n d Stroke w i t h o u t the space indicator condition, the perceived performance was higher w i t h a s p a t i a l i n d i c a tor regardless of which metaphor was chosen. A s well, most of the participants who prefer the H a n d Stroke metaphor prefer it w i t h space. It is possible that people prefer greater expressive power even when they are unable to use i t . P e r haps over time they would find ways to harness it and be able to communicate a wider range of emotional states. T h i s is quite speculative based only on our l i m i t e d results. Waisvisz suggests that musical expressiveness is related to the effort required to create the music [28].  T h u s it may be desirable for longer  t e r m use to have a haptic interaction whose expressiveness is beyond what can be learned and utilized i n a two hour experiment.  6.4  Real World Influences V i r t u a l Interaction  There are several ways i n w h i c h we observe the participants using the interaction metaphor to translate real-world, face-to-face practices into this v i r t u a l haptic interaction. T h i s is seen b o t h i n (a) the strategies people use to convey the four emotions, and (b) the differences between strangers and couples i n reported subjective experience.  6.4.1  Strategies  Distance Indicator a n d Valence  Those w h o study face-to-face interaction  have found the people tend to interact at physically closer distances i n pleasant situations, a n d farther apart i n unpleasant situations [10]. T h u s i n face-to-face interaction, distance is correlated w i t h valence. T h e reported strategies of the participants suggest that they brought a similar interaction p a t t e r n into the v i r t u a l space. Specifically, move close is one of the finger print actions for the positive valence emotion Delighted, a n d move far away is the finger print action for the negative valence emotion Unhappy.  Speed Indicates A r o u s a l  P a r t i c i p a n t s i n our study used speed of m o t i o n  to indicate the arousal level of the emotion.  F o r A n g r y a n d Delighted, move  quickly is one of the fingerprint actions. F o r R e l a x e d , move slowly is the most frequently reported action. F o r U n h a p p y , neither move quickly or move slowly are frequently used, but this is likely because participants used moving far away for U n h a p p y , thus avoiding interaction at any speed. T h u s participants conveyed arousal level i n the v i r t u a l interaction by using higher speeds (higher energy) for high arousal emotions a n d lower speeds (lower energy) to convey low arousal emotions.  C o u p l e Strategies  A n o t h e r place where there is evidence that people use  strategies from face-to-face interaction i n the v i r t u a l haptic interaction comes from couples. F o r at least three of the couples i n our study, knowledge about their partner's real world communication strategies affected the v i r t u a l interact i o n . In one couple, one partner decided to use this knowledge a n d the other decided to use what he thought of as the "default" actions, resulting i n miscommunication. W h e n asked if they thought knowing each other helped them to do the task, one person indicated that it might be easier without k n o w i n g each other since they could then use 'defaults' instead of t a i l o r i n g the interaction to  their partners communication style. A n o t h e r person indicated to the experimenter that for part of the experiment she had being interpreting her partner's signals the way that she w o u l d convey the emotions but eventually realized that he w o u l d do it differently.  6.4.2  Relationship and Metaphor  T h e final i n d i c a t i o n that people b r i n g real world experience into the v i r t u a l haptic interaction v i a the metaphor of interaction is the difference i n metaphor preference and comfort between strangers and couples.  T h e discomfort that  some strangers felt was such that they d i d not want to meet their partner a n d / o r commented on it i n the final questionnaire.  Furthermore, two of the  strangers reported finding the hand stroke interaction sexual. Couples, on the other h a n d , generally preferred and felt more comfortable w i t h the hand stroke metaphor. Some even commenting that it felt as if they were touching. A s i n real w o r l d interactions, comfort is greater when the i n t i m a c y of the interaction is appropriate for the relationship [18].  6.5  Success as an Emotional Communication Device  W e designed our experiment to examine the effect of several different haptic interaction model designs on performance and interaction experience d u r i n g an emotional communication task. U n d e r l y i n g our design is the assumption that a computer mediated haptic communication setup, such as the one used i n our experiment, could be useful for dyad emotional communication. O u r experiment was not designed to test this assumption; however, it is i m p o r t a n t to examine it, and i n this section, we evaluate this assumption based on our experimental results and observation.  E x p e r i m e n t a l Considerations  Since our experiment is not designed to test  this assumption, it is i m p o r t a n t to consider the aspects of our design that prevent us from m a k i n g a definitive c l a i m about the validity of this assumption. F i r s t l y , participants i n our experiment were asked to convey various emotions on a list, and thus they were intentionally conveying cognitive emotion and not felt emotion. It is not obvious that felt emotion w o u l d lead to the same behaviour as cognitive emotion, but as is noted by Collier [10] cognitive emotion is a good first a p p r o x i m a t i o n of felt emotion.  Secondly, the task we asked participants  to perform is more difficult t h a n using such a device i n real w o r l d interaction since there was no context w i t h i n which the haptics signals could be understood. A l s o participants d i d not have any external feedback or other communication channel available by which to co-ordinate their use of the interaction models. F i n a l l y , the task we asked participants to perform was easier t h a n using such a device i n the real world since it was l i m i t e d to four fairly distinct emotion rather t h a n the full spectrum of h u m a n emotional experience. O v e r a l l , we would expect that ability to use context to learn their partner's strategies and modify their own use of the interaction would lead to better performance i n real-world interactions.  Performance  T h e performance of participants suggests that considerable emo-  tional information is communicated. T h e overall average is slightly more t h a n twice what would be expected by chance, and almost a l l of the trials the arousal or valence of the emotion is successfully communicated. Furthermore, each emot i o n is correctly identified more often t h a n it is identified as one of the other emotions. T h e high success rates of some pairs at communicating emotion suggests that it is possible for some people to develop and use a language for c o m m u n i c a t i n g cognitive emotion w i t h our interaction models even without external feedback or context. T h e successfully communication of emotions i n at least 9 0 % of trials  i n at least one of the H a n d Stroke metaphor conditions by five couples (out of eight) suggests that given the right interaction and sufficient knowledge of the other it is possible to communicate emotional information t h r o u g h such a device.  Connection  A number of couples commented that they enjoyed the feeling of  connection they felt w i t h each other while using the interactions. Perhaps more compelling is the fact that individuals involved i n strangers pairs indicated discomfort w i t h the interaction, p a r t i c u l a r l y w i t h the H a n d Stroke metaphor. T h i s is the strongest i n d i c a t i o n that some form of true connect can be achieved w i t h this type of interaction. G i v e n that this type of connection can be achieved, creating a successful computer mediated haptic interaction is a matter of w o r k i n g out the details of such an interaction.  Reported Behaviour and Comments  A m a j o r i t y of participants reported  adapting their strategy to reflect the one they felt their partner was using. T h e performance results suggest that this adaptation was not necessarily successful; however, given more context or co-ordination it is likely that this tendency to adapt communication styles would lead to greater communication success and interest. Several participants also indicated at the end of the experiment that they felt they would have been able to do well or at least better if they had been able to come up w i t h a communication strategy together. Some participants offered comments about whether this k i n d of mediated touch interaction would be appropriate for communicating emotion. Specifically, one participant d i d not t h i n k that this type of interaction w o u l d be a very easy way to communicate emotion since "touch isn't something y o u do that often." W h e n asked if the task was difficult, another participant indicated that he d i d not find it difficult but that it was not interesting to h i m . O n the other h a n d , another participant was enthusiastic about this k i n d of interaction: " H a n d  stroke w i t h indicator was a perfect way of communicating problem / feelings without actually t a l k i n g or looking at one another.  Seems like a great new  form of communication when one person is feeling things they can't express i n words." T h u s the performance d a t a suggest that some emotional information was communicated i n our experimental context. A s well, participants' report that their strategies were d y n a m i c and comment that w i t h an a p r i o r i strategy they w o u l d have been able to do the task. These observations suggest that mediated haptics as a means for emotional communication may be usable. O n the other h a n d , only a l i m i t e d amount of emotional information was successfully c o m m u nicated i n our study and some participants commented that it was a difficult task. These observations suggest that to be appropriate for emotional c o m m u nication mediated haptics w i l l require learning and co-ordination of strategies (unless such strategies become commonplace).  6.6  Remarks on Experiment Design  In this section, we discuss some considerations that w o u l d improve future iterations using this experiment approach.  6.6.1  Training for Dyad Haptic Models  One challenge when designing haptic models for dyads interaction is developing a m a p p i n g that people can understand. If the m a p p i n g is a direct connection, then it is not too difficult to understand how my actions affect you and vice versa; although even i n this case it may not be obvious how m u c h force y o u w i l l feel i n response to my actions. Indirect interactions are harder to explain and understand.  W e t r i e d to mitigate this problem by using metaphors to  design and explain our interaction models. However, d u r i n g i n i t i a l pilots people still had trouble understanding how their actions affected the interaction since  what they felt depended on their partner's actions as well as their own.  To  enable participants on our study to develop an accurate mental model of the interaction, we presented them w i t h b o t h a haptic and visual representation of the interaction d u r i n g initial training. T h i s strategy generally seemed to help participants understand how the i n teraction worked.  However, there are two drawbacks to this approach.  One  drawback, i n this s i t u a t i o n , is that the visual feedback tends to draw attention away from the haptic cues. Different people w i l l be more or less successful at directing their attention to the haptics; people who have a harder time at paying attention to the haptics may find themselves lost when the visual feedback is no longer there. Therefore, we asked participants to close their eyes at various points, d u r i n g t r a i n i n g , to concentrate on the haptics. T h e second drawback is that people may develop strategies for communicating emotion that are more appropriate for the haptic-plus-vision interaction t h a n for the haptic-only interaction. W h e n asked what they thought about the interaction w i t h visuals versus without visuals, most participants reported that it was more difficult without the visuals, especially w i t h the p i n g pong metaphor. T w o participants indicated that it was more interesting, fun or interactive without the visuals. Despite its drawback, presenting a visual representation of the interaction seems to work as a way to help people i n i t i a l l y understand the interaction, since they can see what their partner was doing as well as see and feel their own actions and at the same time feel the haptic output this generates.  6.6.2  Motivation  One factor that can influence performance d u r i n g an experiment is m o t i v a t i o n . In a communicating emotions task, there is stronger motivation for couples to do well t h a n strangers. A couple has a vested interest i n being able to communicate  w i t h one another that strangers do not share. Since using a mediated haptic device for communicating emotion is new, couples who have trouble w i t h the task are likely to blame the device. However, being able to successfully use such a new interaction could be seen as an indication of a strong connection and thus a motivation for some couples that strangers are unlikely to have. W e are not sure what effect this difference i n motivation may have had on our results, but it is a factor that should be considered i n design of future studies.  6.7  Summary  In this chapter, we discussed the meanings and implications of our experimental results and observations. We briefly summarize the m a i n findings below. In Section 6.1, we discussed the role of our control factors i n computermediated haptic d y a d interaction. M e t a p h o r , space and relationship were a l l found to play some role i n this type of interaction: 1. T h e metaphor used to design and explain a haptic interaction model i n fluences (a) Performance  on an emotion communication task,  (b) Subjective experience of interaction 2. In a v i r t u a l haptic interaction space a haptic indicator (a) is Desirable  of personal space  from a subjective point-of-view  (b) Increases the complexity of the interaction 3. T h e relationship  of the pair engaged i n interacting t h r o u g h a haptic device  influences (a) subjective experience of a haptic interaction metaphor (b) knowledge  of partner's  communication  strategies,  i n other media,  w h i c h may help or hinder communication i n the new media  In Section 6.2, we argue that there are common strategies for communicating emotion even i n the new m e d i u m of computer-mediated haptic interaction. Furthermore, using these strategies to convey emotion leads to greater success. W e suggest that if a pair used this m e d i u m to communicate frequently w i t h each other, they may develop more unique yet effective strategies overtime. In Section 6.3, the idea of a n interaction model having an expressive capacity is presented. T h e expressive capacity of an interaction influences how difficult it is to learn and use. If the expressive capacity of an interaction model does not m a t c h that required by a communication task, it may be difficult to accomplish the task. Greater expressive capacity may complicate a simple communication task by increasing the number of possible ways of expressing a concept. However, this greater expressive capacity may still be preferred if it can be gradually incorporated into the interaction, thus allowing a greater variety of concepts or more complex concepts to be expressed. In Section 6.4, we explore how real-world interaction influences interaction i n the v i r t u a l haptic interaction space. In p a r t i c u l a r , communication strategies from face-to-face interaction are used i n the v i r t u a l haptic interaction. T h i s is seen b o t h i n the common strategies and i n the use of intimate knowledge of their partner's face-to-face communication strategies by some couples. A l s o , i n computer-mediated interaction the type of interaction that is preferred and more comfortable depends on the relationship of those engaged i n the interaction. In Section 6.5, the appropriateness of computer-mediated haptic interaction for communicating emotion is discussed. Results suggest t h a t a connection can be created, and some emotional information can be communicate t h r o u g h our computer-mediated haptic interaction models. O v e r a l l , results suggest that an appropriately designed interaction used i n context could be a successful m e d i u m for communicating emotion. F i n a l l y i n Section 6.6, we evaluate the use of a visual representation, and suggest that it is a useful technique to a i d users i n forming an accurate mental  model of the haptic interaction. We also point out that, i n experiments w i t h pairs, the type of relationship between the pair may influence their level of motivation to successfully complete the task and this should be considered i n experiment design.  Conclusion In this thesis, we presented work done to further understanding of designing for computer mediated person-to-person haptic interaction. Specifically, we discussed the interaction models and experiment we designed to test the effect of haptic metaphor, haptic support for awareness of personal space and the relationship of those using the interaction on performance and subjective experience d u r i n g a communicating emotions task. We then set out the results we obtained from this experiment and discussed the meaning and implications of these results. In this chapter, we summarize the contributions of this work a n d suggest directions for future work.  7.1  Contributions  C r e a t i o n of Effective Interaction  U s i n g metaphors and two simple one-  degree-of-freedom haptic devices we were able to create effective haptic interact i o n models for computer-mediated haptics communication of cognitive emotion. E v e n though this is a new interaction m e d i u m and the only c o m m u n i c a t i o n was though the haptic interaction, participants were still able to communicate considerable emotional content. A s well, a strong sense of connectivity was created for some participants.  Systematic A n a l y s i s  We designed and r a n an experiment as a first approx-  i m a t i o n test of the emotional expressiveness of a haptic interaction model. To  our knowledge, this is the first time a formal evaluation has been performed on the use of a haptic device to facilitate communication of affect between people. O u r methodology enables quantitative analysis of performance, and also allows users to give an informed subjective comparison of multiple interactions after having performed an emotional communications task. It provides a means of evaluating the importance of different aspects of haptic interaction design, and of comparing different versions of key components of interaction design.  Metaphor  W e designed and tested four interaction models based on two  metaphors and two levels of support for haptic awareness of personal space: no support, support using a haptic v i b r a t i o n . A n o t h e r approach to design is to create a new interaction w i t h o u t a metaphor associating it to real-world interaction; however, using metaphors can help users to develop a mental model of the interaction. W e found that the interaction metaphor significantly affects performance and subjective performance on a emotion c o m m u n i c a t i o n task. T h e metaphor also affects other aspects of the subjective experience of using the haptic device. T h u s , the choice of the metaphor used to design and explain a haptic interaction model is an essential design element for designing appropriate computer-mediated person-to-person haptic interactions.  Space  A haptic indicator of space i n a haptic interaction space is desirable.  W e found that users prefer and find a haptic indicator of space desirable even though it d i d not improve performance d u r i n g a brief two hour interaction. T h u s , if a haptic interaction model places users into a shared metaphorical space, an indicator of personal space should be present.  Relationship  W e found that the relationship of users of a computer mediated  haptic c o m m u n i c a t i o n device significantly affects the subjective experience of the users and needs to be considered when designing the interaction model. If a metaphor is used to design and explain the interaction model then it should be  socially appropriate for the expected users. Specifically, the i n t i m a c y i m p l i e d by the interaction metaphor should m a t c h the intimacy of the relationship between the users, or else discomfort can result.  C o m m u n i c a t i o n Strategies a n d M e d i a  People b r i n g their strategies for  communicating emotion i n face-to-face situations into a v i r t u a l haptic interaction. Specifically, distance and speed i n the v i r t u a l haptic space are commonly used i n ways that are similar to how they are commonly used i n real-world i n teractions. C o m m o n distinct strategies for each emotion are formed using these associations and should be supported i n the haptic interaction model.  7.2  Future Work  Spatial Awareness  W e found that subjectively, users preferred our interac-  tions w i t h the haptic indicator of personal space. However, the haptic v i b r a t i o n we used d i d not improve performance across conditions. T h u s one direction for future research is a more extensive exploration of how to provide spatial awareness to computer mediated haptic interaction spaces. T h e challenge is to create an indicator that is perceivable, understandable and yet does not overpower the rest of the interaction. A n o t h e r possible research direction is to r u n longer t e r m studies and explore how learning affects the use of a spatial indicator. A different direction is to explore m u l t i - m o d a l interactions that provide haptic interaction but utilize another m o d a l i t y to provide spatial awareness.  D e s i g n i n g for R e l a t i o n s h i p  G i v e n that the relationship between users of  a computer mediated haptic interaction affects what type of haptic interaction model is appropriate, the next challenge is to design relationship-appropriate interactions. There are two research directions that result from this challenge. One is to design interactions that are not o p t i m a l for any one type of relationship, but are generally appropriate. T h e other approach is to take a p a r t i c u l a r  relationship and design an interaction that is tailored to relationships of this type.  O f course, this approach may be taken w i t h any type of relationship  including, but not l i m i t e d to, the two used i n our study. Some possible relationships are strangers, couples, parent-child, acquaintances, colleagues, and close friends. T h i s approach is probably most interesting w i t h relationships such as parent-child, and close friends. People who share these relationships may want to engage i n remote touch interaction to communicate emotion, support or comfort, but the i n t i m a c y they share is not the same k i n d of i n t i m a c y as that shared by a romantic couple.  Field Study  B y doing a controlled experiment, we took the study of computer  mediated haptic communication devices affective communication beyond the level of informal lab testing and demos. T h e next step i n terms of understanding and evaluating computer mediated haptic communication devices is to design a haptic device and interaction model and then do a m e d i u m to long t e r m field study. Such a study could explore several questions. T h e most basic question is whether people w o u l d make use of such a device for communication. A n o t h e r question is how people would use such a device. In p a r t i c u l a r , i n what contexts w o u l d it be used and what k i n d of information/cues w o u l d it be used to convey. There are two major aspects that such a study w o u l d encompass: long-term use and n a t u r a l context.  7.3  Final Words  A telephone is a device for supporting remote verbal communication. H a p t i c s could provide a way to support remote nonverbal communication. A telephone works by encoding and decoding audio signals such that the output is a close a p p r o x i m a t i o n of the i n p u t . F o r haptic devices it is not obvious what a good encoding and decoding should look like.  M a n y possibilities for i n p u t / o u t p u t  mappings and interactions are possible w i t h computer mediated haptic interaction.  In this thesis, we initiated a systematic exploration of some of the  possibilities.  We designed interactions based on metaphors of real w o r l d i n -  teractions. It is our hope that these explorations add to the knowledge about how to design remote haptic interactions. T h e overall goal of this research is to provide a b u i l d i n g block that can eventually be used to create a useful and usable computer mediated remote haptic interaction that support meaningful interpersonal communication.  Bibliography [1] M . A r g y l e . Bodily  communication.  International U n i v e r s i t y Press, 1975.  recited from Collier85. [2] C a g a t a y Basdogan, C h i h - h a o H o , M a n d a y a m A . Srinivasan, and M e l Slater. A n experimental study on the role of touch i n shared v i r t u a l environments. Transactions  on Computer-Human  sue on human-computer  interaction  Interaction  (TOCHI):  and collaborative  virtual  Special isenvironments,  7(4):443-460, 2000. [3] M . Bergamasco, A . Benedetto, L . Bosio, L . Ferretti, G P a r r i n i n , G . M . P r i s c o , F . Salsedo, and G S a r t i n i . A n a r m exoskeleton system for teleoperation and v i r t u a l environments applications. In International on robotics and Automation,  Conference  1994.  [4] Scott B r a v e and A n d r e w Dahley. personal communication. In CHI'97,  intouch: A m e d i u m for haptic interpages 363-364, A t l a n t a , 1997. A C M  Press. [5] Scott B r a v e , H i r o s h i Ishii, and A n d r e w Dahley;  Tangible interfaces for  remote collaboration and communication. In CSCW'98,  Seattle, U S , 1998.  A C M Press. [6] Judee K B u r g o o n . N o n v e r b a l violations of expectations. In J o h n W i e m a n n and R a n d a l l H a r r i s o n , editors, Nonverbal  Interaction,  volume 11 of Sage  Annual  Research, pages 77-111. Sage P u b l i c a -  Reviews of Communication  tions, 1983. [7] A n d r e w C h a n . Designing haptic icons to support a n urgency-based t u r n t a k i n g protocol. M a s t e r ' s thesis, U n i v e r s i t y of B r i t i s h C o l u m b i a , 2004. [8] A n g e l a C h a n g , Sile O ' M o d h r a i n , R o b Jacob, E r i c G u n t h e r , and H i r o s h i Ishii. C o m t o u c h : design of a vibrotactile communication device. In Symposium  on Designing  processes, practices,  Interactive  Systems.  Designing  interactive  systems:  methods and techniques, pages 312-320, L o n d o n , E n g -  l a n d , 2002. A C M Press. [9] R . W Cholewiak. T h e perception of tactile distance: Influences of b o d y site, space, and time. Perception, [10] G a r y Collier. Emotional  28, 1999.  Expression.  Lawrence E r l b a u m Associates, 1985.  [11] D . E . D i F r a n c o , G . L . Beauregard, and M . A . Srinivasan. T h e effect of auditory cues on the haptic perception of stiffness i n v i r t u a l environments. In Symposium  on Haptic Interfaces for  Virtual Environments  and Teleop-  erator Systems, 1997. [12] J . D . F i s h e r , M . R y t t i n g , and R H e l i n . H a n d s touching hands: Affective and evaluative effects of an interpersonal touch. Sociometry,  39, 1976.  [13] B J Fogg, Lawrence D . C u t l e r , P e r r y A r n o l d , and C h r i s E i s b a c h . H a n d j i v e : a device for interpersonal haptic entertainment. In SIGCHI,  page 5764,  L o s Angeles, C a l i f o r n i a , U S , 1998. A C M Press. [14] L . K . F r a n k .  Tactile communication.  Genetic  Psychology  Monographs,  56:209-255, 1957. [15] B . R . Gillespie, M . B . Hoffman, and J Freudenberg. H a p t i c interface for hands-on i n s t r u c t i o n i n system dynamics and embedded control. In Sympo-  sium on Haptic Interfaces for  Virtual Environment  and Teleoperator  Sys-  tems, 2003. [16] E . T . H a l l . The Hidden Dimension. [17] Peter Hartley. Interpersonal  Double-day, G a r d e n C i t y , N J , 1966.  Communication.  Routledge, 1993.  [18] R i c h a r d Heslin and T a r i A l p e r . Touch: A b o n d i n g gesture. In J o h n W i e m a n n and R a n d a l l H a r r i s o n , editors, Nonverbal Sage Annual Reviews of Communication  Interaction,  Research,  volume 11 of  pages 4 7 - 7 5 . Sage P u b -  lications, 1983. [19] A . Jones and M . Berris. T h e psychophysics of temperature perception and thermal-interface design. In Symposium Environments  on Haptic  Interfaces for  Virtual  and Teleoperator Systems, L o s A l a m i t o s , C A , 2002.  [20] J u n g K i m , K i m H y u n , B o o n Tay, M a n i v a n n a n M u n i y a n d i , J o e l J o r d a n , Jesper M o r t e n s e n , M a n u e l O l i v e i r a , M e l Slater, and M a n d a y a m Srinivasan. Transatlantic touch: A study of haptic collaboration over long distance. Presence, 13:328-337, 2004. [21] M a r k K n a p p and J u d i t h H a l l . Nonverbal action.  Communication  in Human  Inter-  T h o m s o n L e a r n i n g , Inc, U S A , 5th edition, 2002.  [22] K a r o n M a c L e a n and M a r i o E n r i q u e z . Perceptual design of haptic icons. In Eurohaptics,  D u b l i n , Ireland, 2003.  [23] K a r o n E . M a c l e a n and J . B . Roderick. A l a d d i n : E x p l o r i n g language w i t h a haptic door knob. Technical R e p o r t 199-058, Interval, 1999 1999. [24] T . H . Massie and J . K . Salisbury. T h e p h a n t o m haptic interface: a device for probing v i r t u a l objects. In Third Annual Symposium for  Virtual Environment  on Haptic  and Teleoperator Systems, 1994.  Interfaces  [25] A . Montague.  Touching:  The human significance  of the skin.  Columbia  U n i v e r s i t y Press, 1971. [26] F l o r i a n M u e l l e r , F r a n k Vetere, M a r t i n G i b b s , Jesper Kjeldskov, and Steve H o w a r d . H u g over a distance. In CHI 2005. A C M , 2005. [27] T N g u y e n , R Heslin, and M . L N g u y e n . T h e meanings of touch: sex differences. Journal of Communications,  25:92-103, 1975.  [28] Sally Jane N o r m a n , M i c h e l Waisvisz, and R y a n J o e l .  Touchstone,  h t t p : / / w w w . c r a c k l e . o r g / t o u c h . h t m , 1998. [29] Ian Oakley, Stephen Brewster, and P h i l i p G r a y , editors.  Communicating  with Feeling, volume 2058/2001 of Lecture Note in Computer Science. 2000. [30] Dinesh P a i , E r i c V a n D e r L o o , S u b a r n a S a d h u k n h a n , and P a u l K r y . T h e tango: a tangible tangoreceptive whole-hand h u m a n interface. In Haptics,  World  pages 141-147, P i s a , Italy, 2005. I E E E .  [31] Jerome Pasquero and V i n c e n t H a y w a r d . Stress: A practical tactile display system w i t h one millimeter spatial resolution and 700 hz refresh rate. In Proceedings Eurohaptics  2003, D u b l i n Ireland, 2003.  [32] C . R a m s t e i n and V . H a y w a r d . T h e pantograph: A large workspace haptic device for a m u l t i - m o d a l human-computer interaction. In CHI'94, ence on Human Factors  in Computer  Systems ACM/SIGCHI  Confer-  Companion,  pages 57-58, 1994. [33] A . F . Rover and H . A . van Essen. H i m : A framework for haptic instant messaging. In CHI 2004, pages 1313-1316, V i e n n a , A u s t r i a , 2004. A C M Press. [34] James R u s s e l l , A n n a Weiss, and G e r a l d Mendelsohn. Affect grid: A singlei t e m scale of pleasure and arousal. Journal chology, 57, 1989.  of Personality  and Social  Psy-  [35] E v a - L o t t a Salinas, K i r r e R a s s m u s - G r o h n , and C a l l e Sjostrom. S u p p o r t i n g presence i n collaborative environments by haptic force feedback. Transactions  on Computer-Human  Interaction  (To  CHI),  ACM  7(4):461-476,  2000. [36] M . A . Srinivasan, G . L . Beauregard, and D . B r o c k . T h e impact of visual information on the haptic perception of stiffness i n v i r t u a l environments. In Symposium  on Haptic Interfaces for  Virtual Environments  and Teleop-  erator Systems, A t l a n t a , G A , 1996. [37] J o n a t h a n Steuer. Defining v i r t u a l reality: Dimensions determining telepresence. Journal  of Communication,  42(4):73-93, 1992.  [38] H . Z. T a n , M . A . Srinivasan, B . E b e r m a n , and B C h e n g . H u m a n factors for the design of force-reflecting haptic interafaces. In Symposium Interfaces for  Virtual Environment  on  Haptic  and Teleoperator System, Chicago, I L ,  1994. A S M E / I M E C E . [39] J o h n W i e m a n n and R a n d a l l H a r r i s o n . Nonverabl cations, 1983.  Interaction.  Sage P u b l i -  Experiment Script These are the instructions read to pairs d u r i n g the experiment.  A.l  Instructions  Welcome. In this study we are exploring using a haptic knob (point to knobs) to communicate emotion.  T h i s study is exploring various ways of interacting w i t h  someone using this type of device. Lets t r y one of these interactions. ( E x p l a i n interaction metaphor and follow metaphor script.) Ok.  N o w notice that there are 4 emotion keys on the k e y b o a r d i n front of  you. D u r i n g the next part of the study you w i l l be asked to use the interaction to convey these emotions. Afterwards you w i l l be asked to fill i n this table to i n dicate how y o u conveyed each emotion. Lets look at it now. (give questionnaire, read directions, ask if they have any questions, take away) (start interaction again) O k . N o w notice that there are 4 emotion keys on the keyboard i n front of you.  D u r i n g the next part of the study y o u w i l l be  asked to use the interaction to convey these emotions. For each emotion t h i n k about how you might do this and t r y it out. Please dont t a l k as you do this since we are t r y i n g to concentrate on the touch, (allow time for this) D u r i n g the actual task I w i l l t u r n off the visual representation and only the touch w i l l be available for communicating emotion. T r y your ideas for communicating the emotions again but this time t r y closing your eyes.  N o w we w i l l move on to the communicating emotions task. I w i l l give each of you i n t u r n a list of emotions to convey to the other. F o r the list of 5 emotions I give you now the interaction w i l l be just as what weve used so far. Y o u w i l l be able to see and feel a representation of what you are doing. N e x t I w i l l give you each i n t u r n a list of 10 emotions to convey and for these there w i l l be no visual representation so everything w i l l be conveyed and received t h r o u g h touch. T h e purpose of these first 5 is to learn what the feels mean so concentrate on the feeling and t r y closing your eyes when y o u have a good idea of what is going on. (give list and talk to h e r / h i m ) W h e n prompted please hit the emotion key for the next emotion on the list. Please wait for the prompt to hit the emotion key. (talk to other) Y o u r task is to figure out what emotion he/she is t r y i n g to convey using the device. W h e n you t h i n k you know the emotion that he/she is t r y i n g to convey, hit the appropriate emotion key. (start interaction and give emotion list)  A.2  Interaction Metaphor Script  P i n g P o n g M e t a p h o r explanation: T h i s interaction is based on the metaphor of a game of p i n g pong.  With  your knob you can move back and forth towards the net and hit a ball. Y o u w i l l be able to feel the ball hit you. I l l t u r n on the interaction now and then explain further,  (turn on interaction) Feel the ball h i t t i n g y o u .  T r y moving  forward when the ball hits y o u -this w i l l speed the ball up. T r y moving back as the ball hits you -this w i l l slow the ball down. N o w t r y doing the same t h i n g w i t h your eyes closed and concentrating on the feeling of the ball. Notice that if you stay still the ball slows down slightly. In the middle of the space there is a net. Represented on the screen by a white line. W h e n you r u n into this line it is like h i t t i n g a wall. Y o u (point to one) t r y r u n n i n g into the net and t r y to feel the difference from when the ball hits you. N o w t r y w i t h your eyes closed.  (Now say the same t h i n g to the other) H a n d Stroke M e t a p h o r explanation: T h i s interaction is based on the metaphor of a hands touching and r u b b i n g across each other. W i t h your knob you can move your hand around i n the space. Y o u w i l l be able to feel the other person moving as well as yourself moving. I l l t u r n on the interaction now and then explain further, (turn on interaction) T r y moving the hands back and forth across each other. T r y going slowly. G o i n g slowly creates a stronger feeling like pressing harder. T r y going quickly. G o i n g quickly feels more like brushing by.  N o w t r y the same t h i n g w i t h your eyes  closed and concentrating on the feeling. Space Indicator explanations: W e are now going to add an indicator of how far apart y o u are from each other to the interaction that we just tried. W h a t this means is that there w i l l be an a d d i t i o n a l haptic v i b r a t i o n that you feel. W h e n you are close together (for hand stroke add but not touching) then the v i b r a t i o n w i l l be stronger. A s you get farther away it w i l l get weaker, (for ping pong add  note the strength  of the v i b r a t i o n does not depend on how far from the net you are. It depends on how far from each other you are.)  E m o t i o n Strategy F o r m T h i s is a copy of the form participants' used after each metaphor/space condition to indicate the strategy they used to communicate each emotion. Instructions: Please put an X i n the boxes to indicate w h i c h actions you used to convey each emotion. For example: T h e X s i n the first c o l u m n w o u l d mean that hit h a r d , move far away and move slowly were used to convey Jealous. P l a c e as many X s for each emotion as is necessary to explain what y o u d i d . If the same action was used for more t h a n one emotion, put an X i n that action row for each of the emotions for w h i c h it was used. If y o u used actions that are not i n this table please add t h e m to the b o t t o m and put i n X s to show w h i c h emotions you used these other actions for.  Appendix  B. Emotion  Jealous hold still move slowly move quickly hit gently hit h a r d move close move far away repeat a n action  Anger  Strategy  Delighted  Form  Relaxed  101  Unhappy  F i n a l Questionaire This is the questionnaire given to each participant at the end of the experiment. Final  Questionnaire  For each question please c i r c l e your  response.  1. The i n t e r a c t i o n metaphor I enjoyed the most was  a) Ping Pong  Hand Stroke  b) With Without an i n d i c a t o r of how close we were  2.  I p r e f e r r e d t h i s i n t e r a c t i o n because  3.  I f e l t most connected using the f o l l o w i n g metaphor  a) Ping Pong Hand Stroke  b) With Without an i n d i c a t o r of how close we were  4.  I f e l t most comfortable u s i n g the f o l l o w i n g metaphor  a) Ping Pong Hand Stroke  c) With Without an i n d i c a t o r of how c l o s e we were  5.  I t was e a s i e s t t o convey emotion u s i n g the f o l l o w i n g metaphor  a) Ping Pong Hand Stroke  b) With Without an i n d i c a t o r of how c l o s e we were  6. I t was e a s i e s t t o perceive emotion u s i n g the f o l l o w i n g metaphor  a) Ping Pong Hand Stroke  b) With Without an i n d i c a t o r of how close we were  7. When you were the p e r c e i v e r d i d you remain s t i l l or d i d you move i n order t o f e e l what was being conveyed?  Moved Stayed S t i l l Depended on:  I n t e r a c t i o n Emotion  8 . When a c t i n g as the p e r c e i v e r , d i d you f i n d y o u r s e l f expressing the same emotion as you b e l i e v e d the conveyor was expressing?  YES NO  9. D i d you change your s t r a t e g y f o r expressing emotions t o be more l i k e your partners s t r a t e g y as you learned what they were doing?  YES NO  For each i n t e r a c t i o n please comment on what was good ( i . e made i t enjoyable, made i t e a s i e r t o convey/perceive emotion) about the i n t e r a c t i o n and what you would change.  10. Ping Pong without i n d i c a t o r of how close we were Enjoyed:  Would change:  11. Ping Pong Enjoyed:  Would change:  with i n d i c a t o r of how close we were  12. Hand Stroke Enjoyed:  without i n d i c a t o r of how c l o s e we were :  Would change:  13. Hand Stroke  with i n d i c a t o r of how close we were  Enjoyed:  Would change:  14. How d i d you f i n d i n t e r a c t i n g with only the h a p t i c r e p r e s e n t a t i o n compared to i n t e r a c t i n g with the h a p t i c and v i s u a l r e p r e s e n t a t i o n s ?  Other Comments:  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0051512/manifest

Comment

Related Items