Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Participant experience studies of interactive artworks : an investigation of laboratory-based methods.. 2008

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata


ubc_2008_spring_deutscher_meghan.pdf [ 2.22MB ]
JSON: 1.0066354.json
JSON-LD: 1.0066354+ld.json
RDF/XML (Pretty): 1.0066354.xml
RDF/JSON: 1.0066354+rdf.json
Turtle: 1.0066354+rdf-turtle.txt
N-Triples: 1.0066354+rdf-ntriples.txt

Full Text

Participant Experience Studies of Interactive Artworks An Investigation of Laboratory-Based Methods Used to Study Echology by Meghan Catherine Deutscher BaSc., The University of Regina, 2003 A THESIS SUBMITTED IN PARTIAL FULFILMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master of Applied Science in The Faculty of Graduate Studies (Electrical and Computer Engineering) The University Of British Columbia (Vancouver, Canada) April, 2008 c© Meghan Catherine Deutscher 2008 Abstract We investigate the use of laboratory-based methodology for studying par- ticipant experience of interactive artworks. The investigation is motivated by two goals: to inform the HCI practitioner of the role of participant ex- perience studies in artwork from the perspective of the artist and to inform the artist of how laboratory-based methodology can contribute to the refine- ment of their techniques and aesthetics. In this thesis three main purposes for participant experience studies in the artist’s process are derived from the roles of artist, art object, and participants in an interactive artwork. Common characteristics of participant experience studies are reviewed, with three cases unique in their use of more formal methodologies examined in detail. This thesis builds on a foundation set forth by these three cases in an investigation of orientation media: media such as text, images, or video designed by the artist to convey supplemental information to participants and thus selectively influence their understanding of different elements in an interactive artwork. Orientation media in the form of instructions cards is used in a study of the interactive sound and video installation piece, Echology. The orientation media is successful in revealing elements of the artwork that, given explicit instructions or not, still cause confusion among participants. A general review of the study methodology is also provided. This includes observations of changes in participant behaviour due to their roles as subjects in a study and implications these changes have on using formal methodologies for studying participant experience. ii Table of Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Problem Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.1 The Interactive Arts . . . . . . . . . . . . . . . . . . . 3 1.1.2 Collaborative Sound and Video Installation Art . . . 6 1.1.3 Participant Experience Studies . . . . . . . . . . . . . 7 1.1.4 Interactive Arts and HCI . . . . . . . . . . . . . . . . 8 1.1.5 Differences Between HCI and the Interactive Arts . . 9 1.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 11 1.3 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.4 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2 Interactive Art and the HCI Process . . . . . . . . . . . . . 16 2.1 Defining Interactive Art . . . . . . . . . . . . . . . . . . . . . 17 2.1.1 Defining “Interactive” . . . . . . . . . . . . . . . . . . 17 2.1.2 Background in Participatory Art . . . . . . . . . . . . 18 iii Table of Contents 2.1.3 The Interactive Art System . . . . . . . . . . . . . . . 19 2.2 Contributors to the Aesthetics of Interactive Artworks . . . . 22 2.2.1 Roles of the Art Object . . . . . . . . . . . . . . . . . 23 2.2.2 Roles of the Participant . . . . . . . . . . . . . . . . . 27 2.2.3 Roles of the Artist . . . . . . . . . . . . . . . . . . . . 28 2.3 Participant Experience Studies of Interactive Artworks . . . 29 2.3.1 Purposes of Participant Experience Studies . . . . . . 29 2.3.2 Characteristics of Participant Experience Studies . . 30 Goals . . . . . . . . . . . . . . . . . . . . . . 30 Methods . . . . . . . . . . . . . . . . . . . . 32 Settings and Subjects . . . . . . . . . . . . . 33 Form of Results . . . . . . . . . . . . . . . . 33 2.4 The HCI Process . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.4.1 User-Centered and Interaction Design . . . . . . . . . 35 2.4.2 Parallels in HCI and Artistic Processes . . . . . . . . 38 2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3 Formal Participant Experience Studies . . . . . . . . . . . . 40 3.1 Case One: The Influencing Machine . . . . . . . . . . . . . . 41 3.1.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.1.2 Artwork . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.1.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.1.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.2 Case Two: Iamascope . . . . . . . . . . . . . . . . . . . . . . 43 3.2.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.2.2 Artwork . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.2.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.2.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.3 Case Three: Façade . . . . . . . . . . . . . . . . . . . . . . . 46 3.3.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.3.2 Artwork . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.3.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 48 iv Table of Contents 3.4 Contributing to Existing Study Methodology . . . . . . . . . 52 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4 Echology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.1 An Interactive Sound and Video Installation . . . . . . . . . 55 4.1.1 Echology Summary . . . . . . . . . . . . . . . . . . . 55 4.1.2 Artistic Motivations . . . . . . . . . . . . . . . . . . . 56 4.2 Echology Design Guidelines . . . . . . . . . . . . . . . . . . . 56 4.2.1 Collaboration, Simplicity, and Approachability . . . . 56 Collaboration . . . . . . . . . . . . . . . . . 57 Simplicity . . . . . . . . . . . . . . . . . . . 58 Approachability . . . . . . . . . . . . . . . . 59 4.2.2 Designing for the Open Media Environment . . . . . 60 4.3 The Echology System Description . . . . . . . . . . . . . . . 60 4.3.1 Beluga Video and Motion Capture . . . . . . . . . . . 63 4.3.2 Soundscape Spatialization . . . . . . . . . . . . . . . 64 4.3.3 Participant Control of Sound Spatialization . . . . . . 69 4.3.4 Graphical Feedback . . . . . . . . . . . . . . . . . . . 70 4.3.5 Phidget Interface . . . . . . . . . . . . . . . . . . . . 71 4.3.6 Lighting . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.3.7 Interaction Table . . . . . . . . . . . . . . . . . . . . 72 4.3.8 Musical Elements . . . . . . . . . . . . . . . . . . . . 74 4.4 Desired Echology Experience . . . . . . . . . . . . . . . . . . 75 4.4.1 Participant Experience Summary . . . . . . . . . . . 75 4.4.2 Connecting with the Belugas . . . . . . . . . . . . . . 76 4.4.3 Participant and Art Object Roles . . . . . . . . . . . 77 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5 Studying Participant Experience of Echology . . . . . . . . 79 5.1 Informal Observations at NIME . . . . . . . . . . . . . . . . 80 5.1.1 Summary of Observations and Feedback Received . . 80 5.1.2 Questions Remaining . . . . . . . . . . . . . . . . . . 81 5.2 Laboratory-Based Experience Study . . . . . . . . . . . . . . 83 v Table of Contents 5.2.1 Orientation Media . . . . . . . . . . . . . . . . . . . . 84 5.2.2 Participants . . . . . . . . . . . . . . . . . . . . . . . 84 5.2.3 Setting . . . . . . . . . . . . . . . . . . . . . . . . . . 85 5.2.4 Study Procedures . . . . . . . . . . . . . . . . . . . . 86 5.2.5 Data Collected . . . . . . . . . . . . . . . . . . . . . . 88 5.3 Study Results . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5.3.1 Visual and Auditory Enjoyment . . . . . . . . . . . . 89 5.3.2 Understanding of Echology Interactivity . . . . . . . 90 5.3.3 Connecting with the Belugas . . . . . . . . . . . . . . 92 5.3.4 Collaboration . . . . . . . . . . . . . . . . . . . . . . 93 5.3.5 Participant Behaviour Tendencies . . . . . . . . . . . 93 5.4 Interpretation of Study Results . . . . . . . . . . . . . . . . . 97 5.5 Discussion of Study Methodology . . . . . . . . . . . . . . . 99 5.5.1 Participant Behaviour as a Study Subject . . . . . . . 99 5.5.2 Orientation Media Review . . . . . . . . . . . . . . . 102 5.5.3 Benefit of In-depth Data Collection . . . . . . . . . . 102 5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.1 Thesis Problems . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.2 Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . 105 6.3 Laboratory-Based Methods in the Artistic Process . . . . . . 107 6.4 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.4.1 Study Methods Based on Artwork Characteristics . . 109 6.4.2 Aesthetics of Study Result Presentation . . . . . . . . 110 6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Appendices A Study Material . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 A.1 Participant Recruitment Notice . . . . . . . . . . . . . . . . 123 vi Table of Contents A.2 Orientation Media: Echology Instruction Cards . . . . . . . 124 A.3 Orientation Media: Echology Introduction Cards . . . . . . . 125 A.4 Interview Question Guide . . . . . . . . . . . . . . . . . . . . 126 A.5 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . 127 B Study Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 B.1 General Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 B.2 Questionnaire Results . . . . . . . . . . . . . . . . . . . . . . 131 B.3 Interview Transcript Excerpts . . . . . . . . . . . . . . . . . 134 C BREB Certificate of Approval . . . . . . . . . . . . . . . . . . 185 vii List of Tables 1.1 Differences in Studies Conducted in HCI and in the Interac- tive Arts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3.1 Summary of Three Cases of Formal Participant Experience Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.1 Design Guidelines for Echology as a Collaborative Musical Experience . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.2 Echology’s Roles as an Art Object . . . . . . . . . . . . . . . 77 4.3 Participant Roles in the Echology Art System . . . . . . . . . 78 B.1 Study Participant Information . . . . . . . . . . . . . . . . . 128 B.2 Echology Experience Durations . . . . . . . . . . . . . . . . . 130 B.3 Questionnaire Results: Free Input Responses . . . . . . . . . 131 B.4 Participant Statements Related to Echology as a Game and Echology as Art . . . . . . . . . . . . . . . . . . . . . . . . . . 134 B.5 Participant Statements Related to Understanding of Echol- ogy Interactivity . . . . . . . . . . . . . . . . . . . . . . . . . 139 B.6 Participant Statements Related to Sensation and Enjoyment . 152 B.7 Participant Statements Related to Connecting with the Bel- ugas and the Live Webcam Feed . . . . . . . . . . . . . . . . 161 B.8 Participant Statements Related to Participant Expectations and Technique . . . . . . . . . . . . . . . . . . . . . . . . . . 169 B.9 Participant Statements Related to Breakdowns . . . . . . . . 181 B.10 Participant Statements Related to Collaboration . . . . . . . 184 viii List of Figures 1.1 Thesis Scope Diagram . . . . . . . . . . . . . . . . . . . . . . 5 1.2 Thesis Problem Scope . . . . . . . . . . . . . . . . . . . . . . 14 1.3 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.1 The Art System . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.2 Art Classification Systems . . . . . . . . . . . . . . . . . . . . 21 2.3 Roles in the Art System . . . . . . . . . . . . . . . . . . . . . 23 2.4 Parallels in HCI and Artistic Processes . . . . . . . . . . . . . 37 3.1 Iamascope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.2 Façade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4.1 Echology Sound and Video Installation . . . . . . . . . . . . . 55 4.2 Echology System Diagram: Physical Components . . . . . . . 61 4.3 Echology System Diagram: Inputs and Outputs . . . . . . . . 62 4.4 Layering of Beluga Webcam View . . . . . . . . . . . . . . . . 64 4.5 Steps of Video and Motion Processing . . . . . . . . . . . . . 65 4.6 Two-dimensional Sound Plane . . . . . . . . . . . . . . . . . . 67 4.7 Eight Reflection Points . . . . . . . . . . . . . . . . . . . . . . 67 4.8 Reflection Point Direction . . . . . . . . . . . . . . . . . . . . 68 4.9 Sound Path Example . . . . . . . . . . . . . . . . . . . . . . . 69 4.10 Interaction Table Graphics . . . . . . . . . . . . . . . . . . . 70 4.11 Suspended Light . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.12 Interaction Tabletop . . . . . . . . . . . . . . . . . . . . . . . 73 4.13 Interaction Table with Screen Removed . . . . . . . . . . . . 73 5.1 Echology Shown at NIME . . . . . . . . . . . . . . . . . . . . 80 ix List of Figures 5.2 Study Session Type Matrix . . . . . . . . . . . . . . . . . . . 83 5.3 Study Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 C.1 Certificate of Approval from the Behavioural Research Ethics Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 x Acknowledgements To my supervisor Sid, for your knowledge and unique perspective. I always leave our discussions inspired, motivated, and seeing the world framed in a new way. This work could not have been completed without your great patience and support. To friends at HCT, particularly Tony and Ian, for all your help and for every word of advice. To roommates Eric and Hendrik, for your knowledge and for taking time to listen to me stress and worry about my thesis. To best friends Kim and Jemma, I would not have had the confidence to finish this work without your encouragement. Finally, to my mother Donna, sister Sarah, and brothers Luke and Mark, I am so incredibly fortunate to have you all as my loving family. Thank you for believing in me. xi Dedication This is for dad – In his memory I am able to live life fully and with passion. xii Chapter 1 Introduction Advancements in digital technologies and modern computing have brought about rapid development in new technologies and computing practices. Two fields have emerged in this development: Human Computer Interaction (HCI) and the Interactive Arts. Both of these fields are concerned with the relationship between people and computing technologies and accord- ingly, there are cases of collaboration between HCI practitioners and artists. Furthermore, there are people who work as both HCI practitioner and artist. Recently this collaboration has become more prominent as computers have become ubiquitous and interactive art has moved into the mainstream. As the study of aesthetic, emotional, and social factors becomes vital to creating good computing experiences, HCI practitioners seek creative inspi- ration and understanding from the interactive arts. In the interactive arts, artists no longer engage through the “wow factor” of new technology, but through good interface design. Furthermore, an artwork’s success is not based on what the artist has created alone, but is also highly dependent on the interaction that develops between the artwork and its audience. The audience becomes a participant. Thus to evaluate, understand, and refine art techniques, artists engage in studies of participant experience, often appropriating user study methods used by HCI practitioners. This thesis explores participant experience studies of interactive artworks with two motivations: • To inform the HCI practitioner of the purpose for conducting partici- pant experience studies. • To inform the artist of how formal laboratory-based methodology can contribute to the refinement of their techniques. 1 Chapter 1. Introduction Here ”formal” is used to refer to methods that use planned procedures in controlled settings in an attempt to maintain consistency among participant experiences studied and data collected. The work begins with a review of art theory, articulating the roles of artists, participants and “art objects”, or the artifacts created by artists, in the production and life of an artwork. Three main purposes for conducting participant experience studies are derived from these roles: to develop un- derstanding of interaction and evaluate art object design, to feedback what was learned into new versions of an art object, thus creating art objects that indirectly learn and evolve, and to create a record of interaction that becomes part of the artwork’s content. Common study characteristics are summarized, showing that they often use informal methods, relying on the artist’s recollection of observations and participant comments made during a work’s exhibition. Following is a review of the HCI user-centered or inter- action design process. This process is paralleled with the iterative artistic process of design, implementation, and study. Similarities and differences between user studies conducted in HCI and participant experience studies are noted. The above provides a foundation from which three cases of formal par- ticipant experience studies in the HCI literature can be reviewed in detail. These cases exemplify the benefits of methods for in-depth data collection and controlled experiment settings. They provide a foundation to which this work makes its contributions with an investigation of methodology for laboratory-based participant experience studies through a study of the in- teractive sound and video installation piece, Echology. Echology’s artistic motivations, design, and desired participant experi- ence are presented and then the design of a laboratory-based participant experience study using orientation media is detailed. Orientation me- dia is media such as text, images, or video designed by the artist and given to a study participant before or during their experience with an interactive artwork. The media is designed to convey additional information that will influence a participant’s understanding of different artwork elements in order to distinguish the effects understanding of those elements have on interaction 2 Chapter 1. Introduction experience. Experiences of participants given orientation media, in this case in the form of instructions and descriptions of Echology’s functions, are compared against a control group given no media. The orientation media is successful in revealing elements of the artwork that, given explicit instructions or not, still cause confusion among participants. Finally, study methods used are reviewed. Observations of changes in participant behaviour due to their roles as subjects in a study and implica- tions these changes have on using formal methodologies for studying partic- ipant experience are presented. 1.1 Problem Scope 1.1.1 The Interactive Arts This research considers user study methodology for evaluating and under- standing participant experience of interactive artworks. In Chapter 2 defi- nitions of this art form are discussed more deeply but prior to this, a short introduction is due. Generally, interactive art is considered “art intended for the viewer’s direct participation” [3]. When experiencing interactive art, viewers or spectators are no longer only passive audience; they are partici- pants. They become engaged in “a dialog between the piece and the partic- ipant; specifically, the participant has ‘agency’ (the ability to act upon) the piece. . . ” From the 1960s onward, interactivity in art was pushed forward by an increased interest in participatory artwork [41] as well as developments in electrical and digital technologies. The new aesthetics of interactive art are concerned with the relationship between the participant and the artwork. A quintessential example from Myron Krueger, the pioneer of interactive art [81], is his 1970 installation VIDEOPLACE [61]. VIDEOPLACE is an artificial reality environment con- sisting of a video camera and other sensors to track a participant’s movement in front of a projection screen. Video displayed on the projection screen re- sponds to participant gestures with over 50 compositions. The compositions manipulate participant silhouettes and create graphical worlds around them 3 Chapter 1. Introduction containing virtual objects and organisms. The participant in turn can inter- act with movement of their whole body as well as by directly manipulating elements of the compositions. In VIDEOPLACE, artwork content is only revealed through participant interaction. Full expression relies on partici- pant understanding of how they can interact and furthermore, engagement in an exploration of the content space. Content may also depend on the participant to the extent that the par- ticipant becomes creator of the content. In “The king has. . . ” (2005) Krister Olsson and Takashi Kawashima explore the types of secrets people are will- ing to disclose given an anonymous communication channel [75]. They post their own secrets in public, inviting strangers to return a secret by sending a short message (SMS) from their mobile phone. Secrets received are printed on wooden blocks with red ink to be displayed in gallery spaces. Engaging the public outside of the gallery space is another common theme in the interactive arts. Many works are found on the Internet or have been taken to the city street, activating participation among a more diverse audience. Blast Theory‘s “Can You See Me Now?” (2001 - present) is a game of chase played between a set of runners equipped with handheld computers in a city’s streets, and people participating online from their computer [15]. Online participants are virtually dropped in random locations around the city and the runners use their handhelds to track and chase them in the real world. Online participants navigate their virtual position and can eavesdrop on a stream of audio from the runners. A participant is “caught” if a runner gets within 5 meters of their virtual position. “Can You See Me Now?” may seem to be crossing a line between art and game but many interactive artworks do indeed involve participant play and game-like elements. In these works, an artist may be concerned with how to create game-like interaction experiences where participants still critically reflect on the meaning of the artwork. This thesis focuses on developing a general understanding of the contrib- utors to aesthetics in interactive art and accordingly, the purpose of studying interaction and participant experience. The above examples illustrate a se- lection of issues concerning participant experiences. They do not cover the 4 Chapter 1. Introduction breadth of concepts explored and technologies used in the past 30 to 40 years and there is no attempt to provide a classification for these in this research. Instead, theoretical definitions of interactive art are used to understand its aesthetics and within these, purposes for conducting participant experience studies. Research is confined to a study of electronic- and computer-based inter- active artworks. Digital technologies have become somewhat synonymous with interactive art. However, it is important to note that interactive art- works need not be computer- or electronics-based. Also, not all computer or electronic artworks are interactive. To clarify, the Venn diagram in Fig- ure 1.1 shows where there is an overlapping area of interactive artworks that are computer or electronic-based or both. com p u ter artel ec tr on ic  a rt interactive art research area Figure 1.1: Venn diagram highlighting the scope of this research in the cross-sections of electronic- and computer-based interactive artwork. 5 Chapter 1. Introduction 1.1.2 Collaborative Sound and Video Installation Art An investigation of methodologies for studying participant experience is con- ducted through studies of “Echology” [37] – an interactive art piece designed and develop by the author of this thesis in collaboration with Sidney Fels, Sachiyo Takahashi, and Reynald Hoskinson. On top of being a computer- based interactive artwork, Echology is also a collaborative sound art- work and an installation piece. It is important to note these last two classifications, as they have a significant impact on study goals specific to Echology. Studies of collaborative sound artworks are concerned with the social behaviours of participants simultaneously and cooperatively engaged in the manipulation of sound and musical elements. Behaviours of interest become ones such as cooperation, competition, territoriality, and ability for strangers to communicate and connect (see for examples [20], [51], [63], [77]). In studies of artwork that allow participants to create or manipulate musical elements, artists are interested in the balance between supporting the beginner and at the same time, engaging the virtuoso. There is a heavy focus on expression as the way in which a participant becomes engaged with the artwork. Simple, easy to learn interfaces provide limited expression and more complex interfaces provide many dimensions of expression [100]. Blaine and Fels [20] discuss this in their work on collaborative musical ex- periences for novices and have identified many of the qualities to organize multi-person interactive sound installations. They provide guidelines based on these qualities to make sound interaction successful. These guidelines were integrated into the design of Echology and will be discussed in Section 4.2. Studies of installation pieces are generally concerned with the audience experience of a space or an environment. They are interested in the extent that the artwork overcomes the space in which it is installed (such as a gallery room) and immerses the viewer in the synthesized environment [30], [38]. An installation “addresses the viewer directly as a literal presence in the space” [19]. This contrasts with artwork providing a specific focal 6 Chapter 1. Introduction point for the audience such as a framed painting or sculpture and possibly targeting a specific sense, as well as with artwork that does not rely on the context of the viewer such as art delivered over the Internet. 1.1.3 Participant Experience Studies Interactive artists frequently study participants interacting with their art- works to develop an understanding of the aesthetics of interaction and to refine their techniques. For example, artist David Rokeby speaks of his exhi- bitions as “a public research laboratory” where his “ideas about interaction and experience are tested, affirmed, or shot down” [85]. These participant experience studies usually include accounts of how people were observed ap- proaching and interacting with a piece. This act of observing makes sense considering that the product of an interactive artwork is no longer a static, fixed “thing” that an audience views or experiences. The product includes the interaction created between audience and some system and the concerns of the artist involve the aesthetics of this interaction. Thus “experimenta- tion” [65] cannot be done without the activity of the participant [69]. Determinants of success and study goals are difficult to define as they vary depending on the artwork. They may or may not involve participant performance, pleasure, understanding, and so on. For now the generalization that artists have the ultimate goal of creating experiences that “engage” the participant will be made. Muller and Edmonds summarize the approaches to these studies as oc- curring in two areas: in museums and galleries and in HCI. In the former, the focus is mainly on educational and interpretive uses of interactive technolo- gies [69]. They do not necessarily consider the significance of the interaction in the context of an artwork, nor do they inform the development of an artist’s techniques. In the latter, there are a number of studies that draw together HCI and the interactive arts. HCI methodology is appropriated to study and evaluate participant experience of interactive artworks and the results are presented to both the HCI and the arts communities. In both cases, studies are often based on informal methods and observa- 7 Chapter 1. Introduction tion of interaction in the artwork’s natural context. The characteristics of these studies will be further detailed in Section 2.4. In contrast to common informal methods are a number of cases where more formal methodologies have been used. Of note is research conducted in the Interface Ecology Lab at Texas A&M University [4] and in the Creativity and Cognition Studios at the University of Technology in Sydney, Australia [1]. Three specific studies will be presented as case studies in Chapter 3 ([29], [54], [59]). These stud- ies have shown that laboratory-based studies and in-depth data collection methods are valuable for identifying and documenting bad artwork design, participant behaviours and participant techniques. This research builds on their formal methodology by considering methods for studying variations in participant understanding of interaction in a controlled setting. 1.1.4 Interactive Arts and HCI There has been a shift in HCI “from a concern with functional efficiency to a broader interest in the overall user experience; this has brought greater prominence to affect, engagement and pleasure.” [33]. This shift makes sense given the injection of computing practices into our daily life fabric. No longer limited to work-related applications, computing technologies have penetrated entertainment, social, home, urban, transportation, and other daily life spheres. Research in HCI has broadened [87] to consider elements of user experience such as aesthetics (see for example [9], [45], [52] [55], [79]), affect (see for example [22], [35], [73], [80]), and play ([21], [66]). Consequently, HCI looks to the interactive arts, among other fields, for user experience knowledge and creative inspiration. A recent CHI workshop on methodology and evaluation shared between HCI and the new media arts discusses collaborations as occurring in HCI explorations of “artful interac- tion, tools to support creative engagement, and constructed user experiences meant to elicit creative responses.” [8] In the interactive arts, new interac- tion concepts can be attempted and understanding of relationships between humans and computers beyond those we are familiar with today can be de- veloped [84]. The artist can explore areas not on the standard HCI research 8 Chapter 1. Introduction agenda and confront challenges for the future of computing practices. Also, while the interactive arts can draw from HCI study methods, HCI practitioners can draw from artists’ study results to develop understanding of how participant or user experience may be investigated. User experience design goals are highly subjective [82] and can involve multiple interpre- tations [89] thus it can be difficult to evaluate to a single, “good” design. Interactive artists deal directly with multiple interpretations of participants [84] and the manner in which they absorb and use participant interpretations can be quite informative to the HCI practitioner. 1.1.5 Differences Between HCI and the Interactive Arts Artists and HCI practitioners can use similar methods but there will be differences such as study goals, results expected, experiences being studied, and view of errors (see Table 1.1). If HCI practitioners are to look towards studies conducted in the interactive arts for understanding and inspiration, an understanding of artistic goals and the significance of methods chosen is needed to interpret the work. User studies and evaluation in traditional HCI have been generally con- ducted with goals of increasing a user’s comfort and efficiency with a com- puting technology. On the other hand, an artist is more interested in in- creasing a participant’s engagement with an art piece and this may or may not depend on comfort and efficiency. As discussed above, there has been increasing importance placed on engagement in HCI research, but there are still differences in study purposes in terms of how the outcomes are used and valued. While HCI evaluation studies are part of an iterative design pro- cess strictly to inform design and redesign, Höök et al. note “when artists do user studies, they are likely to see the user study itself as part of the communication through the artwork and another opportunity to shape the ‘message’ of the artwork” [54]. These differences in goals translate to differences in study results. In HCI, objective and quantifiable data is highly valued as reliable whereas artists are usually more interested in the subjective components of interac- 9 Chapter 1. Introduction tion and the insights that can be gleamed. While an HCI practitioner may also be interested in subjective elements, qualitative data tends to be av- eraged, generalizing a study’s results to make them repeatable and broadly applicable. Artists are less likely to generalize because they are interested in rich narratives of interaction and peculiarities that arise in single instances of behaviour [33] [54]. Table 1.1: Differences between characteristics of user stud- ies conducted by HCI practitioners and those of participant experience studies conducted by artists. User Studies in HCI Participant Experience Studies in the Interactive Arts Goals Increasing usability; inform iterative design. Increasing participant engage- ment; further communication of the artist’s message to par- ticipants. Results Objective, quantifiable data highly valued; data averaged. Subjective, qualitative data highly valued; interested in each unique case of participant interaction. Experience Task models; predefined objectives and paths to reach them. Experience models; Experi- ence objectives not necessarily defined. Errors Errors and negative experiences are undesirable; user should have a clear mental model of system. Errors and frustration can be intentional; artists may use techniques of illusion or ambi- guity. 10 Chapter 1. Introduction User experiences in HCI are usually based on tasks and have predefined objectives and paths to reach them [16]. For most artworks, there may only be vaguely predefined experience objectives and ways to get there, if any at all [33]. An artwork could perhaps be considered the proposition of a topic or question and the study is an artist curiously waiting for a variety of responses and opinions. Finally, in traditional HCI, user errors and negative experiences are to be avoided and a design goal is for the user to have a clear mental model of the system they are interacting with. On the other hand, it may be acceptable for a participant of an interactive artwork to encounter errors and frustration in their experience and artists quite often rely on techniques of illusion or ambiguity [46] and not clear mental models. 1.2 Problem Statement There are great benefits in collaboration for both sides, but for collaboration to be successful, some of the above conflicts need to be addressed. This work aims to do this by providing a basis for each side to better understand the goals and methods of the other. More specifically, this thesis addresses three problems: 1. We do not fully understand the aesthetics of interactive art- works and accordingly, the purposes for conducting partic- ipant experience studies. HCI practitioners may have difficulties accepting the qualitative and interpretive nature of an artist’s study methods. This is likely due to limited understanding of the artistic motivations behind conducting studies. Yet for collaborations between HCI and the arts to occur, a common language and understanding of artistic intentions and visions needs to be established [26]. 2. We do not fully understanding how laboratory-based user study methodology can be beneficial for the development of interactive art experiences. Many artists may question the ne- cessity for conducting participant experience studies at all. Then, if 11 Chapter 1. Introduction they do include this in their practice, they may strongly believe that laboratory-based studies take a person away from the natural con- text of interaction and as a result, they only use informal methods for studying participant experience. While there are cases of more formal methodology used ([54], [29], [59], [97]), there is still room for more commentary on the benefits and implications of using these methods. A strong role for formal studies conducted in conjunction with informal studies in the artist’s design process needs to be established. 3. We have no investigation of methods to compare experiences of participants with varying levels of understanding of art- work elements in a controlled laboratory setting. Interactive artworks require some exploration on part of the participant and the artifacts created by the artist should support this exploration. If a participant is unable to develop understanding of functional and ex- pressive elements of an artwork, their experience may be limited. For example, on approaching an artwork that uses body gestures as input, participants may not immediately understand how their movements affect the artwork. Or they may see the effects of their movements but they may be unable to attribute expressive meaning to their ac- tions and the artwork’s reactions. Thus, key concerns of the artist are: does the artwork properly support development of participant under- standing? How much is the experience actually based on “figuring the artwork out”? And how does specific understanding of different artwork functional and expressive elements affect experience? These questions suggest the need for methodology that would allow artists to selectively influence participant understanding of specific artwork elements in a controlled setting. 12 Chapter 1. Introduction 1.3 Goals To address the above problems, the work in this thesis is approached with the following goals: 1. To make the argument that participant studies are not only a neces- sary activity in the artistic process for refining technique. They are also part of an artwork’s content and contribute to the aesthetics of interactive art. 2. To review three cases of participant experience studies conducted using more formal methodology appropriated from HCI, highlighting the benefits accrued. 3. To conduct and comment on a study of participant experience with the interactive artwork Echology in a controlled laboratory setting using pre-experience information or orientation media to selectively influence participant understanding of different artwork elements. 1.4 Overview The research area and problems taken on in this thesis are summarized in Figure 1.2 and an overview of work conducted is given in Figure 1.3. The next chapter begins by reviewing background information in interactive art theory and characteristics of participant experience studies. These studies as part of an iterative artistic process are compared with user studies as an activity in the HCI process. Chapter 3 reviews three specific cases of participant experience studies conducted using more formal methodology appropriated from HCI. These studies provide a foundation for this thesis to build upon with an investigation of methodology through a study of the artwork Echology. Chapter 4 introduces Echology, discussing artistic motivations, the Echology system and the intended participant experience. Chapter 5 summarizes an initial informal study of Echology then details the design and execution of a laboratory-based study using orientation media. Study results are discussed and a review of the study experience is 13 Chapter 1. Introduction provided. Chapter 6 concludes with a review of the problems taken on in this work and the contributions made. Areas for future work in the development of methods for studying participant experience of interactive artworks are suggested. The appendices include material used in the Echology study and resulting study data. Human Computer Interaction Interactive Arts Participant Experience Studies In fo rm at io n  a n d  In sp ir at io n S tu d y  M et h o d s Research Area Problems Identified Root Problem: Lack of understanding of the role for participant experience studies in the artistic process. Problem: It is difficult for HCI researchers to interpret and apply results of participant experience studies. Problem: Artist unaware of benefits of laboratory-based studies using more formal scientific methodology. Figure 1.2: Research area and problems identified. 14 Chapter 1. Introduction Drawing from interactive art literature, make a case for the role of participant experience studies in interactive art aesthetics and the artist's process. Summarize common characteristics of participant experience studies. Compare artistic process and studies to HCI design process and user studies. (Chapter 2) Present three detailed cases of more formal laboratory- based studies that use HCI methodology. (Chapter 3) Propose, design and conduct laboratory-based study of participant experience with Echology using controlled experiment setting and variability through the use of orientation media. (Chapter 5) Design and develop interactive sound and video artwork "Echology". (Chapter 4)  Study participant interaction through informal observations of Echology at NIME '05. (Chapter 5) compare Thesis Work Figure 1.3: Thesis work overview and chapter breakdown. 15 Chapter 2 Interactive Art and the HCI Process This chapter begins by defining “interactive” and interactive art in order to distinguish it from other art forms. The definitions reveal the interac- tive artwork as no longer only a static “art object”, but as an “art system” consisting of the artist, art objects, and participants where the artwork de- velops through dynamic relationships between these art system entities. Its aesthetics are also no longer only determined by the art object but depend on the artist, participant, and behaviours within the system. To understand these new aesthetics, the roles of art system members are summarized in Section 2.2. Purposes for conducting participant experience studies are de- rived from these roles. These studies are revealed as an activity necessary in an artist’s process for the development of their technique, as an enabler of learning, evolving art objects and as a contributor to the aesthetics and content of interactive artworks. Common characteristics of these studies are reviewed in Section 2.3. Following is a brief review of the HCI process in Sec- tion 2.4.1 and in Section 2.4.2 parallels between this process and the artistic process are considered in order to highlight similarities and differences in user studies and participant experience studies. 16 Chapter 2. Interactive Art and the HCI Process 2.1 Defining Interactive Art What makes an artwork interactive? “Interactive” has become a term used very loosely [50]. It is a “buzzword” [56], [64] used to speak of interactive television, interactive web content, interactive information booths and so on, and as such, has lost some strength in its meaning. Furthermore, referring to art specifically, there is the view that all works are interactive in the sense that an artwork presents the creator’s message to an audience and the audience responds with, at the very least, some internal interpretation [56] [84]. Thus, to distinguish the interactive arts from other art forms and to begin discussion of its unique aesthetics, it is necessary to consider the meaning of “interactive”, a background history in participatory art, and a model for interactive artworks that is used for the remainder of this thesis. 2.1.1 Defining “Interactive” The dictionary provides us with the definition: interactive (adjective) 1. influencing each other 2. (of a computer or other electronic device) allowing a two- way flow of information between it and a user, responding to the user’s input (Compact Oxford English Dictionary, 2005) Key elements of this definition are “each other”: implying that interac- tion involves multiple parties, “influencing”: implying that a party to the interaction causes some change to the state or actions of the other party, and ”two-way flow of information”. Further perspective of interaction exists in the social sciences in reference to interaction between people. The term has also come to be used when refer- ring to the relationship between humans and computers or machines where “interaction between people and machines implies mutual intelligibility, or shared understanding”, ([62], Suchman p. 6). This perspective implies that 17 Chapter 2. Interactive Art and the HCI Process parties of an interaction use some model of each other to attempt to es- tablish a common language: words, actions, protocols, with which they can interact. Summarizing the above, we have the following elements of interactivity: • It involves two or more parties. • It results in some form of change in the states or actions of all parties. • There is a two-way flow of information. • Parties maintain some mental model of each other. • A common language is established for the exchange of information. The above elements can be realized in relationships between people and interactive things in a number of ways, showing that there is a spectrum to the interpretations of “interactive”. In Section 2.1.3 we will see that this spectrum has been acknowledged by art theorists in their definitions of interactive art. 2.1.2 Background in Participatory Art As mentioned in Chapter 1, the early interactive arts were pushed forward by an increased interest in participatory art. In participatory art the spec- tator becomes an “active spectator” or participant: an audience that does not just passively view artwork but plays some kind of active role in shap- ing it. Motivation for enabling the participant stems from art activities in the first half of the 20th century. In the Duchamp, Dada, and Surrealism groups, artists pulled away from the traditional arts, making attacks on art institutions and the idea of artwork being a precious, fine object [30]. They challenge the traditional roles of artists and artwork and sought to “reinte- grate art with life-praxis” [24] – to take it out of the gallery and into the audience’s everyday life. One manifestation of this is giving the audience a role as creator. For example, Peter Bürger comments on Tristan Tzara’s instructions for making a Dadaist poem – “This represents not only a polem- ical attack on the individual creativity of the artist; the recipe is to be taken 18 Chapter 2. Interactive Art and the HCI Process quite literally as suggesting a possible activity on the part of the recipient.” [24] Distinguishing between artwork for passive and active spectatorship, Roy Ascott creates two categories among which all artwork can be classified: “de- terministic” and “behavioural” [8], [11]. In “deterministic” art, the artwork produced by the artist is a final product, a “container of information” fully determined by the artist [28]. In “behavioural” art, the artwork is a “trig- ger” [12] designed to stimulate unpredictable behaviour. Ascott writes, “the necessary conditions of behaviourist art are that the spectator is involved and that the artwork in some way behaves.” In this way, the final product of an artwork can be unpredictable, shaped by indeterminable individual or social responses fed back into the work. An example of an early “be- havioural” artwork is Ascott’s piece “Change Painting” (1959). It consists of paintings on sliding panels that audience can move to create their own composition. 2.1.3 The Interactive Art System To better reflect the product of behavioural art as including artifacts cre- ated as well as processes and events that occur after conception, Cornock and Edmonds use the term “art system” in place of “artwork” [28]. As shown in Figure 2.1, the art system incorporates all involved in an artwork: the active spectators or participants, the artist, and art objects. It is the product of these members and the behaviour that arises in the relationships between them that amount to the artwork. The artist no longer communi- cates simply through a static art object but through the design of a system where interaction between system members is key [101]. Cornock and Edmonds envision interactive art as a flavour of behavioural art especially enabled by digital media. Breaking down Ascott’s division, they categorize all art based on a level of interactivity capable in the art object (see Figure 2.2). The result is four scenarios considering the relation- ships between the artist, the art object, the audience or participant, and the art object’s environment: 19 Chapter 2. Interactive Art and the HCI Process Artwork The Art System Artwork Traditional Artwork Artist Audience Artist Participant Art Object creating viewing creating viewing and affecting Figure 2.1: Traditionally the artist creates an artwork which is viewed by an audience. In the art system the artist creates an art object which is viewed and affected by audience who have become participants. The artwork is a product of artist, art object, participants, and behaviours that arise in their relationships. Static: The art object does not change. The audience observes, but does not affect the art object. Dynamic-Passive: The art object can change over time or be modi- fied by environmental factors such as sound or light. Given that the artist specifies the art object’s rules for change, the ways in which it changes are predictable. The audience still passively observes. Dynamic-Interactive: This is the same as the dynamic-passive sce- nario with the added capability for the audience to influence changes in the art object. The audience no longer passively observes and it may be said that they are participants. In this scenario the artist still specifies the ways in which the art object may change, but it is unpredictable to the extent that participant behaviour is unpredictable. Dynamic-Interactive (Varying): This is the same as the dynamic- interactive scenario with an added set of rules or what can be called “matrix” programmed by the artist that influences the behaviour of the art object. 20 Chapter 2. Interactive Art and the HCI Process Dynamic Behavioural D y n am ic -P as si v e D y n am ic -I n te ra ct iv e D y n am ic -I n te ra ct iv e (V ar y in g ) Static D y n am ic R ec ip ro ca l P ar ti ci p at o ry In te ra ct iv e Deterministic less interactive more interactive Cornock and Edmonds (1973) Ascott (1967) Cornock (1977) Figure 2.2: Classification of art systems according to Ascott [11], Cornock and Edmonds [28], and Cornock [27]. Adapted from Graham’s summary of taxonomies for types of interactivity [50]. Categories are listed from left to right according to less to more interactivity. Behaviour in the art system is fed back to the matrix and the art object changes according to rules set in the matrix. The matrix also responds by adapting itself, causing further change in the behaviour of the art object, and finally, changing behaviour within the art system. The art system learns and evolves through this feedback loop. Cornock later updates this categorization to better detail dynamic art systems (see bottom of Figure 2.2). At a higher level he maintains the division between static and dynamic art systems. He then further subdivides organizationally dynamic art systems ([27] as summarized by [14]): 21 Chapter 2. Interactive Art and the HCI Process Dynamic Art Systems: The artifact (or art object) has organizational dependence on some environment variable(s). Reciprocal Art Systems: The audience is treated as an environment variable (voluntarily or involuntarily) that causes the system to pass through a set of states. Participatory Art Systems: The artist defines a matrix of time, place, and set of ideas or constraints. The artwork is composed of the events, including interpersonal reactions of a group of participants to a situation, that take place within the matrix. Interactive Art Systems: There is a mutual exchange between man and machine that approaches the exchange that occurs during conversation between people. To achieve this the art system “should exhibit the proper- ties of a learning system”. These classifications are useful for exemplifying the spectrum of inter- activity as discussed in Section 2.1.1. Cornock’s classification also suggests that an ideal for “truly” interactive art exists. In this extreme, a partici- pant is able to engage in “real conversation” with an art system where real conversation is “an evolving, unpredictable exchange of ideas” [50]. In the remainder of this thesis the “art system” model of interactive artwork is used. The stance that all members of the art system contribute to an artwork’s aesthetics is taken. The categorizations of dynamic art systems are not discussed further save for the idea of there being “truly” interactive art systems. 2.2 Contributors to the Aesthetics of Interactive Artworks In this section the aesthetics of interactive artworks are explored in a discus- sion of the roles of art system members: art object, participant, and artist. The roles are summarized in Figure 2.3 and they set a basis from which the purposes for participant experience studies can be derived. 22 Chapter 2. Interactive Art and the HCI Process Interaction Art Object Roles: Triggering and Guiding Participant Interaction Supporting Participant Interpretation through Representations Facilitating Communication Participant Roles: Imaginary Contribution Creative Contribution Involvement in a Social Space Artist Roles: Implement Study Design Concept Figure 2.3: Roles of artist, art object, and participant. 2.2.1 Roles of the Art Object In his thesis on “Participatory Art and Computers” [14], Bell presents the art object having a physical interface and a program. The physical interface consists of input and output devices related to one or more of the human senses: sound, vision, touch, smell, and taste. It may also have input devices for sensing its environment or elements not associated with the participant. The program processes input signals from the physical interface and controls information sent to the output devices. Both physical interface and program 23 Chapter 2. Interactive Art and the HCI Process are designed to create some interaction experience envisioned by the artist. Bell provides a classification of design parameters for physical interface and program that are considered by the artist to create art objects that facilitate envisioned interaction experiences. Three roles for the art ob- ject are derived from a review of these parameters: triggering and guiding participant interaction, supporting participant interpretation through rep- resentations, and facilitating communication. Triggering and Guiding Participant Interaction Encountering an interactive artwork for the first time, a participant may be unaware of the possibility to interact. The art object should have “attrac- tors”, “things that encourage the audience to take note of the system in the first place” [40]. For example, it may be able to detect the presence of a new participant and initiate interaction. Once a participant’s attention has been captured, they may need guid- ance in interaction. Cues can be given through interface elements with strong affordances [72] as well as with the use of existing interface and program conventions. Bell gives the example of the cursor controlled by a mouse. Conventionally, its movement on the screen will follow that of the mouse in physical space. Most people will quickly become familiar in- teracting with an art object using this same convention. The art object may continue guidance by operating in a way that allows the participant to clearly see their effect on the art object [38]. On the other hand, the artist may wish to use novel or unconventional in- terface and program elements, requiring the participant to explore the space and learn the art object. The art object may be ambiguous in directives and responses so that participants “interpret the situation for themselves”, al- lowing them to “establish deeper and more personal relations” with the meaning presented [46]. Both of the above design choices have their consequences, that of the former being the participant’s potential boredom and the latter, possible confusion and frustration. For participant enjoyment, it can be desirable for 24 Chapter 2. Interactive Art and the HCI Process the art object to be balanced between the two extremes [31], [43]. Supporting Participant Interpretation Through Representations Participants develop unique interpretations of the art object shaped by their individual backgrounds and expectations. These interpretations are also supported by possible representations taken on by the art object: narration, virtual entities, virtual spaces, and mirrors. Art objects with narrative representations present stories in a linear or nonlinear fashion. Grahame Weinbren’s “The Erl King” (1982 – 1985) [98] allows a participant to explore two nineteenth century texts on a touch- screen display, discovering connections between the two and controlling the narrative flow. Representations of virtual entities may present characters or life forms to the participant. Laurent Mignonneau and Christa Som- merer’s “A-Volve” (1994 - 1997) [91] explores interaction with artificial life and intelligence. Participants can interact with virtual creatures displayed in a pool of water. Art objects with virtual space representations allow a participant to explore or enter new worlds, realities, or new spaces created through telecommunications. Finally, art objects that act as mirrors al- low participants to see representations of themselves. Iamascope, discussed further in a case presented in Chapter 3, is an artwork that uses a video camera to capture a participant moving in front of a projected image (see Figure 3.1). The participant’s image is processed to create kaleidoscope im- agery projected in the image space [44]. Interaction is much about playing with an abstraction of one’s own image. The above representations may be given to the participant explicitly or implicitly and with varying amounts of realism, thus requiring varying amounts of imagination on the part of the participant. The desired result is that rather than seeing the art object as the physical system it is, the participant becomes immersed in the world of the representation. 25 Chapter 2. Interactive Art and the HCI Process Facilitating Communication A final role of the art object is to facilitate occasions for communication among members of the art system. Bell [14] discusses occasions for commu- nication as occurring between: 1. Artist and participant through the art object. 2. Artist and art object in a reflective activity on part of the artist. 3. Participant and art object – a reflective activity on part of the partic- ipant and “allowed” for by the artist. 4. Several participants through the art object. 5. Participant and one or many autonomous art objects. 6. Participant and an art object acting as participant. In cases 1 to 4, intelligent communication is possible because all commu- nicators have some internal world models as well as understanding of their partners’ world models. This understanding is used to interpret information received from a communication partner and to choose appropriate responses. In cases 5 and 6, the art object must autonomously communicate with a participant and its “ability to respond appropriately depends on the qual- ity of the perceptual system and the computer’s ability to interpret what it perceives” [60]. Intelligent communication depends on the art object main- taining some model of participants and their behaviours and the possibility for this model to evolve through interaction or learn. This resonates with the ideal of a “truly” interactive art system spoken of at the end of Section 2.1.3 – one that “should exhibit the properties of a learning system” [27]. The art object can also create the impression of intelligence by having “in- terpretive affordances” that “support the interpretations an audience makes about the operations of an AI system.” [67] 26 Chapter 2. Interactive Art and the HCI Process 2.2.2 Roles of the Participant As the audience becomes participant they are invited to play a more active part in the artwork. Besides simply going through the physical motions of interaction the participant has roles of: imaginary contribution, creative contribution, and involvement in a social space. Imaginary Contribution Bell notes that there will be a difference in how a participant perceives a program to work and how it actually works [14]. He refers to what is perceived by the participant as the “virtual machine”: an interpretation of the actual machine or art object. For most artworks, it is the intention of the artist to have participants perceive this virtual rather than actual machine or art object. While this is supported by the art object’s representations, it also “relies on imaginary contribution from participants.” [14] Creative Contribution This role acknowledges the extent that an artwork is dependent on partici- pant behaviour and the motivation for artists to include inducing participant creativity in their artistic goals and as part of the aesthetic experience [13]. The quality of interactivity can be seen as depending “upon the extent to which the work of art can encourage both critical reflection and creative en- gagement” [30]. At an extreme, this creativity involves a participant taking “ownership” of the artwork [34]. Involvement in a Social Space Participants can be given the role of actors in a social space. The art object is often constructed as a mediator of communication between people. This can be done indirectly by creating a social situation or “environment of com- munication possibilities” [49], or directly, by being a channel through which participants may share information with each other, or become involved in collaboration or competition. 27 Chapter 2. Interactive Art and the HCI Process 2.2.3 Roles of the Artist The traditional artist role is broken down in the interactive arts. Tradition- ally, the artist is the star and there is high value in art objects made from the artist’s own hands [25]. In the interactive arts: “the artist no longer decides everything and projects it as a whole in some definitive and final composition (. . . ) he now initiates a dialogue, or set of events, which, when taken up by the au- dience, whether in a group or individually, will be shaped into totally unpredictable and indeterminate forms and experiences.” (Ascott, 1968) [12] The artist initiates this dialogue by relinquishing authorial control and creating relationships [84] through the design of an art object’s interface and program. Returning to the art object roles, it is the artist’s role to design art objects that support participants in an interaction experience, shape partic- ipant interpretation through representations and facilitate communication between members of the art system. At the same time, the artist must be aware of how participant roles are being supported. The “techniques” of the participant are “non-existent” [78] thus it is the artist’s role to create an interaction space where a partic- ipant may develop the needed techniques. This requires an understanding of participants in the context of interaction, as well as experimentation con- ducted with both art object and participants to test that desired interaction aesthetics are supported. Thus it becomes the artist’s role to include studies of interaction between art object and participants in the cycle of design, implementation and study as depicted in Figure 2.3. Results of these studies are fed back into art object redesign, future concepts [84] and the refinement of technique. 28 Chapter 2. Interactive Art and the HCI Process 2.3 Participant Experience Studies of Interactive Artworks “But accepting responsibility is at the heart of interactivity. Re- sponsibility means, literally, the ability to respond. An interac- tion is only possible when two or more people or systems agree to be sensitive and responsive to each other. The process of de- signing an interaction should also itself be interactive.” (Rokeby, 1995) [84] 2.3.1 Purposes of Participant Experience Studies Three purposes for conducting participant experience studies are derived from the roles of art system members: 1. Heightened theory development: to develop understanding of interac- tion and to evaluate art object design. 2. Design iteration: to feed back what was learned into new versions of an art object, thus creating art objects that indirectly learn and evolve. 3. Build content: to create a record of interaction within the artwork, documentation that becomes part of the artwork’s content. In the first purpose, the artist studies participant experience in order to develop an understanding of participant behaviour and perception in the context of interacting with the art object. The art object is evaluated in its ability to support the participant through the desired interaction experience. New understanding is fed back to develop the techniques of the artist. The second purpose is for new understanding developed in studies to be fed back into redesign of the art object. The art object then indirectly learns about the participants and its responses are improved. This feedback role can perhaps be interpreted as one where studies produce the evolving and learning art objects that are necessary to support “truly interactive” art systems as defined by Cornock [27]. 29 Chapter 2. Interactive Art and the HCI Process The third purpose presents participant studies not only as a process needed to develop and refine artwork and technique but also as part of the artwork’s content itself. To repeat what was stated in Section 2.1.3, it is the product of art system members and relationships between them that amount to an artwork [101]. Thus, other than the art object, interaction behaviour and individual participant experiences can also be viewed as an artist’s output. Participant experience studies document this product and the manner in which they are conducted also contribute to aesthetics. This adds further meaning to the statement from Höök et al. in Section 1.1.5: that studies are another opportunity for the artist to shape the message of the artwork [54]. 2.3.2 Characteristics of Participant Experience Studies In this section the common goals, methods, settings and subjects, and form of results of participant experience studies are discussed with the support of published cases drawn from the HCI and media arts literature ([10], [17], [18], [23], [47], [70], [93], [62], [97]). Goals In Section 1.1.3 it is proposed that generally, artists are interested in par- ticipant engagement. Part of an artist’s role is to observe and evaluate how the art object fulfills its role in supporting participants in their roles and thus, how it supports an engaging experience. Now that roles have been reviewed, the question of engagement can be translated to more detailed questions that may be posed about the relationship between the participant and art object, about the participant him or herself and about the social behaviours that emerge among multiple participants experiencing an art- work. Examples of questions that an artist may pose when conducting a participant experience study are: 30 Chapter 2. Interactive Art and the HCI Process Concerning the relationship between the participant and an art object: • How do participants approach the art object? How do they first ex- plore the interactions presented to them? • Can participants see the relationship between their actions and the art object’s actions and reactions? Does this relationship satisfy them? • Do participants understand and become engaged with art object rep- resentations? • Is the art object designed for a range of interaction skill and in what ways are participants able to better their skills towards some level of expertise? Focusing on participants themselves: • Generally, do they enjoy their experience? How long do they stay engaged and how does engagement end? • In what ways does the artwork foster personal reflection? • Does a participant move beyond performing “obvious” interactions designed into the art object to a more creative active involvement? • Are participants able to transition into states of critical thinking about the artwork concepts and aesthetics? Concerning social behaviours among multiple participants: • What social behaviours occur among people participating in the art- work either simultaneously or asynchronously as allowed for by an art object that creates a communication channel? • Does the artwork promote communication among participants? • Does interaction in the artwork environment cause breakdowns in so- cial norms? 31 Chapter 2. Interactive Art and the HCI Process The results of studies posing these questions may be applied to the eval- uation of an artwork’s technical or artistic success, to the identification of participant behaviour applicable to theories of participant aesthetics and experience, and to the redesign of future versions of the art object [10]. Methods Artists commonly study participants through informal observation of their interactions with an art object [18]. The artist may choose to not speak with the participants, only observe. They may offer assistance or expla- nation about the artwork. Supported by the social space created around many interactive artworks, they also might engage in discussion about the artwork with participants, obtaining valuable feedback and interpretations [10]. What is strongly held as important is the view that artworks be studied in context; when they are being experienced and in their “natural” setting. This is when meaning of the interaction will develop for participants [69] [92]. It is also common for the artists to make a point of not giving par- ticipants instructions [70] or not specifying that the piece is an interactive artwork [17]. Cases taking a more formal approach involve video recording of the art- work space, interviews requested of people after they experience the artwork [17], and questionnaires. A study by Aley et al. uses a “Wizard of Oz” [58] simulation in an early study to inform the design of their artwork [10]. Other exceptions to the informal methods are those of the cases that will be reviewed in Chapter 3 and studies of audience interaction in the gallery setting conducted by Beryl Graham in her PhD thesis [50]. Eth- nomethodological approaches have also been used for studying behaviour around interactive exhibits [62] and for studying artwork in the home [47] and small community environments. 32 Chapter 2. Interactive Art and the HCI Process Settings and Subjects As stated in the previous section, it is often held important that artworks are studied in their “natural” setting. “Natural” often means an art gallery. Still, as mentioned in Section 2.1.2, the interactive arts have a background in forms that seek to bring art into the audience’s everyday life. Thus, artworks have been studied in other public places such as museums, cafés, train stations, in the streets [17], [18], and even in people’s homes [47]. These diverse settings enable artists to reach a broad range of partici- pants outside of the art communities, often people having no previous ex- perience with interactive artworks [18]. Of particular interest is that broad- ening the audience allows artists to observe groups not so commonly repre- sented within art communities, for example, the elderly and children. Both are significantly interesting subject groups; the former with experience in a time that computing technology was non-existent or new, and the latter not knowing a life without it. Finally, artworks are often studied at special events and exhibitions of limited duration. Some studies hold special exhibitions or parties with the specific purpose of studying an artwork [70], [97]. Installation and mainte- nance of interactive artworks can be difficult as often, only the artist has knowledge of how to setup, operate, and fix the art objects, which can be prone to bugs and breakdowns. Thus, it can be more difficult to find places for permanent installation. Fortunately, temporary exhibitions can be good for receiving feedback on an artwork and then having a chance to redesign the art object before future exhibitions. Form of Results The manner in which results are presented is highly dependent on where they are being presented. For published results, a paper’s audience may be more interested in learning about the art system as a whole, about the design and roles of the art objects, or about the playing out of participant roles. Results may then be isolated to a single paragraph near the end of the paper or they may be discussed in further degrees of detail. 33 Chapter 2. Interactive Art and the HCI Process Results are almost always of a qualitative nature. Some exceptions are the measurement of participant demographics, average times spent inter- acting with the artwork, and quantities derived from questionnaires [97]. Otherwise they often take the form of general observations in statements such as “we saw many people who tried to. . . ”, “people often said that. . . ”, and so on [70]. The observations can be supported with examples of be- haviour among one or many participants, or with short statements made by the participants: “There was one group that. . . ”, “A woman said that she felt. . . ”, etc [97], [17]. More in-depth discussion is often conveyed in a narrative form, telling of participant interaction in general and garnish- ing it with the instances that stood out, being exceptionally surprising or interesting. Some reports construct interaction stories giving participants pseudonyms and establishing their character to personalize the experience being told [47]. Examples of topics that an artist may cover when discussing the results of a participant experience study are responsive to the questions posed in Section Concerning the relationship between the participant and an art object: • How participants approach the artwork space. • How participants explore the interaction space. • Relationships discovered between actions and art object reactions. • Stages of interaction and how it would normally begin and end. • Metaphors and concepts used to make sense of the art object. • Comments and criticisms. Focusing on the participant him or herself: • Relating the artwork to personal things like previous experiences. • Emotions experienced. • Strategies of interaction taken. 34 Chapter 2. Interactive Art and the HCI Process • Moments of creativity. • Skill levels obtained. • Satisfaction with personal performance. Concerning social behaviours among multiple participants: • Conversation occurring among strangers. • Social norms and barriers broken. • People engaging in play behaviour. • Interaction between people with some prior relationship. 2.4 The HCI Process This section briefly reviews the traditional HCI process of user-centered, or interactive design, focusing on the role of user studies in this process. Section 2.4.2 considers the parallels between the HCI and artistic processes in order to re-examine similarities and differences between user studies and participant experience studies. 2.4.1 User-Centered and Interaction Design User-centered design is broadly a design process in which the user’s needs and context influence design [7]. Goals driving this process are known as ”usability goals”. Preece et al. list these as: effectiveness, efficiency, safety, utility, learnability, and memorability [82]. Guidelines and rules for design- ing for these goals are suggested in several sources [71], [74], [90], all having common points such as: • Make the system status visible to the user at all times. • Match between the system and the real world, speaking the users’ language and using real world conventions. 35 Chapter 2. Interactive Art and the HCI Process • Follow existing standards and conventions set for the design space. • etc. . . As mentioned in Section 1.1.4, HCI has broadened from being only con- cerned with usability to having concern for the user experience of a system or a product as a whole. This is reflected in the proposition of interaction design [102] – which is founded in user-centered design but also considers user ”experience” goals. Preece et al. list these goals as creating systems that are: satisfying, enjoyable, fun, entertaining, helpful, motivating, aes- thetically pleasing, supportive of creativity, rewarding, and emotionally ful- filling. The interaction design process is an iterative one with four general ac- tivities as defined by Preece et al: • Identifying needs and establishing requirements • Developing alternative designs • Building interactive versions of the designs • Evaluating designs Through iterations of these activities, a system or artifact transitions from low to medium to high-fidelity prototypes and eventually, to a final product. Designers involve users in the process to both inform design requirements and to evaluate designs implemented. Methods used in the early stages of a project may be interviews and questionnaires on user backgrounds and work processes, focus groups, on-site observations, role playing, walkthroughs, and simulation [7]. When a product begins to reach its final form, evalu- ation of the product’s fulfillment of usability and user experience goals is conducted. 36 Chapter 2. Interactive Art and the HCI Process Implement User Studies Design User Needs and Requirements Existing usability and user study heuristics HCI Process Implement Participant Studies Design Concept Part of final artwork Artist Process Artistic goals Figure 2.4: Iterative processes of design, implementation, and studies com- monly used by both HCI practitioners and by artists. 37 Chapter 2. Interactive Art and the HCI Process 2.4.2 Parallels in HCI and Artistic Processes The artistic process depicted in Figure 2.3 can be compared to the HCI pro- cess as discussed in the previous section. Figure 2.4 shows how both artist and HCI practitioner iterate through activities of design, implementation, and evaluation and involve studies with users or participants in these cycles. However, while recommendations for different methods suitable to different stages of the cycle have developed quite extensively in HCI, recommenda- tions of the same caliber do not exist for methods of studying participant experience. This work takes the opinion that this is because participant experience is often studied as an afterthought and, as discussed in Section 2.3.2, methods used are often those of informal observation and interview in an artwork’s exhibition setting. Still, as will be illustrated in the next chap- ter by three cases of more formalized approaches to participant experience studies, there are benefits to be gained by exploring various methods that could be used at different stages of an artwork’s lifetime. If HCI methods are appropriated to study participant experience, it is beneficial to acknowledge several differences between the HCI process and the artist process: • The HCI process begins with identifying needs and requirements; the artistic process begins with an experience concept. • User studies evaluate products against existing goals and heuristics common across many products; an artist evaluates according to unique artistic goals that may or may not be informed by previously estab- lished heuristics. • Participant experience studies do not only inform design, they are also part of the final artwork. 38 Chapter 2. Interactive Art and the HCI Process 2.5 Summary This chapter has distinguished the interactive arts from other forms by pro- viding background to the meaning of “interactive” and by presenting the interactive artwork as an art system consisting of artist, art objects, and participants. Roles for each member of the art system are defined and re- veal participant experience studies to be both an activity in an iterative process taken on by the artist and a contributor to the artwork itself. Com- mon characteristics of participant experience studies are reviewed, revealing a tendency to use open and informal methods in the artwork’s natural con- text. Finally, the HCI process is briefly reviewed and compared with the artist’s process. Differences between the processes are highlighted to better understand how HCI methods may be appropriated by artists to fulfill their artistic needs and goals. 39 Chapter 3 Formal Participant Experience Studies This chapter presents three cases of participant experience studies conducted using more formal methods. These cases are chosen for their attempts to establish more rigorous methodology for studying interaction, prioritize the role of studying participant interaction in the development of their artworks, and communicate their experiences to both the art and the HCI worlds. They set a basis to which this thesis makes its contributions. For each case study goals, artwork studied, methodology used and results presented are detailed. A summary is provided in Table 3.1. The chapter concludes with a discussion of the benefits of formal study methods as exemplified by the case studies and a proposal of further work taken on in this thesis. 40 Chapter 3. Formal Participant Experience Studies 3.1 Case One: The Influencing Machine 3.1.1 Goals In their study of the “Influencing Machine” [54], Höök, Sengers, and An- dersson have the research aim to develop methodology better suited to the interests of artists. They take the stance that HCI evaluation methods can be used to better understand and improve interactive systems. But they also note conflicts that exist in the differences between artist and HCI prac- titioner perspectives and that HCI methodology may not produce results that are of interest to artists. They investigate a laboratory-based method- ology based on HCI evaluation methods while keeping in mind the needs of artists. The methodology is used to determine what thoughts their artwork, the Influencing Machine, is capable of provoking in participants. They also want to distinguish, in the case that a user becomes frustrated during in- teraction, whether frustration is rooted in not being able to directly control the Influencing Machine or in fact, poor design choices. 3.1.2 Artwork The Influencing Machine is an artwork that explores an element of affective computing: how people relate to a machine that they can influence but not control. Its purpose is to provoke thoughts on a machine’s capability to be emotional. The work consists of a wooden mailbox around which post- cards are scattered, video projected onto a wall, and a sound system for playing musical sounds. In the projected video, childlike drawings contin- uously appear, build up together and fade out. The postcards are printed with other artworks or a color field, and a barcode. The postcards are an input mechanism and when one is dropped into the mailbox, the Influencing Machine detects which card has been dropped and, based on an emotional model, plays sounds and alters how images are drawn in the video projec- tion. Users can drop postcards in varying combinations and intervals and the Influencing Machine will emotionally react. The interpretation of the reaction is open-ended and highly based on the background of the user. 41 Chapter 3. Formal Participant Experience Studies 3.1.3 Method The experiment setting is a room of an old house set up to not look like a laboratory or office space. The Influencing Machine is placed on a clothed table under which the computer is hidden. Participants are brought in groups of varying size and a co-discovery method [39] is used to capture a participant’s understanding while experiencing the artwork, but without forcing them to speak out loud. This allows them to observe group dynamics around the artwork and have the experience approach a setting more similar to a non-laboratory setting. Before viewing the artwork short interviews are conducted. Knowing from previous experience that it is inadequate to classify users based only on basic demographic information (age, gender, education, etc.) they use these interviews to collect information on perspectives of computer culture and art installations. Then, participants are told that the machine they will view is “about emotions and that they will be posting postcards into a mailbox”, that they can do as they please and leave when bored, and that they will be video-taped. The study is done in two parts according to previously established guide- lines for evaluating affective interactive systems; the first part should assure that emotions expressed between the system and the user are understood by both, the second part checks that this understanding leads to the desired interaction effect [53]. In a first study, the amount a participant can influ- ence the artwork with postcard input is constant. In a second study they revise the artwork design to speed up the drawings and include a display of the machine’s emotional levels as well as musical feedback. Some groups are tested with the display and some without. Groups also use either only the art postcards or only the color field postcards, and the amount of influence capable is varied. After viewing the artwork, an open-ended interview is conducted where the interviewer is careful not to use language that associates metaphors to the artwork. 42 Chapter 3. Formal Participant Experience Studies 3.1.4 Results In the first study users were first curious and then became frustrated that they could not figure out the relationship between the postcards and the drawings projected. Other observations were that participants found the drawings too simple or slow, they liked putting things inside the mailbox, there was some success in having participants reflect on the meaning of the artwork, and there was frustration over the lack of control. After revising the artwork and conducting the second study, they conclude that the desired results were achieved and comment on the influence of a number of the art object components. They also learn about differences in behaviour among theory formers and those that did not form theories about what was going on. Results are presented as follows. Results from the first study are pre- sented as general observations supported by quotes from the participants. Results from the second study are presented in the following parts: detailed experience accounts of 3 out of 9 groups studied, followed by a summary of the experiences of all groups, summaries of responses to 5 interview ques- tions with key words and quotes highlighted, a summary of metaphors used by the participants, and finally a discussion of their overall impressions. A highlight in the presentation of their results is the narrative fashion in which they detail the experience accounts of the 3 groups. This narration tells of interaction with the machine, interaction among participants, the- ories formed, strategies used, and participant reflections. The presentation manner is successful in revealing the experiences in a holistic manner while still maintaining intricacies and unique characteristics. 3.2 Case Two: Iamascope 3.2.1 Goals The second example [29] is a user study of the camera and video system Iamascope [44], a piece by artist Sidney Fels. This case is not a study con- ducted by the artist himself but by another group of artists and researchers 43 Chapter 3. Formal Participant Experience Studies well known in the field of interactive arts. They are not interested in eval- uating Iamascope directly, but instead use it as a case with which they can “find useful methodology for recording and analyzing the situated experi- ence of interactive art”. They aim to find methodology that fits in the gap between methods of in-depth data collection in controlled settings such as that used in the previous example, and more informal observational research conducted in real world settings. Using as a basis a framework created by Fels from his own observations [43], they wish to determine if their meth- ods of data collection can “enrich these observations by showing how such categories are produced and operate in audience experience.” 3.2.2 Artwork Iamascope is an interactive kaleidoscope using a video camera to capture participants moving in front of a projected image space (Figure 3.1). Video of a participant is processed to create kaleidoscope imagery displayed in the projection space, thus reflecting an abstraction of the participant’s image back to the participant. Based on his own observations of participants interacting with the art- work [43], Fels developed a framework for relationships that occur between a person and an interface. The four relationships: response, control, reflec- tion, and belonging, provide different types of pleasurable experience that contribute to intimacy achieved between the person and the interface. 3.2.3 Method The artwork is viewed in an exhibition space, Beta space, in Sydney, Aus- tralia’s PowerHouse museum. They conduct three studies with three sub- jects. To stimulate a normal museum experience, the subjects are invited to tour the PowerHouse museum as they please but asked to, at some point, visit Beta space. Video-cued recall [92] is used to capture the situated experience with- out heavy cognitive load for the participant and to help the participant remember details during a session of retrospective reporting. When a sub- 44 Chapter 3. Formal Participant Experience Studies Figure 3.1: Iamascope (Fels, 1997). ject arrives at Beta space they are video-taped (along with the artwork) for the entire duration of their interaction. Afterwards they are taken to a room nearby where they watch the video of their interaction and are asked to comment on what they had been think- ing. They can move back and forth through the video and pause as they please. This is also video-taped so that when sessions are later analyzed, comments can be associated with video events. This session is wrapped up with an interview about experience with Iamascope and the video-cued recall method. Data is analyzed using a method based in grounded theory [48]. Verbal recordings are transcribed and video events coded. With no existing coding scheme for this type of experience a new coding scheme is created, incorpo- rating terms used by the participants. Codes are grouped into categories of movement and cognitive states. The latter is further broken into categories of assessing system, referring to self, response, and described behaviour. The coded data is then compared with Fels’ framework and examined for ways in which the data confirms or contradicts categories of the framework. 45 Chapter 3. Formal Participant Experience Studies 3.2.4 Results They report that from the data, they are able to associate classes of move- ment, vocabulary and behaviour with each category of Fels’ framework. They are also able to observe transitions among the categories and deter- mine common patterns or “trajectories” of interaction. These results are discussed in relation to each category in Fels’ framework, including relevant observations and participant quotes. They also conclude that the participants reacted positively to the video- cued recall method as it helped them more deeply discuss the experiences they had. Still, they caution that the verbal data should be considered an “interpretation” of the experience because, even though it is supported with video, it is produced retrospectively and out of the experience context. A final remark they make about the reliability of the method is that there were noticeable changes in study participant interaction behaviour from that observed in participants not being studied. The study participants all stayed longer and made sure to explore the artwork in its entirety. 3.3 Case Three: Façade 3.3.1 Goals A third example is the evaluation of the interactive drama Façade by Knick- meyer and Mateas [59]. They aim to present methodology that can be used to study aesthetic and experiential aspects of a system rather than task accomplishment. To focus their study, they define a successful experience with Façade as “one in which players experience a sense of agency, maintain engagement, and are motivated to replay in order to try different interaction strategies”. They test to see if players of Façade can maintain a certain level of en- gagement in an experience that does not have concrete goals and, due to limitations in the AI technology used to create Façade’s responses, is prone to interaction failures. 46 Chapter 3. Formal Participant Experience Studies 3.3.2 Artwork In Façade a single “player” sits at a computer and enters a virtual world in which he or she may move around, interact with objects, and have typed natural language conversations with AI-controlled characters (see screenshot in Figure 3.2). A participant “plays” but Façade is not a game in that there are no clear goals. Their interactions influence the virtual characters and thus, shape the procession of the story. This gives the drama “replay” value as different story trajectories can be experienced each time. Figure 3.2: Screenshot of interaction with characters Grace and Trip in interactive drama Façade (Mateas and Stern, 2005). 3.3.3 Method Retroactive Protocol Analysis [42] is used as follows. The screen is video- taped while a participant plays Façade. After playing, the participant watches the video taken and is given a script of game play to read. During this time the participant is asked to describe what is going on and what his or her reactions mean. The participant then plays Façade a second time and a post-experience interview is conducted. The above is all recorded as data for the protocol analysis. To analyze the data they use an existing coding scheme developed to code gaming ex- periences and expand it with their own categories of Agency, Exploration, 47 Chapter 3. Formal Participant Experience Studies Strategy, and Disinterest. 3.3.4 Results Overall, six out of the eight people who participated in the study said that they would like to play Façade again. In accordance with their definition of a successful experience, they take this as an indicator of Façade’s success. Beyond determining the success of their system, they are able to use the data to identify standard patterns of play: one where a player explores game strategies and actively tries to manipulate certain outcomes and another where a player spends less time strategizing but seems more immersed in the game role and tries to act like him or herself. They also identify three types of common behaviour in the face of interaction breakdowns: the player taking advantage of the breakdown, the breakdown actually triggering an affective state such as sympathy or frustration for the characters, and the player, after understanding that there has been a failure with the system, changing strategy. They are able to confirm that with all three behaviours engagement was maintained. Results are discussed using references to examples of the data in graph- ical formats. In one graphical form, the total times spent talking for each category is plotted in a bar chart for each player. In another form, categories are plotted over time and from this, transitions between categories can be observed. 48 C h a p ter 3 . F o rm a l P a rticip a n t E x p erien ce S tu d ies Table 3.1: A summary of the Influencing Machine, Iamas- cope, and Façade participant experience studies. Influencing Machine Iamascope Façade Study Goals Determine thoughts pro- voked by artwork; distin- guish causes of frustration. Observe how categories of an existing experience framework are played out in the experience of an artwork. Test that participants stay engaged interacting with no clear goals and possible interaction failures; test re- play value. New Meth- ods Better suited to interests of artists. To record and analyze situated experience of in- teractive art using method between informal, real world observation and formal laboratory-based studies. To study aesthetic and ex- periential aspects of a sys- tem. Continued on next page 49 C h a p ter 3 . F o rm a l P a rticip a n t E x p erien ce S tu d ies Table 3.1 – continued from previous page Influencing Machine Iamascope Façade Artwork Art object representing emoting being; interact by inserting postcards into art object. Art object representing participant, mirror; full body interaction. Art object representing characters and a story; text-input dialogue inter- action. Setting Room in a house. Exhibition space in a mu- seum. Not specified; assume lab- oratory. Method Co-discovery; video record- ing; interviews; methods for evaluating affective in- teractive systems. Video-cued recall; Retroactive Protocol Analysis with custom coding scheme. Video-cued recall; Retroactive Protocol Analysis with adapted coding scheme; replay. Continued on next page 50 C h a p ter 3 . F o rm a l P a rticip a n t E x p erien ce S tu d ies Table 3.1 – continued from previous page Influencing Machine Iamascope Façade Results Identify art object fea- tures that frustrate partici- pants; observe different be- haviours among theory for- mers and non-theory form- ers; results applied to re- design of art object. Associate participant be- haviours with framework; identify patterns of transi- tion among framework cat- egories. Determine artwork’s suc- cess; identify standard pat- terns of play; identify three common behaviours in the face of interaction break- down. Form of Re- sults Discussion of observations highlighted with selection of quotes; keyword sum- maries; detailed narrative of 3 group experiences. Discuss observations and interviews according to framework categories; relate with examples of participant behaviour and quotes. Discuss results making references to a selection of categorized interview statements displayed in graphical format. 51 Chapter 3. Formal Participant Experience Studies 3.4 Contributing to Existing Study Methodology The three cases presented in this chapter are unique in their formalized approaches. Goals for the study are explicitly established beforehand. They study artworks in their natural interaction contexts but under controlled settings and with participants who are recruited as study subjects. In-depth data is collected, including records of both participant and art object actions and the data is analyzed according to protocols and heuristics suitable for the context of the study. Their approaches exemplify a number of benefits to be achieved for the artist using more formal methodology: • Allowing controlled investigation of different design options for art objects. • Concluding with suggestions for art object redesign. • Identifying general participant behaviour types and placing them into existing or new experiential frameworks for participant aesthetics. • Identifying participant techniques used. For example, those used while trying to understand an art system (Influencing Machine) or in the face of interaction breakdown (Façade). • Acquiring a collection of documented interaction experiences. • Maintaining uniqueness and depth among collected participant expe- riences. They also provide a basis from which further inquiry into the use of formal methodology to study participant experience can be made. This research takes on a problem that was only briefly touched upon in the Iamascope case: that of the effect of participant bias in the context of a formal study. They made a short remark about noticeable changes in the behaviour of participants of the study in comparison with the be- haviour of participants not studied: studied participants stayed longer and 52 Chapter 3. Formal Participant Experience Studies explored the artwork in its entirety. There is a wealth of psychology and so- ciology research on the effects that experiment settings have on participant behaviour. The Hawthorne effect [68], [76] and experimenter effects [86] are two examples. Generally, it is considered that participants who are aware that they are “subjects” of a study will behave differently. This work seeks to observe changes in the behaviour of participants of an interactive artwork study and to make suggestions on how these changes should be considered when interpreting study results. A second problem taken on in this research is the exploration of the use of pre-experience information, or orientation media, administered to participants in order to selectively influence their understanding of different artwork elements. Variability in a controlled study through the adminis- tration of different orientation media to different participant groups may help reveal the importance and effect of participant understanding of vari- ous elements of the art system in the participant’s overall experience of the artwork. 3.5 Summary This chapter has detailed three cases of participant experience studies con- ducted using more formal methodology. With controlled settings and meth- ods of in-depth data collection the artists and researchers are able to evaluate art object design and develop understanding of participant behaviour while still maintaining their artistic goals. These studies provide a foundation for the further investigation into formal methods for studying participant ex- perience. That is, observing the manner in which a controlled experiment setting changes participant behaviour and developing methodology to influ- ence participant understanding of artwork elements in order to distinguish the effects understanding of those elements have on interaction experience. 53 Chapter 4 Echology The interactive sound and video installation Echology [37] is used as a case study for investigation into laboratory-based methods for studying partici- pant experience. Echology was designed and developed for exhibition in a public atrium at a university. This chapter first presents the artistic motiva- tions and design guidelines that served as a basis for Echology’s development. Then, a detailed description of the installation as it was implemented for exhibition at the New Interfaces in Musical Expression conference (NIME) and for the subsequent study is presented. Finally, the intended participant experience of Echology is discussed. 54 Chapter 4. Echology 4.1 An Interactive Sound and Video Installation 4.1.1 Echology Summary Echology (Figure 4.1) is an interactive sound and video installation for par- ticipants to collaboratively play with directional sounds initiated by the playful movement of Beluga whales in the water captured with a single live webcam provided by the Vancouver Aquarium [6]. Figure 4.1: Echology installed in the Open Media Environment at the University of British Columbia during NIME 2005. This webcam feed provides an effective means to bring the visual ele- ments of the Beluga whales into the piece and to drive a sound space using video processing. The amount of motion resulting from processing the video is mapped to the amplitudes of four sound sources. In these sources, the sounds of the Beluga whales, recorded over 3 years using underwater micro- phones, are remixed and spatialized in a surround sound system. Partici- pants play with the spatialized sounds through a button and display tabletop interface. 55 Chapter 4. Echology 4.1.2 Artistic Motivations The playful, graceful motions of Beluga whales swimming in water create a mesmerizing motion space to watch. These highly communicative creatures use a variety of vocalizations, physical expressions and physical contact in their navigation, social interaction and survival [96]. Artistic motivations stem from an appreciation for watching the Belugas swim, play and com- municate at the Vancouver Aquarium, and wonderment over how watching the Belugas through the aquarium’s webcam can be similarly engaging. We were inspired to create an interactive soundscape that provides a representa- tion of the space inhabited by Beluga whales and that, through interaction with sounds, gives participants the feeling that they are communicating or playing with the Beluga whales. We focused on enjoyment, understanding and engagement as being the key elements of the expressive meaning of this interactive installation. 4.2 Echology Design Guidelines 4.2.1 Collaboration, Simplicity, and Approachability Echology was created with the intention of installing it in a public atrium space that will be discussed in section 4.2.2. Our vision for the piece was to create a focal point in the large and normally empty atrium so that passersby would come to interact and socialize with other participants. Hence, from the beginning we focused on supporting participant collaboration. Blaine and Fels, referring to D’Arcangelo [32], recommend that “when designing collaborative musical experiences for first-time players in public places, the amount of time necessary to learn an interface must be minimized, coupled with achieving a balance between virtuosity and simplicity” [20]. Thus we also focused on making interaction with Echology simple. Finally, because Echology would be placed in a space outside of the museum or gallery where behaviour norms are more conducive for exploration, we focused on creating an approachable space. These three focuses: collaboration, simplicity, and approachability, were used as guidelines throughout the entire development 56 Chapter 4. Echology process and are discussed below. Collaboration As discussed in Section 1.1.2, Blaine and Fels [20] have identified many of the qualities to organize collaborative, multi-participant interactive sound installations and guidelines based on these qualities to make the sound in- teraction successful. Echology was placed in their classification system and the corresponding guidelines were integrated into its design. The relevant qualities and recommendations followed are listed in Table 4.1. Table 4.1: Echology’s placement in Blaine and Fels classifi- cation system for collaborative musical experiences [20] and corresponding guidelines integrated into its design. Quality Echology Design Recommendation capacity multiplayer, single interface Use turn-taking protocols as well as having clear relationships between ac- tion and sound with a multi-person, single interface. aptitude novice Prioritize that the sound be engaging at the expense of virtuosity. media sound, video Use video to strengthen the relation- ship between action and sound with- out being too distracting. player interaction buttons Participants use 8 buttons arranged in a circle to reflect sounds; each player has the same type of interaction so that they can learn from each other. Continued on next page 57 Chapter 4. Echology Table 4.1 – continued from previous page musical range Players control spatial parameters of music; restrict sound controls to help create an engaging, satisfying experi- ence for novices. physical interface buttons Provide Norman-style affordances [50] to engage novices and make it easy for them to join the group. directed interaction low Attendants explicitly direct novices so they understand how the interaction works. learning curve fast Use a direct mapping between gesture and sound to speed up learning. path to expert none Compromise virtuosity for ease-of-use as the sound installation is intended for novices. levels of physicality medium Arcade-style buttons encourage mod- erate levels of physical interaction that “lay the foundation for devel- oping intimate personal connections with other players. . . ” [20]. Simplicity Toshio Iwai’s Composition on the Table [57] provided inspiration artistically and technically for the sound and interaction method used. Iwai’s work uses a large horizontal projection surface with a light grid on the display. At each node of the grid, a button allows the player to change the direction of the arrow associated with the node. A coloured ball of light moves along the lines of the grid and then follows a path decided upon by the direction of the 58 Chapter 4. Echology arrow at each node it encounters. When a node is hit, a specific MIDI note plays. There are four lights moving at different speeds. Thus, with careful selection of the direction of the arrows, complex loops and rhythms can be created. Working with multiple players allows very complex sounds that cannot be made as effectively by a single person, since there are too many lights active at the same time. Yet even though the resulting compositions are complex, the barrier to interaction is extremely low. A participant has only one method of interaction – the button press, only four moving lights of visual feedback to follow, and a straightforward mapping of notes played on light-node collision events. Along with this simple interaction, Iwai achieves simple and comfortable ambience through the use of minimalist aesthetics. We draw from what Iwai has achieved with Composition on the Table and keep participant input limited to button presses as well as attempt to make mappings between the Beluga webcam video, button presses, visual elements, and the soundscape as straightforward as possible. Approachability Echology would be installed in a large, open, and central public atrium where the behaviour norm is to pass through. Other than during organized events held in the atrium the space is usually empty. We were aware that people would possibly hesitate to enter the interaction space, either because of shyness to break a social norm or because of being occupied and not having time to stop and look. Our plan to counter this was to create an approachable space with an air of mystery and allure. To create an inviting space, we constructed the Interaction Table to ap- pear fun and exciting. We did this through our choice of covering it in a fluffy white fabric and using large, lighted, blue buttons to entice spectators to the table. Our original design included a projection of the Beluga webcam video so that those not at the table could observe and appreciate the map- ping between Belugas and sound. At the same time, the visual elements and sound maintained a mysterious ambience of the underwater world, a strange and unknown place that provokes curiosity in many people. 59 Chapter 4. Echology 4.2.2 Designing for the Open Media Environment Echology was designed to be an interactive multimedia installation piece for exhibition in the Open Media Environment situated in the atrium of a build- ing at the University of British Columbia. The Open Media Environment incorporates a center stage area and a movable, suspended, circular theatri- cal truss that has eight attached speakers and can support various lighting attachments. The circular truss has a rear projection screen stretched inside it. An initial motivator to create Echology was to provide an artwork that could showcase the Open Media Environment’s functionality. The infras- tructure available highly influenced Echology’s system design. Using this space also produced deployment issues that had to be taken into consider- ation. The space is in the center of a building housing offices, laboratories, and classrooms. The installation would be active during the daytime on weekdays so we had to assure that sound levels would not disrupt the peo- ple working in the building. 4.3 The Echology System Description The installation consists of a custom made Interaction Table, 8 surround speakers, a PC running Max/MSP and Jitter [2], 8 lights and a lighting controller, and the live webcam at the Vancouver Aquarium. Figure 4.2 depicts the arrangement of these physical components. Figure 4.3 depicts Echology from a systems point of view, indicating its inputs, outputs, and processing done in software. It is from this point of view that I will first briefly review the components of Echology. Input to the Echology system consists of the Vancouver Aquarium we- bcam [6] and the 8 Interaction Table buttons. These buttons represent the reflection points on the edge of the soundscape (i.e. the eight loudspeakers). The 8 buttons are arranged symmetrically around the perimeter of the ta- ble (Figure 4.12) and are used to control the movement of the sounds in the sonic aquarium. The Interaction Table also houses a monitor to display our visualization 60 Chapter 4. Echology Figure 4.2: Echology system diagram: physical components. of the Beluga webcam and sound space for the participants. The display gives graphical visual feedback to the participants so they can see where the sounds are and where they will go. The sound space is created using 8 speakers distributed symmetrically just above head level. A light is mounted below each speaker and lights up momentarily to indicate when a sound passes by the speaker to enhance the spatialized sound. All sound, video, and button event processing is done using the Max/MSP graphical environment for multimedia programming [2] and Jitter, a col- lection of video and graphics objects to be used in Max/MSP. We chose Max/MSP because of its support for managing complex installation systems involving multiple devices and media in real time and the ease of creating intuitive graphical interfaces for system monitor and control. Max/MSP is also commonly used in the video and sound art community, so support for installation-specific issues was readily available. 61 Chapter 4. Echology Figure 4.3: Echology system diagram: inputs, outputs, and processing done in Max/MSP/Jitter. The above system description is of the original installation of Echol- ogy in the Open Media Environment. For the participant experience study the installation was moved to an empty room in a laboratory building and modified somewhat because of hardware constraints. At this time only six speakers were available so the system’s program was modified to use only 6 speakers but to maintain, in the geometry of spatialized sound, the 8 re- flection points arranged symmetrically around the edge of the soundscape. The new space was also unsuitable for hanging lights without the necessary wiring becoming dangerous so the 8 lights were not used. Instead, 4 lights were placed around the room as ambient lighting. 62 Chapter 4. Echology 4.3.1 Beluga Video and Motion Capture The Vancouver Aquarium has four Belugas in the tank shown through the webcam. From the webcam, a variety of different Beluga activities can be observed. They have various swimming patterns, sometimes play with bubble rings, do headstands, bob their tails up in the air and perform trained movements during the aquarium show times. The Echology installation processes the live webcam feed to capture these Beluga motions and uses them to control sound in the Echology soundscape. Motion types of the whales change given the different areas of the cam- era’s view. For example, the top portion of the view contains the tank’s water surface. The Belugas spend most of their time near this surface and often stay fixed in one place, bobbing up and down; this activity is partic- ularly high during the training shows that occur every 1.5 to 2 hours when the aquarium is open. We used this idea of different camera view areas to divide the view into four “layers” where the motion detected in each con- trols one sound signal. These four layers are evenly spaced from top to bottom as shown in Figure 4.4 . The layers were given labels of “Splash”, “Play”, “Swim”, and “Deep” for layers 1 to 4 respectively. The 4 sound signals associated with these four layers are composed to be aesthetically and conceptually characteristic of Beluga motions within their given layer. For example, we have a playful motif for the sound associated with the Play layer. The nature of the musical elements will be discussed further in Section 4.3.8. 63 Chapter 4. Echology Figure 4.4: Layering of the Beluga webcam view. Beluga motion in each layer is mapped to the amplitude of the asso- ciated sound signals. Motion is calculated by subtracting grayscale pixel values in consecutive frames of the Beluga webcam feed. If the difference between 2 frames is greater than a threshold value, the intensity of motion (or magnitude of difference) is used to modify the sound’s amplitude. To avoid detecting the small, rapid motions of light ripples in the water, the frame’s brightness, contrast, and saturation are adjusted and the frame is passed through a blurring filter. For further smoothing over consecutive frames, motion values are averaged over a sliding window of 13 frames; the averages are then normalized and sent to the sound controller. This entire process is shown in Figure 4.5. 4.3.2 Soundscape Spatialization Before spatialization, the sound controller scales the amplitude of four sound signals by the associated values received from the Beluga motion capture; the more motion there is in a video layer the louder the associated sound signal is. If there is no motion, the sound is not heard. Often the Belugas move in the webcam view for only a short duration of time. This makes sounds disappear too quickly from the soundscape to be able to interact with it. To remedy this we added a 10 second sustain to keep the sounds 64 Chapter 4. Echology Convert to binary. If pixel value > threshold, pixel = 255, else pixel = 0. Cut frame into layers Splash, Play, Swim, Deep. Previous Video Frame Adjust contrast, brightness and saturation. Perform convolution blur with cross-shaped kernel. Video Frame Convert to grayscale. Calculate absolute difference between current and previous frames. For each layer: calculate mean pixel value. Scale to value between 0 and 100. For each layer: calculated weighted average mean value over 13 frame window. Send 4 average values to Sound Controller Figure 4.5: Steps of Video and Motion Processing of Beluga Webcam Feed Frames playing after they pass a threshold volume level. The four sound signals are spatialized around the installation space using a spatial sound engine based on the Max/MSP plugin for Vector-Based Amplitude Panning (VBAP), developed by Ville Pukki [83]. VBAP provides sound spatialization using an arbitrary specification of speaker position in two or three dimensions. This is an essential feature for installations such as Echology that require a high degree of control over sound positions and are set up in disparate environments using fixed infrastructure. Amplitude panning involves sending the same sound signal to a number of different loudspeakers, with amplitudes based on the loudspeaker posi- tions and the location of the intended sound signal. Using VBAP allows 65 Chapter 4. Echology us to easily fit the loudspeaker setup to the space our installation will in- habit. With the VBAP plugin, the speaker placements can be changed on the fly in a modular fashion that does not affect any of our other software components. This method of spatialization is used in a panner that accepts up to eight channels of input sound signals. Each channel can exist in a unique sound- scape position at the same time. We use four of these, one for each sound signal. A panning controller calculates and maintains x and y coordinates for each sound signal. These coordinates are sent to the panner, as well as to the graphics and lighting controllers discussed below. By default, sound is spatialized along the perimeter of the installation space defined by a circle made by the symmetrical placement of the speak- ers (Figure 4.6). We also define 8 locations positioned symmetrically on this circle as “reflection points” (Figure 4.7). Each reflection point has a “direction” that is set to direct to one of the other 7 reflection points. As a sound signal passes a reflection point, it continues along the path of that reflection point’s current direction. This way, a sound signal can travel along any path between two reflection points, and thus, on paths within a 2-dimensional plane bounded by the spatialization perimeter. A reflection point can also direct to itself and in this case, a sound that arrives at the point becomes caught until the point’s direction is changed again. This is shown as the rightmost state of reflection point B in Figure 4.8. The de- fault position for each reflection point is to direct towards its neighboring reflection point so that sounds travel in a circle around the soundscape as in Figure 4.7. 66 Chapter 4. Echology Default sound path Figure 4.6: Two-dimensional sound plane above the Interaction Table de- fined between ring of 8 speakers. A reflection point Figure 4.7: Eight reflection points placed at equally spaced positions around the sound plane. A reflection point is represented by a circle with a pie slice removed. The pie slice indicates the point’s direction of reflection. 67 Chapter 4. Echology  D E F Sound travels to: B AG H B A C D E F G H Figure 4.8: The eight directions of reflection point B. This diagram shows the orientation of B as it directs towards reflection points C to A. In the rightmost, doughnut shape depiction of B it is directing towards itself. 68 Chapter 4. Echology  B C D E F G H Figure 4.9: In this example of a sound path between reflection points, points A, C, and E have directions to C, E, and A respectively. Sound will travel in a path between A, C, and E. This is illustrated with the example shown in Figure 4.9. The circles with pie-shaped cutouts represent the reflection points. When a sound hits reflection point A, it reflects towards reflection point C. When it hits reflec- tion point C, it reflects to point E. When it reaches point E, it returns back to point A. This pattern continues until the direction of reflection point A, C, or E is changed. 4.3.3 Participant Control of Sound Spatialization Eight buttons on the Interaction Table correspond to the 8 reflection points. The panning controller accepts button press via a Phidget interface and when a button is pressed, the direction of reflection for the associated reflec- tion point is adjusted. Each press of the button rotates the reflection point direction by one in a counter-clockwise direction including pointing to itself. 69 Chapter 4. Echology Thus, by pressing the 8 Interaction Table buttons, participants can change the directions of sound spatialization in the soundscape. 4.3.4 Graphical Feedback Participants receive feedback of current reflection point directions and spa- tial positions of the sound signals through graphics overlaid on the web- cam video shown in the Interaction Table display (Figure 4.10). The pan- ning controller sends x and y coordinates for each sound signal and button press events to the graphics controller which generates graphics using Jit- ter’s OpenGL capabilities. The graphics, as detailed below, are intentionally kept simple so as not to detract from the live Beluga webcam feed in the background. Figure 4.10: Interaction Table display of webcam feed with graphical feed- back overlay. As introduced in Section 4.3.2 and shown in Figures 4.7, 4.8, and 4.9, reflection points are depicted by the row of eight circles with a pie slice 70 Chapter 4. Echology removed. Sound reflects in the direction of the pie slices wide end. A reflection point directed to itself is depicted by a doughnut shape, indicating that it will ‘catch’ sounds. Four circles of different colours move about the Interaction Table display. Each circle directly represents the location and amplitude of the sound sig- nal associated with one of the webcam video layers. The following colour mapping is used to differentiate the signals: Splash is purple, Play is yellow, Swim is green, and Deep is blue. The amplitude is mapped onto the alpha value of the circles, so that they fade in and out as the level of activity of the Belugas increases or decreases in the corresponding layer of the tank. When there is no activity and therefore no sound, the circle disappears. 4.3.5 Phidget Interface A Phidget [5] interface is used to detect button presses on the Interaction Table and to send an output signal to the lighting controller. The Phidget 8/8/8 is an inexpensive digital acquisition board that provide simple mech- anisms to connect and control various sensors and actuators to a computer via USB. It allows for 8 analog inputs and 8 analog outputs. We chose to build Echology with this board as it is easy to use and a C/C++ API makes for easy integration of its operations into other software applications. To access the input and output channels, a Max/MSP external Phidget object was written. A button press causes the object to send a signal in the panner controller software module, changing the direction of a reflection point. This same software module sends a signal to the Phidget outputs to turn on lights when a sound hits a reflection point. 4.3.6 Lighting Eight lights are suspended from the truss (as shown in Figure 4.11) at po- sitions corresponding to the positions of associated reflection point on the soundscape perimeter. The panning controller sends the x and y coordinates of the four sound signals to a lighting controller which turns the 8 lights on and off through the Phidget interface. The Phidget interface output is con- 71 Chapter 4. Echology Figure 4.11: Light hanging from truss speaker. nected to a lighting controller board (a Light-o-Rama MC-TB08 8 Channel Triac Board) that allows eight 110V light bulb sockets to be switched under computer control. Thus, when a sound hits a reflection point, the lighting controller momentarily lights the associated reflection point. 4.3.7 Interaction Table The functions of the Interaction Table buttons and display have been dis- cussed in the previous sections, but it is important to note the design of the table itself. The Interaction Table (Figures 4.12 and 4.13) is the focal point of Echology. It is placed in the center of the installation space and is designed to provide the aesthetics of an aquarium atmosphere and to create and inviting, collaborative space. Specific requirements of the table are to display video on the tabletop, to have eight buttons arranged symmetrically around the video display, and to have space to contain all wires, circuitry, and ideally, the PC controlling Echology. The tabletop is circular with sufficient diameter to allow a larger number of participants to comfortably stand around it and see what is going on. We chose large, glowing buttons to draw people towards the table and entice 72 Chapter 4. Echology Figure 4.12: Interaction Tabletop. Figure 4.13: Interaction Table with screen removed. pressing. The buttons, purchased from a supplier of arcade machine parts, Happ, are light sea blue, 3 inches in diameter and illuminated inside with a small DC lamp. When a button is pressed, the light turns off to provide 73 Chapter 4. Echology immediate, direct, visual feedback. The large buttons also fulfilled the design requirement of having an interface robust enough to withstand sustained use by the general public. The tabletop is a custom-designed, wooden board with eight circular holes for the buttons and one larger square cutout for the display. Four “L”-shaped blocks of wood affixed around the display cutout hold an LCD monitor in place. The blocks can be loosened and rotated to the side so that the monitor may be easily placed in and taken out. Four small clamps are used to fix the tabletop onto the table’s base. Rather than use a traditional four-legged table form and curtains to hide hardware placed under the table, a cylindrical-shaped base is used for a softer appearance. An oil drum provides this shape; it is also a sturdy base and can contain the Phidget boards and wires needed for button and lighting control. A hole drilled at the base of the drum allows power, ethernet, video, USB cabling, and wiring for the lighting controller to be fed out. The original plan was to also place the PC inside the drum but because of overheating issues this was not done. Instead, during NIME the PC was hidden under the atrium stage and for the study it was placed some distance away from the interaction space. Finally, the table base and top is covered in a fluffy white fabric for an alien, underwater aesthetic. The tabletop itself has a clear plastic cover to give the impression of a Beluga’s skin. It also has the functional property of protecting the monitor surface. 4.3.8 Musical Elements To enrich the soundscape musically, various sound sources including actual Beluga sounds and composed segments were selected. Although each sound source had to be elementally simple enough to achieve successful interaction and spatial perception, we also wanted the soundscape to be rich enough to give the audience the feel of playing with the Belugas underwater. Two categories of sounds are used in Echology. The first consists of short samples of Beluga whale voices received from the Vancouver aquarium 74 Chapter 4. Echology research team. The second category is a number of composed synthesizer tracks meant to create an aquatic atmosphere and provide continuity and context to the soundscape. The sound signals of the four layers in Echology’s soundscape are as follows: 1. “Splash” contains a synthesizer motif creating an aquatic atmosphere when a whale moves across the top of the aquarium. 2. “Play” loops several short samples of Beluga voices that are randomly triggered by the Beluga movements. 3. “Swim” consists of additional Beluga voice samples accompanied with the drone of synthesizer sound. 4. “Deep” contains low frequency mass to create a drone. We selected high frequency Beluga voices since they provide better cues for spatial perception. Atmospheric background synthesizer sounds are de- signed to give continuity to the entire soundscape. In general, there is enough variation in the soundscape to be engaging and interesting because of the variety of motion of whales in the aquarium. 4.4 Desired Echology Experience 4.4.1 Participant Experience Summary Echology allows participants to experience a sonic aquarium while feeling linked to the live, organic and mesmerizing movements of the Beluga whales. Participants can play with the spatialized sounds initiated by Beluga whale motion and watch the webcam feed in the table’s display. Using the buttons, participants control the path of sound overhead, or if desired, they can just watch and listen to the sounds move along their predetermined course. The melon of the Beluga (the fleshy part on the top of their head) inspires this use of redirected sound. We intentionally made the interaction simple, so that 75 Chapter 4. Echology participants need only focus on the spatial aspects of the sound rather than the tonal qualities. The imagery of Beluga whales swimming blends with the sound spatialization and visualization to make a rich, playful, and enjoyable mediascape. The whales’ play results in movement patterns that are fairly repeatable, but unpredictable and organic. A participant can listen to the soundscape from anywhere inside the atrium; however, the “sweet-spot” is at the Interaction Table located in the centre. 4.4.2 Connecting with the Belugas Using a webcam feed of live Belugas raises interesting ethical questions we have tried to address in the artwork. Though they are in an aquarium envi- ronment, the Belugas are wild animals that cannot be fully controlled. They are there for us to learn from them, but demand proper respect. For this reason, we gave participants only partial control over the sounds. Sounds fade in and out according to the activity of the Belugas in each layer; if there is no Beluga activity at all for a certain amount of time, the installation will become silent until the Belugas return. Only when there is a significant level of activity in all layers will all the sounds be present simultaneously. This aspect of the interaction encourages participants to reflect on the Beluga whales as living creatures seen in real time. As such, participants are not able to demand performance from the whales, but must patiently wait for them to swim into view. 76 Chapter 4. Echology 4.4.3 Participant and Art Object Roles To create the above described experience, it is desirable for the Echology art object and participants to play out their roles as detailed in Tables 4.2 and 4.3. Table 4.2: Desired manner with which Echology plays out its roles as an art object. Participant Interaction Trigger Trigger interaction with an approachable space and large buttons that afford being pressed. Interaction Guidance Support interaction with limited button press control and simplistic mappings between web- cam feed, button presses, visual elements, and the soundscape. Representation Create an atmosphere representative of the space inhabited by Beluga whales and present the Belugas as real living creatures that par- ticipants may admire and connect with. Communication Facilitated Create a social space for collaboration and communication among participants; give par- ticipants the sense that they may play and communicate with the Beluga whales. 77 Chapter 4. Echology Table 4.3: Desired manner with which participants of the Echology artwork play out their roles. Basic Interaction Understanding relationship between button presses, graphical feedback, and the sound- scape; being able to distinguish between dif- ferent sound channels and the positions of sounds being spatialized. Imaginary Contribution Realizing that the Belugas are those living at the Vancouver aquarium; knowing that the video is live and aquiring a sense of connection to the Belugas. Creative Contribution Experimenting with different patterns of sound spatialization. Involvement in a Social Space Creating a conversation space around the In- teraction Table, hanging out, communicating with strangers, and collaborating to create patterns of sound spatialization. 4.5 Summary This chapter has discussed Echology, an interactive sound and video in- stallation designed for collaboration, simplicity, and approachability. The Echology system as designed and implemented for presentation in the Open Media Environment was detailed. The intended interaction experience and expectations for art object (the Echology system) and participant roles are presented. This frames motivations behind the participant experience stud- ies of Echology discussed in the next Chapter. 78 Chapter 5 Studying Participant Experience of Echology This chapter summarizes informal observations of participant interaction with Echology during its first public presentation at a conference. This experience confirmed that initially established expectations for participant experience as discussed in Section 4.4 were mostly achieved. Yet still, ques- tions remained about factors that may have influenced participant engage- ment and the levels of participant understanding that had been achieved. Section 5.1.2 poses these questions and then Section 5.2 details the design of a laboratory-based study conducted to more rigorously investigate them. The study methods used are based on those of the cases reviewed in Chap- ter 3 with the added use of orientation media for comparative analysis. Section 5.3 presents study results by first summarizing resulting participant experiences and then providing interpretations and implications of these re- sults from two perspectives: an artist’s and an HCI practitioner’s. Finally, Section 5.4 reflects on the study experience itself; presenting insight into the laboratory setting’s effect on participant behaviour and then reviewing the use of orientation media and formal methodology in general. 79 Chapter 5. Studying Participant Experience of Echology 5.1 Informal Observations at NIME Echology was exhibited for the first time in a public setting during the 5th International Conference on New Interfaces for Musical Expression (NIME ’05) in Vancouver, Canada [36]. It ran for one day, during which time substantial feedback from conference attendees and members of the gen- eral public was received. More than one hundred people were informally observed interacting with Echology. The next section summarizes general observations and interpretations made of them. The following section raises questions that remained after digesting the NIME observations. Figure 5.1: Echology visitors watching Belugas at NIME. 5.1.1 Summary of Observations and Feedback Received There was a prominent pattern in visits to the installation space. A partici- pant would walk up to the table, glance at the display, and then look at the overhead lights and speakers. He or she would then focus on the display and either press a button to receive a reaction, or watch what other people were doing. Most people asked questions about the installation, how it worked, how it was made, and so on. After discussing the installation itself, the conversation often opened up to other topics or participants would spend a few quiet moments by themselves, enjoying the Belugas and soundscape. We achieved the quality of approachability by creating an inviting space. Participants were immediately drawn to the Interaction Table upon walking 80 Chapter 5. Studying Participant Experience of Echology into the installation space. They did not hesitate to try pressing the buttons. Moments of silence when there were no whales did not feel awkward amongst participants at the table. They would patiently wait in a manner similar to a guest at the aquarium quietly waiting for an aquatic creature to swim into a more visible area. They were comfortable with being silent for a few minutes or spent time conversing about the Belugas, easily breaking the silence when desired. This allowed some participants to stand around the table and chat for lengths of time up to 20 or 30 minutes. Thus the piece worked very well as a “calm” technology [99]. Also, without explicit instruction participants chose a button to stand in front of, and subsequent use of that button was solely by that participant. That button became their “territory”. Territoriality has been consistently observed in research of tabletop displays for computer-supported collabora- tive work [88] [94] and an effect of it is that it helps people mediate their social interaction through the very act of laying claim to a space [95]. For some, the connection between the Belugas and the mode of interac- tion was not well understood. The abstraction between how Belugas com- municate, their actions, and what participants could do was not obvious to them. Usually an attendant had to explicitly state that the Beluga webcam images were indeed live and not prerecorded. It is difficult to determine the importance of the audience understanding that the Belugas were live and how much that understanding changes the metaphor. Because of the live nature of the Beluga camera, some people wanted to take the next step – to be able to communicate directly with the Belugas. Many people asked, “can the Belugas hear the music too?” 5.1.2 Questions Remaining The above experience proved Echology to be approachable, a provoker of curiosity and exploration in people, as well as collaborative in that it sup- ported a social space for communication among participants and joint play with the Belugas. At the same time, we were not convinced that our goal of creating a simple and easy to learn interface had been achieved. Questions 81 Chapter 5. Studying Participant Experience of Echology remaining were: 1. If an artist were not available to explain how interaction and Echology worked, would the art object alone be able to support participant inter- action and understanding? Furthermore, how much was engagement based on figuring the system out rather than enjoying the soundscape and visuals, collaboration with other participants, or feeling connected with the Beluga whales? 2. How much was enjoyment based on the Beluga video being from a live webcam feed? Would it matter if it was pre-recorded? 3. How did interacting with Echology collaboratively differ from experi- encing the piece alone? This motivated a need to investigate Echology in a more in-depth man- ner than what could be conducted at NIME, or in any other exhibition setting. With a more formalized approach in a controlled laboratory setting participant experiences could be investigated in greater depth. Variations in experience could be administered as media given to the participants to orientate them towards understanding of different functional and conceptual aspects of the system. Conducting a study in a controlled laboratory setting and using orientation media as described in Section 3.4, we sought to ob- serve differences in interpretation and understanding between participants with and without: 1. An explanation of interaction with Echology to address the first ques- tion. 2. Knowledge that the Beluga webcam feed is live to address the second question. 3. Another person to experience Echology with to address the third ques- tion. 82 Chapter 5. Studying Participant Experience of Echology 5.2 Laboratory-Based Experience Study A study with 8 session types was designed to observe the effects of the above three points. The session types were based on combinations of three variables: whether the participant received instructions or not, whether the participant was told the Beluga feed is live or prerecorded or not and whether the participant viewed the live webcam feed or a prerecorded version (Fig- ure 5.2). Furthermore, sessions were conducted with either a single partici- pant at a time or with participants in pairs. Figure 5.2: Eight session type study matrix based on variables: (a) whether participant receives instructions or not, (b) whether the participant is told the Beluga feed is live or prerecorded or not, and (c) whether the participant views the live webcam feed or a prerecorded version. All participants experienced Echology without the presence of the ex- perimenter so that there would be minimal chance to ask questions and re- ceive information influencing their interpretation. Also, an effort was made to maintain consistency in dialogue used and information given across all sessions. This care was taken from when the experimenter first met the participants to during a final interview, at which point conversation could become more open. 83 Chapter 5. Studying Participant Experience of Echology 5.2.1 Orientation Media Two classes of orientation media were created. The first class addressed the question of how a person’s understanding of interaction affects their experience of the artistic concepts and aesthetics. Here orientation media in the form of a set of cards with instructions as shown in Appendix A.2 was used. Participants in the “instructions” session types were given these cards to reference while viewing Echology. Participants of the “no instructions” sessions were not. The second class of orientation media consisted of a short description of Echology written on a card placed on the Interaction Table. One of three possible cards (with descriptions as shown in Appendix A.3) were placed for every participant, the descriptions on each card being almost identical with the exception of one part specifying one of the following: 1. Echology uses a LIVE video feed of the Beluga whales. 2. Echology uses a PRERECORDED video feed of the Beluga whales. 3. Echology uses a video feed of the Beluga whales. Echology was accordingly presented with the live feed or a prerecorded version as specified in the first two cases, or in the third case, either. 5.2.2 Participants A total of 26 paid participants, 14 women and 12 men, were recruited via an Internet-based experiment recruiting system used within the University. People were invited to participate in a “User Interface and Experience Study of ‘Echology’: an Interactive Spatial Sound and Video Artwork”. The entire recruitment advertisement is shown in Appendix A.1. Ages ranged from 19 to 36 with an average age of 23.7. All Participants except for one person were students at the University of British Columbia, the majority in the engineering and computer science departments, and used computers on a daily basis. Half of the participants had no experience with interactive artworks while among the other half with some experience, 84 Chapter 5. Studying Participant Experience of Echology only 3 people had had several experiences. When asked about video game experience and musical education, most participants indicated having some or more while only 2 people had no experience playing video games and 6 people, no musical education. Participant data listed according to session type can be found in Appendix B.1. Sixteen people participated in the study alone while 10 people partici- pated in pairs resulting in 5 groups of 2 people. In all 5 groups, participants had not previously met their partner. 5.2.3 Setting The study was conducted in a usability lab on campus. Two rooms were used: a small room inside of the usability lab where participants were met, orientated, and later interviewed, and a larger, mostly empty room, outside of the usability lab area inside which Echology was installed. The relation- ship between the two rooms is shown in Figure 5.3. The small room was sparsely equipped with a desk, 3 chairs, a monitor for viewing video, and a laptop used for audio recording of the interviews. For the installation space, we had considered showing Echology in the Open Media Environment, which neighbored the building housing the usability lab, but decided against this for a number of reasons. One being to avoid participants feeling that they would be watched by people passing by in a public space and another being that the studies would be conducted over the course of more than a week but for security reasons, Echology would need to be taken down every evening. Instead Echology was placed in a secure room, empty save for some scaffolding and a few large screen displays. Echology was installed as shown in Figure 5.3. A video camera was placed in the room to record participant interaction. Notable differences between the laboratory and the NIME installations are that, first of all, there were only 6 speakers available for use in the laboratory installation. The eight reflection points were still arranged sym- metrically around the soundscape but instead of each point being in the same position as a speaker, most lied between speakers. A second difference 85 Chapter 5. Studying Participant Experience of Echology ~ 25 metres between rooms Testing room Cam Figure 5.3: Study setting: Echology installed in room on left-hand side, interviews conducted in room on right-hand side was that during the NIME installation, the lights did not seem very useful for providing feedback of the sound paths. Instead their blinking appeared difficult to follow, almost random, and distracting. They were not used for feedback in the laboratory installation. Four of the eight lights were placed around the space and kept on the entire time to create an aquarium-like atmosphere. 5.2.4 Study Procedures Orientation Participants were met in the usability lab and given a short introduction to the study. They were told that they would be taken to another room to see an interactive art installation piece, that their experience would be recorded on video, and that afterwards, they would be interviewed about their experience. It was stressed to them that this was not a “task” and they were not being “tested” on performance or understanding. They were then given orientation media dependent on which session type they belonged to. All participants received one of the Echology introduction cards and some received the set of instruction cards. 86 Chapter 5. Studying Participant Experience of Echology Viewing After the orientation, participants were taken to view Echology. They were told that they were free to touch and play and could spend as much or as little time in the room as they wanted. To avoid the close presence of the experimenter and concern about when to return, participants were given a walkie-talkie. They were to use it to notify the experimenter when finished viewing Echology. After verifying the participants understood how to use the walkie-talkies, they were taken to the installation room. To avoid initial reactions directed to the experimenter, the experimenter did not follow participants as they entered the room. Interview After receiving notification that participants were finished viewing Echol- ogy, the experimenter came and took them back to the initial experiment room. They were seated in front of a computer monitor from which they could watch the video just taken of their experience. While watching the video, an open-interview guided by questions listed in Appendix A.4 was conducted. Following the interview, participants answered a short question- naire (Appendix A.5). Video-Cued Recall The study was initially designed to use the video-cued recall method. Partic- ipants would be able to view a recording of their interaction with Echology during the interview. During pilot studies it was found that the video did little to help participants recall specific moments during their experience. This was explained for by the low range of participant physical interaction and the repetitiveness of the art object’s states. Concluding that video-cued recall would not be very helpful, further studies were conducted without the method. Still, video was prepared for viewing in the event that it could support the interview. 87 Chapter 5. Studying Participant Experience of Echology 5.2.5 Data Collected The following forms of data were collected: • video footage of participants at the Interaction Table • transcribed audio recordings of interviews • questionnaire results including general demographic information, re- sponses to multiple choice questions, and free writing responses Interview transcripts were analyzed to pinpoint areas of participant mis- understanding and frustration in order to make comparisons between sub- jects with instructions and those without. It was also desirable to determine the different levels of participant engagement as well as interpretations and metaphors used to describe the experience. After reading the transcripts for a first time, a classification for statements of interest was developed. During subsequent readings, statements were placed among the following categories: • Echology as a game and Echology as art includes statements re- lated to discussion held with all participants on their views of whether Echology was more a game or more art, as well as random statements making reference to Echology being a game or an artwork. • Understanding of Echology Interactivity includes statements re- vealing participant mental models of interactivity and Echology’s func- tions, as well as expressions of confusion and frustration or confidence in understanding. • Sensation and Enjoyment includes statements related to Echol- ogy’s atmosphere, musical elements, metaphors attributed to the art- work and general expressions of enjoyment. • Connection with the Belugas and the Live Webcam Feed in- cludes statements about the Beluga whales and the webcam feed. 88 Chapter 5. Studying Participant Experience of Echology • Participant Expectations and Technique includes statements re- lated to participant expectations of the experience and the purposes and paths they create in interaction. • Breakdowns includes statements made during discussion of break- downs that occurred during a small number of participant experiences. • Collaboration includes statements related to collaboration. All participants were given a pseudonym for easy identification. These names and relevant participant information can be found in Appendix B.1. Questionnaire results are listed in Appendix B.2 and statements from the interview transcripts are listed in Appendix B.3 according to the above classification system. 5.3 Study Results To address the questions posed in Section 5.1.2, study results are discussed considering the following facets of participant experience: visual and audi- tory enjoyment, understanding of Echology interactivity, connecting with the Belugas, collaboration, and participant tendencies. 5.3.1 Visual and Auditory Enjoyment Echology was successful in creating an immersive, aquarium-like atmosphere through musical composition, lighting, and the Interaction Table and space design. Participant impressions were revealed in responses to interview ques- tions on their first impressions of Echology, what they liked and did not like about the experience, and what they may have related the experience to (Ap- pendix B.3, Table B.6). They were also apparent in questionnaire responses to the request for 3 words that they could relate to their experience (Ap- pendix B.2, Table B.3). Below are keywords extracted from the responses to summarize these impressions: • comfort, calmness, relaxing, and peaceful (Anne, Carl, Beth and By- ron, Ben, Barbara, Ambrose, and Claude) 89 Chapter 5. Studying Participant Experience of Echology • the unknown, curious, mystical, mysterious (Brian, Chris, Carl, An- drea, Byron, Ben, Claude, and Arthur) • immersion (Chris: “like a total surrounding environment that you are actually involved in”, Barbara: feels in the middle of where sound from, Cory: “feel like you are actually in this place”) • aquariums and underwater (Andrea, Charles, Denise and David, Cory, Cassandra) • a different space or world all together (as opposed to the laboratory room) (Carl: feels separation from outside world, Andrea: like star wars in spacecraft, Byron: “environment is very fantasy”, in a for- est, feels removed and far away, Ambrose: “quasi-futuristic”, Claude: ghost house, Denise: space-like) 5.3.2 Understanding of Echology Interactivity Interview statements expressing correct and incorrect beliefs of how the system worked, questions remaining, and feelings of confusion or frustration (Appendix B.3, Table B.5) were considered to assess participant understand- ing of Echology interactivity. From these statements general understanding of the following aspects of Echology was extracted for each participant: the control of reflection points with the buttons, the movement of graphical balls in relation to the reflection points, the relationship between the sound and the graphical balls, the movement of sound in the soundscape, and the relationship between Beluga whale movements and sound. Changing reflection point direction with buttons All participants understood immediately or quickly that they could interact by pressing the buttons. Almost all participants saw that when a button was pressed, the associated graphical reflection point would change and that pressing a number of times would cycle the point through its seven reflecting directions and donut-shaped phase. They understood that this was the 90 Chapter 5. Studying Participant Experience of Echology extent of their “direct” control. Five subjects expressed some belief that pressing a button could trigger a sound (Arthur, Beth, Byron, Deborah), or that its control was extended to changing the characteristics of the sound (Ben). Reflection point direction defining paths of graphical balls All participants understood that reflection point directions controlled the path taken by the graphical balls. They also saw that if a reflection point were pointing to itself, it would catch the sound. This last point caused some indirect frustration for two subjects (Andrea, Byron) as it gave them the impression that an objective of Echology was to catch the balls. After catching balls in a reflection point and then pressing the button once more, balls were released to continue their original movement. They were left wondering how they could really catch the sound. Movement of sound in soundscape and graphical ball representation Almost all participants could hear the movement of sound around the sound- scape and understand that the graphical balls represented this moving sound. Confusion arose in the fact that there were multiple balls with different colours and their appearance and transparency varied (Anne, Arthur, Aman- da, Carl). A few participants were able to distinguish that the different coloured balls appeared when different types of sound played, but others were not able to distinguish the separate sources of sound as well. Relationship between Beluga whale movements and sound A generally satisfactory level of participant understanding broke down when it came to understanding how the movement of Beluga whales controlled the volume of sound. Only a few participants perceived the sound being related to the Belugas (Andrea, Ambrose, Cory, Chris, Charles, Deborah). 91 Chapter 5. Studying Participant Experience of Echology 5.3.3 Connecting with the Belugas Feelings of connectedness with the Belugas were probed during the inter- views by asking directly what was thought or felt about the Belugas, and indirectly by posing the question: “If you could talk with the Belugas at the aquarium, what would you say to them?” (Appendix B.3, Table B.7) A questionnaire question asking participants about what they believed the general artistic concept of Echology to be also revealed to what extent the Belugas left an impression on them (Appendix B.2, Table B.3). The resulting number of participants who were left with impressionable thoughts on the Belugas was quite low. Some noted that they did not pay attention to the Beluga video, as they were concentrating on the sound or graphics (Andrea, Deborah, Ben, Barbara, Claude, and Arthur). Other participants were annoyed by the video’s slow frame rate and low resolution (Anne, Chris, Carl, Beth, Ambrose, and Denise), with two participants (Carl and Beth) remarking that they were unengaged by the video as it appeared to be a “repeated sequence of images”. There were no remarkable differences in attitudes towards the Belugas between participants told the feed was live and those not. Still, there were two cases of particularly strong reactions to the live feed that are of interest. In Ben’s case, he became aware of the feed being live only after his expe- rience, during the interview. He was quite amazed by this and it changed his perception of the artwork as a whole: “It’s really interesting, that in- teresting. . . that’s real whales over there (. . . ) omigosh, I didn’t realize. I just thought it’s like a computer game. . . ” In Burke’s case, he was already aware that the video feed was live but this was further reinforced when the video froze due to network problems. Interestingly, he was fairly accepting of the breakdown and said that he understood how the webcam feed might freeze up. He later mentioned wanting to go to the aquarium sometime to see the Belugas with his own eyes. Ways in which other participants connected with the whales were by imagining that they were swimming with them (David), wondering if the whales are bored in their habitat (Charles), wanting to touch the whales 92 Chapter 5. Studying Participant Experience of Echology (Barbara), and wondering if the real whales could be fed through the artwork (Carl). 5.3.4 Collaboration Behaviour observed in the five cases of paired participants was different from the relaxed, informal social space observed during NIME. Viewing the video recording of their experiences, the pairs seemed to not mind each other’s company and there was some joking around and laughter, but for the most part, they concentrated on their “task” and most discussion centered on understanding how Echology worked. This focus on “task” is discussed when considering participant behaviours as study subjects in Section 5.4.1. There also seemed to be less collaboration in creating patterns of sound movement, apparent in behaviours such as one person pressing many buttons while the other only watched, or a participant reaching across the table to press a button directly in front of the other participant instead of communicating that they would like the button to be pressed. The atmosphere of the study setting likely had high influence on collabo- rative behaviour and may have actually contributed to creating a less social, personal space. Many participants expressed, in a positive way, feelings of solitude and loneliness. Even among the pairs, one participant expressed: “I feel that I’m alone, but I’m not lonely” (Byron). The room in which they viewed Echology was dark and quite isolating in comparison to the open and public setting of the Open Media Environment during NIME. The space created experiences that were highly individual and introspective. 5.3.5 Participant Behaviour Tendencies Participant behaviour was shaped by tendencies for the participant to inter- act in modes that surfaced in discussions of expectations, actions taken, and enjoyment (Tables B.6 and B.8 in Appendix B.3). These modes are play, seeking goals, directions, and performance indicators, figuring things out, and passive spectatorship. Tendencies towards one or more of these modes influenced the degree and manner with which participant sought control in 93 Chapter 5. Studying Participant Experience of Echology the artwork and consequently, what engaged them. Tendency to Play A participant with a tendency towards play sought and received enjoyment from activity: acting on something and receiving a reaction. Their sense of control was in actions available and the corresponding changes in the system: “. . . if you play with these buttons and you can actually change the sound, how the system sounds, you can like, if I do this I can affect something. I can enjoy what I did, if I find some particular setting I can get better sound effect” (Carl). In Echology, this was the action of pressing buttons to cause changes in the reflection points, thus influencing the motion of sound in the soundscape. A participant seeing the opportunity to control sound by creat- ing patterns in its movement or by catching it could amplify this amount of control. Still, for simplicity, control in Echology is limited and a participant with no other tendencies could lose interest quickly. David said that he liked pushing buttons and looking at the screen but, “it would be better if the screen has something more different, more interactive, more interesting than the pacmen turning around (. . . ) Cause you have to wait for the balls to get to the point where the sound is resurrected right? (. . . ) When there’s no sound you get frustrated.” Another participant (Carl) wondered why he didn’t have more control over the sound or over the Belugas: “. . . you adjust the orientation of those reflection points maybe it is easier for them to get food then you won’t, sounds won’t be sad and you get cheerful sounds”. Tendency to Seek Goals, Directions, and Performance Indicators For participants with this tendency, lack of direction or purpose imposed a sense of lack of control and thus, reduced enjoyment. Audrey, who experi- enced Echology with Amanda, said, “. . . it was hard to just sort of. . . just enjoy the balls and we were trying to figure out some sort of point.” She expressed frustration not being able to see this point. This tendency is strongly influenced by the context of the study. One participant (Cassan- dra) sought direction because of the study setting and, not knowing what to 94 Chapter 5. Studying Participant Experience of Echology do, she felt confused. Other participants viewed Echology as more a game but to be completely a game, they felt there should be rules or goals. Four participants (Anne, Beth, Carl, David) related to Echology as an interactive exhibit that could be used for education on the Beluga whales. Here expec- tations were in the form of learning goals and Echology, lacking informative content, did not fulfill their expectations. Tendency to Figure Out Participants with this tendency found enjoyment in understanding the art object. They felt control in understanding either how they could interact and how the art object would respond or in understanding how the art object worked from a technological perspective. An example of the latter is Burke who was interested in the programming technology for sound spatialization and Arthur, “I like to know how stuff is working so that when I first came into the room instead of enjoying it as art I just, I figure out what does this do. . . ” For these participants, it was not so important to have goals built into Echology because they created their own goal to figure the system out. Steps towards understanding resulted in satisfaction and enjoyment. “I want to know how it works but, I don’t know how it works and then I just start playing and then once I know a little bit more I start to enjoy it because I know a little bit more of it.” (Arthur) On the downside, participants limiting themselves to this satisfaction would possibly find no further enjoyment in interaction after figuring the system out. Tendency to be a Passive Spectator These participants took enjoyment in what could be sensed immediately, vi- suals and sound, and less in the interaction. One person (Anne) mentioned passive enjoyment explicitly: “It would be nice to sit in the middle and watch a movie”, and another (Denise) hinted at a preference for passive ap- preciation: “(speaking about interactive art) . . . it’s kind of frustrating too, if you don‘t know what it’s meant to do or what you can do with it. You can press buttons or do things on the screen and nothing happens or, I think it’s 95 Chapter 5. Studying Participant Experience of Echology kind of frustrating when you try to interact so I just watch.” While partic- ipants could find enjoyment in Echology’s atmosphere and soundscape, the artwork’s total expression would not be experienced without appreciation for the interactive elements. This could especially cause disappointment for someone actively searching for Echology’s expressive elements as an artwork. It is suspected these tendencies were strongly influenced by a view of whether Echology was more an artwork, or more a game. This was inves- tigated by directly posing the question “Did you see Echology more as an art piece or as a game or neither?” Responses are summarized in Appendix B.3, Table B.4. Ten participants felt it was more a game, 4, more art, 3 re- sponded neither and 2 felt it was a mixture of both. Many participants who felt Echology was more a game said so with some apprehension. It seemed like a game because of the Interaction Table buttons and screen and because you could change things. Still, it could not be a game completely as it lacked directions and goals. A reason for considering Echology as an artwork was that it was “artistically laid out and involved the senses” (Ambrose), but participants could not say it was art with immediate confidence, possibly because they could not relate it to art that they were familiar with, this being something that you passively look or listen to (Barbara), to “observe it and then make an analysis” (Cory), and that has “meaningful things that can be expressed or displayed” (Cassandra). The strong leaning towards Echology being a game indicates that par- ticipants likely had expectations of game-like qualities for their experience. Upon entering the room, they knew they would be able to play and inter- act, but they probably also expected challenges, rules to figure out, goals, or measures of performance like a score. As an artwork, Echology lacked in these thus some participants felt confusion or that something was missing. As summarized in Section, many enjoyed the atmosphere and sound- scape – elements that could be taken in passively. They considered these the “artistic” elements of Echology. But to fully enjoy the expressive meaning of Echology, participants would have to transcend notions of traditional art where expression is through a fixed art object. 96 Chapter 5. Studying Participant Experience of Echology Other influences to behaviour tendencies were the study settings and the fact that participants were subjects in a study, as well as biases rooted in to participant backgrounds. This is discussed further in Section 5.5’s review of the study methodology. 5.4 Interpretation of Study Results The results discussed in Section 5.2 reveal facets of participant enjoyment in the Echology experience as well as areas where there was a lack of participant understanding. These results can inform future versions of Echology but the manner in which they do depends on how they are interpreted. This section highlights the key points taken from the results for interpre- tation. They are not immediately labeled “problems” as whether or not it is a problem depends on the interpretation. Each point is interpreted from both the traditional HCI viewpoint and the interactive artist’s viewpoint and suggestions for changes or considerations in future versions of Echology are made. Result: (Section 5.3.2) Not a single participant made a clear relationship between the types of sound heard and movement in different areas of the webcam feed. Furthermore, some were confused by the use of four different coloured graphical balls to represent the sound signals. Many could distin- guish that there were different “types” of sound, but were not able to relate it to anything they saw. Suggestion: From an HCI viewpoint, indication of the webcam feed image layers, as well as of the relationships between layers, associated sound, and coloured ball representation should be provided. Indication could be given, for example, by a periodic graphical display of layer divisions with text labels and colour coding. From an artist’s viewpoint, it is suggested that some level of ambiguity is maintained. Participant understanding of the relationships between layers, sounds and balls will not necessarily promote connectedness with the Belugas and the sense of wonder and curiosity intended for. If 97 Chapter 5. Studying Participant Experience of Echology these relationships are clear participants may lack space in which they can question and interpret the actions of the Belugas (in the artwork). Still, participants should be given some cue that the Belugas are controlling part of the installation. This suggests tuning the video processing and sound control to make the relationship between Beluga movement and sound more obvious. Result: (Section 5.3.3) Six subjects expressed that they did not pay at- tention to the Beluga video because they were focused on the sound and graphics. The Belugas were often referred to as being “in the background” and as a result, connection with the Belugas became only a “background” aspect of Echology. Suggestion: From both viewpoints, the design of the graphical feedback overlay on the Beluga webcam content should be reconsidered. The Beluga video could be brought into the foreground by not overlaying the graphics, or by highlighting and bringing attention to the webcam feed in some way. It may be tempting to use a higher quality video feed but from an artist’s viewpoint, low video quality is more strongly associated with live video thus it is suggested that even if higher quality were available, it should still not be used. Result: (Sections 5.3.3 and 5.3.5) Knowledge that the Belugas were live had a great effect on a participant’s interest in and connection to the Belugas and made the participant more accepting of the limited amount of control they had on the sound. Suggestion: From an HCI viewpoint, cues that the Beluga webcam feed is live should be provided. Subtle cues could be provided by, for exam- ple, showing the updated date and time generated by the webcam feed, or displaying information specific to the Belugas at the Vancouver Aquarium such as their given names: Imaq, Kavna, Aurora, and Qila. From the artist’s viewpoint, while agreeing with the HCI viewpoint, it is still suggested that 98 Chapter 5. Studying Participant Experience of Echology some care be taken in the manner in which cues are provided. Explicit cues like the ones suggested above might promote the view of Echology as an educational interactive exhibit and increase participant expectation of information provided. Subtler cues could be provided by, for example, re- designing the Interaction Table or installation space to be a replica of the space where Belugas are watched at the Vancouver aquarium. Result: (Section 5.3.5) Echology did not give enough control to fully en- gage a participant with a tendency for play. It also did not support partici- pants seeking purpose in interaction. Suggestion: More engaging control of sound should be provided. Control over variables of sound other than its motion could be given to the par- ticipant, allowing for a richer space of play and expression. For example, participants could have some control over the type of sound, or generation of sound that is then modified by the Beluga whales. Or they could have full control over a set of sounds, while the Belugas would control another set. Play between participant and Beluga would occur in the mixture of these sounds. From an HCI viewpoint, purpose in interaction could be designed in the form of connections with the Belugas: through interaction participants could be generating virtual food for the whales or, as reward for reaching some end goal, participants could control a virtual character swimming on screen with the Belugas. From an artist’s viewpoint, providing a specific purpose to Echology could limit interaction paths and interpretations to ones directed towards achieving the goal, thus reducing participant creativ- ity and reflection. 5.5 Discussion of Study Methodology 5.5.1 Participant Behaviour as a Study Subject As discussed in Section 3.4, there is general acknowledgment that experiment settings have an effect on participant behaviour. In this study it was found 99 Chapter 5. Studying Participant Experience of Echology that the root of much study setting-influenced behaviour was a participant’s desire to “perform” well. Even though it was stressed that there were no expectations or right and wrong answers, a number of participants still made remarks indicating some sense of obligation to do well. Byron, receiving no instructions, suggested that instructions would have been good so that he could “perform better”. Charles read the instructions he received 3 or 4 times, telling me that he was trying to find “something like a clue about your questions”. Participant behaviour was further influenced by belief that the study would be a usability or user study. Many participants had previously par- ticipated in one or more user studies conducted by various HCI practitioners who use the same Internet-based recruiting system. These participants had expectations of how user studies were conducted based on study designs commonly used. As Arthur remarked, “. . . all of my studies, they are all like that. Like, one person makes a program and I have to like, play with it or like, see which one does better or something. Lot’s of them have to do with graph, plots, and like, also personality stuff.” Finally, it should be acknowledge that participant background likely played a strong role in resulting behaviour. As mentioned in Section 5.2.2, the majority of participants were students in the engineering and computer science departments. Some of the behaviours discussed in this section, such as focusing on the functional aspects of an artwork, could be explained for by the predisposition of these participants to be interested in technology and how things work. Considering the above influences, prominent types of behaviour that were observed are as follows: Participants focused on functional aspects of the artwork over its content: Many study subjects indicated their main goal being to figure out how things worked. The advantage of this was that they were likely to attempt exploring all aspects of interaction and it could be easy to observe what could not be figured out. The disadvantage was that participants spent less time reflecting on artistic qualities of the work. 100 Chapter 5. Studying Participant Experience of Echology Participants were prepared to express what they did and did not understand, as well as their personal level of comfort: This be- haviour was advantageous for learning how people approached and tried to figure the artwork out but on the other hand, they were not prepared to express opinion on or critique Echology as an artwork. Participants expected a task, clear instructions for that task, and possibly some measure of performance: Some participants expressed confusion about what they were supposed to do with Echology once left in the room and not about Echology itself. Anne said she “should’ve checked what Echology was before” coming so that she could have a better sense of what she was supposed to do. Used to user study designs that lead them through tasks step by step, they felt uncomfortable having to self-direct their experience. This made it somewhat difficult to distinguish a participant’s confusion or discomfort as caused by the artwork design, or by what was expected from them in the study. Participants were more likely to work through a breakdown: If something appeared to not be working properly, reasons attributed were: the art object – in which case participants were confident that the experimenter could fix the problem, something they did wrong – in which case they may have felt obligation to work through the breakdown, and as part of the study design – in which case they may have thought their behaviour in the face of a breakdown was under observation. The latter was a tactic used in the Façade study. However, care must be taken not to consider a participant’s ability to work through a breakthrough as overly positive because in a regular setting, participants do not have the same performance obligations and tolerance and would likely quickly end their experience. Participants in groups maintained individual agendas: As men- tioned in Section, collaborative behaviour observed in the laboratory study was remarkably different from that observed at NIME. NIME was a public space and a sociable event. The laboratory study was closed-off and 101 Chapter 5. Studying Participant Experience of Echology the participants, taking on the role of study subjects, were focused on their own performance. This suggests that the laboratory-based method was not suitable for studying informal collaboration and socialization around the artwork. Though if the study had been designed in a manner assigning each participant a role to play within a group agenda, it is possible that social dynamics could be better observed. 5.5.2 Orientation Media Review To review, orientation media was used to distinguish the effects that understanding how the system works as well as knowledge of the Beluga webcam feed being live as opposed to pre-recorded had on a person‘s overall experience. Though after reviewing the data, there were no distinguishable effects across groups with and without either form of orientation media. In all session types participants had varying levels of enjoyment and observed connection with the Belugas. It is considered that this is because the extent that a participant will be receptive to the orientation media is dependent on that participant’s tendencies to want to figure things out and to expect instructions and goals. The orientation media did help reveal elements of the system for which participants without instructions could eventually formulate mental models of and elements that were confusing regardless of whether or not a partic- ipant received prior instructions (such as the relationship between Beluga motion and sound). 5.5.3 Benefit of In-depth Data Collection Adding to benefits exemplified by the case studies in Chapter 3, the Echology study demonstrates how a protocol for categorization of data to analyze experience can be developed (Section 5.2.5). This protocol is an expression of the intended aesthetic pursuits in Echology created by making repeated approaches to the rich data that was collected. It would not have been possible to develop the protocol from the informal observations made during NIME. 102 Chapter 5. Studying Participant Experience of Echology These protocols are important for framing and justifying interpretations made of study results. They are similar to the heuristics that inform user studies but are often arrived at only after study results are reviewed. Excep- tions are cases like the Iamascope and Façade studies that use pre-existing protocols. Another benefit was achieved through the tendency for participants as study subjects to make a large effort to figure the system out. In-depth data collection resulted in multiple accounts of approaches taken by participants to understand the artwork and the metaphors and mental models they used in the process. 5.6 Summary This chapter began with a summary of informal observations of participant interaction with Echology during its first public presentation at a confer- ence. To address questions remaining after this event a laboratory-based study was designed and executed as detailed in Section 5.2. The results of this study were summarized and interpreted in Section 5.3 and Section 5.4 discussed the study methodology and resulting experience itself. Observa- tions of changes in participant behaviour due to their roles as subjects in a study and implications these changes have on using formal methodologies for studying participant experience are discussed. The orientation media is shown to be successful in revealing elements of the artwork that, given explicit instructions or not, still cause confusion among participants. 103 Chapter 6 Conclusion In this chapter the problems introduced in Chapter 1 are restated and con- tributions made in this thesis work towards addressing these problems are described. The chapter concludes with a discussion of future work suggested for the development of methodology for studying participant experience of interactive art. 6.1 Thesis Problems Chapter 1 stated three problems to be addressed in this thesis: 1. We do not fully understand the aesthetics of interactive art- works and accordingly, the purposes for conducting partic- ipant experience studies. HCI practitioners may have difficulties accepting the qualitative and interpretive nature of an artist’s study methods. This is likely due to limited understanding of the artistic motivations behind conducting studies. Yet for collaborations between HCI and the arts to occur, a common language and understanding of artistic intentions and visions needs to be established [26]. 2. We do not fully understanding how laboratory-based user study methodology can be beneficial for the development of interactive art experiences. Interactive artists may believe that laboratory-based studies take a person away from the natural context of interaction and as a result, they tend to use informal methods for studying participant experience. While there are cases of more formal methodology used ([54], [29], [59], [97]), there is still room for more commentary on the benefits and implications of using these methods. 104 Chapter 6. Conclusion 3. We have no investigation of methods to compare experiences of participants with varying levels of understanding of art- work elements in a controlled laboratory setting. Interactive artworks require some exploration on part of the participant and the artifacts created by the artist should support this exploration. If a par- ticipant is unable to develop understanding of functional and expres- sive elements of an artwork, their experience may be limited. Thus, key concerns of the artist are: does the artwork properly support de- velopment of participant understanding? How much is the experience actually based on understanding development or, “figuring the art- work out”? And how does specific understanding of different artwork functional and expressive elements affect experience? These questions suggest the need for methodology that would allow artists to selec- tively influence participant understanding of specific artwork elements in a controlled setting. 6.2 Thesis Contributions This thesis can be summarized by three contributions made by this work: A relationship between the Interactive Arts and Human Computer Interaction (HCI) is identified. Art theory is reviewed, demonstrating the roles of art object and participant and the artist’s role as designer of art objects that can support participants in their role. Purposes for participant experience studies in the artist’s pro- cess are derived from these roles. Knowledge acquired through studies is used to refine techniques and interaction aesthetics. In the iterative activ- ity of design, study, and re-design, an art object learns and evolves in an indirect sense through interaction with participants. Finally, study activity documents interaction thus is itself part of an artwork’s content. The artist’s process is compared with the HCI practitioner’s user-centered design process to see how HCI user study methods may be appropriated by 105 Chapter 6. Conclusion artists to study participant experience. Having established deeper under- standing for the purpose of participant experience studies, the differences in goals and desired outcomes between artists and HCI practitioners become more clear. Echology: an interactive artwork and new interface for musical expression is created. Echology is an interactive installation piece for participants to play with spatialized sound in collaboration with fellow participants and with the Bel- uga whales at the Vancouver Aquarium. Observations of interaction with Echology during a public presentation confirm that its design is successful in being an approachable artwork that supports participant collaboration and a social space around the Interaction Table. Further studies in the labora- tory confirm that, aside from some ambiguity in the relationship between Beluga whale motion and sound, it effectively allows participants to hear, visualize and control sound spatialization. Further understanding of HCI user study methods applied to studying participant experience of interactive artworks is developed. Building on research of more formal, laboratory-based methodology used to study participant experience, the use of orientation media to selectively influence a participant’s understanding of artwork elements in a controlled study setting is proposed, practiced, and reviewed. Part of the interactive art experience is figuring out how one may interact. This game-like quality can add to enjoyment and engagement. At the same time, if the challenge of figuring the system out is too great, if the participant is left confused and unable to formulate interpretations, the experience can be un-enjoyable. Using orientation media in the form of instructions for the interactive artwork, it was possible to compare two groups of participant understanding processes. In one group participants approached the artwork with no prior knowledge of how it works and what to do. In the other group participants 106 Chapter 6. Conclusion were given prior knowledge of interaction: how they can act and how and why the system will react. What emerged from interviews with participants was knowledge of: 1. The elements of the system for which participants without instructions could eventually formulate mental models. 2. Which elements were confusing regardless of whether or not a partic- ipant received prior instructions. This work also investigates behaviour influenced by a participant’s role as a subject in an experimental setting and considers its implications for study design and interpretation. In Section 3.3 it was noted that laboratory- based studies could not be conducted without some change in participant behaviour due to the study settings but at that time, the possible changes had not been well observed. Five behaviours that are likely results of the study settings are observed in the study of Echology and the implications for results interpretation and future study design are noted. 6.3 Laboratory-Based Methods in the Artistic Process This work presents one case of participant experience studies of a specific artwork. The study design, results, and interpretations are specific to goals of Echology and to the artists’ pursuits whereas other forms of study design may be more appropriate to other artworks. It is hoped that the takeaway of this thesis is that studies conducted outside of an artwork’s ”natural” context, in controlled settings, with methods of in-depth data collection, and with participants brought in as study subjects, are in general seen as an important activity in the development of interactive artworks. Also, while this work is motivated by a focus on more formal study methods, it is not intended for these methods to be considered ”better” than informal ones. Rather, it is desirable that both formal and informal methods are seen as 107 Chapter 6. Conclusion useful for studying different aspects of participant experience and at different periods of design iteration. In the case of the Echology studies, the informal study presented in Sec- tion 5.1 allowed for observations of participant socialization, collaboration and natural behaviour in the intended artwork context. It also provided the opportunity to receive valuable feedback concerning participant reflections on the artistic expression of the artwork. This study was useful for under- standing how participants would approach and interact with Echology but less useful for confirming their levels of understanding the piece functionally and expressively. On the other hand, the laboratory-based study detailed in the rest of Chapter 5 was useful for collecting rich data on participant understanding of interaction, the processes used to obtain this understand, as well as metaphors and descriptions that were assigned to the artwork. It is from this study that recommendations for Echology redesign became apparent, more so than after the informal observations. So to return to the design processes discussed in Section 2.4, similarly to different types of user studies being suitable for different phases of design iteration, it is recom- mended that more formal methods, as well as informal methods be used during an artwork’s lifetime. 6.4 Future Work It is foreseeable that as computing technologies and practices continue to rapidly change, artists will respond with work commenting on the effects that these new technologies have on daily life. They will also continue to engage the audience in active participation, challenging them to inter- act with art objects that push the boundaries in technology and interface concepts. Thus, studying interaction will remain an important activity for understanding the relationships between participants and the art, for feed- back in an iterative art object design process, and to record part of the artwork’s content – the interaction. Two avenues for future work in the development of methodologies for studying participant experience are dis- cussed in this section: a framework to recommend study methodology based 108 Chapter 6. Conclusion on artwork characteristics and exploration into the aesthetics of study result presentation. 6.4.1 Study Methods Based on Artwork Characteristics This thesis did not provide much attention to the suitability of certain study methodologies as related to specific characteristics of interactive artworks. Still, in the experience studying Echology some of the initially planned meth- ods did not work out, likely because of characteristics of Echology as an artwork. For example, behaviour between paired participants was remarkably dif- ferent from that observed between participants at the NIME installation (see Sections 5.3.4 and 5.5.1). It was concluded that laboratory-based stud- ies involving participants who are aware that they are study subjects may not be suitable for studying informal collaboration and socialization around an artwork. Another example is that initially, the study was designed to use the video-cued recall method to support participant discussion of their expe- rience. Though during pilot studies, participants did not refer to the video. On reflection, this may have been because all of the interactive, visual, and auditory elements of Echology that may appear in the video are fairly sim- plistic and repetitive, offering little in the way of experience narrative to cue memories. These experiences point towards a need to better understand what study methods and designs are suitable for which types or characteristics of inter- active artwork. This work could involve using an existing artwork classifi- cation system such as Bell’s [14]) and making recommendations for study design based on characteristics of an artwork such as the level at which a participant physically interacts and the extent to which the artwork has game-like qualities. 109 Chapter 6. Conclusion 6.4.2 Aesthetics of Study Result Presentation In the reviews of participant experience studies of Chapter 2 and 3, the manners in which study results are presented were discussed. Although not examined in detail, this is actually a very important topic in the consider- ation of interactive art aesthetics. Considering the purpose for participant experience studies as documenting interaction in an art system and that this documentation becomes part of the artwork’s content, the manner in which results are presented ultimately affects interpretations of the artwork. Thus, it is recommended that future research explore the characteristics of results and the aesthetics of their presentation. 6.5 Conclusion In this thesis purposes for studying participant experience of interactive artworks in the development of interactive art techniques and aesthetics, as well as in the creation of a final artwork were proposed. Three cases using formal methods appropriated from HCI highlighted the benefits of controlled settings and in-depth data collection. Building from these cases, a method using orientation media was proposed and practiced in a study of Echology. The study design presented is just one such design that can be used to evaluate and deepen understanding of an artwork. Through its processes, a particular experience was being created for people, an experience within which they would experience the artwork. The study was not only an activity that allowed development in understanding of interaction with Echology, but was also another creation allowing for expression between artist and participants, another part of the artwork that is Echology. 110 List of Artworks Ascott, R. (1959). “Change Painting”. Blast Theory. (2001 – ). “Can You See Me Now?”. Fels, S. and Mase, K. (1997 – ). “Iamascope”. Iwai, T. (1999). “Composition on the Table”. Krueger, M. (1970). “Videoplace”. Mateas, M. and Stern, A. (2005 – 2006). “Façade”. Olsson, K. and Kawashima, T. (2005). “The king has. . . ”. Sengers, P., Liesendahl, R., Magar, W., and Seibert, C. (2003). “The Influ- encing Machine”. Sommerer, C. and Mignonneau, L. (1994 – 1997). “A-Volve”. Tzara, T. “How to make a Dadaist poem”. Weinbren, G. (1982 – 1985). “The Erl King”. 111 Bibliography [1] Creativity and cognition studios website. [2] Cycling 74 max/msp product site. [3] Glossary of terms from education @ turnerhouse., Accessed Jan- uary 2008. [4] Interface ecology lab website. [5] Phidgets Inc. [6] Vancouver aquarium beluga cam. [7] C. Abras, D. Maloney-Krichmar, and J. Preece. User-centered design. In W. Bainbridge, editor, Encyclopedia of Human-Computer Interac- tion. Thousand Oaks: Sage Publications, (in press). [8] P. D. Adamczyk, K. Hamilton, M. B. Twidale, and B. P. Bailey. HCI and new media arts: methodology and evaluation. In CHI ’07: CHI ’07 extended abstracts on Human factors in computing systems, pages 2813–2816, 2007. [9] L. Alben. Quality of experience: defining the criteria for effective interaction design. interactions, 3(3):11–15, 1996. [10] E. Aley, T. Cooper, R. Graeber, A. Kerne, K. Overby, and Z. O. Toups. Censor chair: exploring censorship and social presence through 112 Bibliography psychophysiological sensing. In MULTIMEDIA ’05: Proceedings of the 13th annual ACM international conference on Multimedia, pages 922–929, 2005. [11] R. Ascott. Behaviourist art and the cybernetic vision (part II). Cy- bernetica, 10(1):25–56, 1967. [12] R. Ascott. The cybernetic stance: My process and purpose. Leonardo, 1(2):105–112, 1968. [13] R. Ascott. Telematic Embrace: Visionary Theories of Art, Technology, and Consciousness. University of California Press, Berkeley, CA, USA, 2003. [14] S. Bell. Participatory Art and Computers: Identifying, Analysing and Composing the Characteristics of Works of Participatory Art that Use Computer Technology. PhD thesis, Loughborough University, 1991. [15] S. Benford, A. Crabtree, M. Flintham, A. Drozd, R. Anastasi, M. Pax- ton, N. Tandavanitj, M. Adams, and J. Row-Farr. Can you see me now? ACM Trans. Comput.-Hum. Interact., 13(1):100–133, 2006. [16] D. Benyon. The role of task analysis in systems design. Interact. Comput., 4(1):102–123, 1992. [17] Z. Bilda, E. Edmonds, and D. Turnbull. Interactive experience in a public context. In C&C ’07: Proceedings of the 6th ACM SIGCHI conference on Creativity & cognition, pages 243–244, 2007. [18] D. Birchfield, K. Phillips, A. Kidané, and D. Lorig. Interactive public sound art: a case study. In NIME ’06: Proceedings of the 2006 con- ference on New interfaces for musical expression, pages 43–48, 2006. [19] C. Bishop. Installation Art: a critical history. Tate Publishing, Lon- don, 2005. [20] T. Blaine and S. S. Fels. Collaborative musical experiences for novices. Journal of New Music Research, 32(4):411–428, 2003. 113 Bibliography [21] M. A. Blythe, K. Overbeeke, A. F. Monk, and P. C. Wright, editors. Funology: from usability to enjoyment. Kluwer Academic Publishers, Norwell, MA, USA, 2004. [22] K. Boehner, R. DePaula, P. Dourish, and P. Sengers. Affect: from information to interaction. In CC ’05: Proceedings of the 4th decennial conference on Critical computing, pages 59–68, 2005. [23] J. E. Boyd, G. Hushlak, and C. J. Jacob. Swarmart: interactive art from swarm intelligence. In MULTIMEDIA ’04: Proceedings of the 12th annual ACM international conference on Multimedia, pages 628– 635, 2004. [24] P. Bürger. Theory of the Avant-Garde. University of Minnesota Press, Minneapolis, MI, USA, 1984. [25] J. Burnham. The aesthetics of intelligent systems. In E. F. Fry, editor, On the Future of Art, pages 95–122. The Viking Press, New York, NY, USA, 1970. [26] L. Candy and E. Edmonds. Modeling co-creativity in art and tech- nology. In C&C ’02: Proceedings of the 4th conference on Creativity & cognition, pages 134–141, 2002. [27] S. Cornock. Art and interaction. 1977 monograph from Leicester Polytechnic, UK. [28] S. Cornock and E. Edmonds. The creative process where the artist is amplified or superseded by the computer. Leonardo, 6(1):11–16, 1973. [29] B. Costello, L. Muller, S. Amitani, and E. Edmonds. Understanding the experience of interactive art: Iamascope in beta space. In IE2005: Proceedings of the second Australasian conference on Interactive en- tertainment, pages 49–56, 2005. [30] G. Coulter-Smith. Deconstructing Installation Art: Fine Art and Me- dia Art, 1986-2006., 2006. On- line book. 114 Bibliography [31] M. Csikszentmihalyi. Flow: The Psychology of Optimal Experience. Harper Perennial, New York, NY, USA, 1991. [32] G. D’Arcangelo. Creating contexts of creativity: musical composition with modular components. In NIME ’01: Proceedings of the 2001 conference on New interfaces for musical expression, pages 1–4, 2001. [33] S. B. Davis. Its art, but is it hci? – testing the boundaries. In ACM SIGCHI’05 Workshop on Graduate Education in Human-Computer Interaction, 2005. Invited paper for workshop on education. [34] R. Debatty. Interview with Milk and Tales. we make money not art:, Oct 2007. Blog entry. [35] P. Desmet and E. Dijkhuis. A wheelchair can be fun: a case of emotion- driven design. In DPPI ’03: Proceedings of the 2003 international conference on Designing pleasurable products and interfaces, pages 22– 27, 2003. [36] M. Deutscher, R. Hoskinson, S. Takahashi, and S. Fels. Echology. In NIME ’05: Proceedings of the 2005 conference on New Interfaces for Musical Expression, page 274, May 2005. [37] M. Deutscher, R. Hoskinson, S. Takashashi, and S. Fels. Echology: an interactive spatial sound and video artwork. In MULTIMEDIA ’05: Proceedings of the 13th annual ACM international conference on Multimedia, pages 937–945, 2005. [38] T. Dove. Theater without actors: Immersion and response in instal- lation. Leonardo, 27(4):281–287, 1994. [39] J. S. Dumas and J. C. Redish. A Practical Guide to Usability Testing. Ablex Pub. Corp., 1993. [40] E. Edmonds, L. Muller, and M. Connell. On creative engagement. Visual Communication, 5(3):307–322, 2006. 115 Bibliography [41] E. Edmonds, G. Turner, and L. Candy. Approaches to interactive art systems. In GRAPHITE ’04: Proceedings of the 2nd international conference on Computer graphics and interactive techniques in Aus- tralasia and South East Asia, pages 113–117, 2004. [42] K. A. Ericsson and H. A. Simon. Protocol Analysis: Verbal Reports as Data. The MIT Press, Cambridge, MA, USA, 1993. [43] S. S. Fels. Intimacy and embodiment: Implications for art and tech- nology. In Proceedings of the ACM Conference on Multimedia, pages 13–16, 2000. [44] S. S. Fels and K. Mase. Iamascope: A graphical musical instrument. Computers and Graphics, 2(23):277–286, 1999. [45] J. Fogarty, J. Forlizzi, and S. E. Hudson. Aesthetic information col- lages: generating decorative displays that contain information. In UIST ’01: Proceedings of the 14th annual ACM symposium on User interface software and technology, pages 141–150, 2001. [46] W. W. Gaver, J. Beaver, and S. Benford. Ambiguity as a resource for design. In CHI ’03: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 233–240, 2003. [47] W. W. Gaver, J. Bowers, A. Boucher, H. Gellerson, S. Pennington, A. Schmidt, A. Steed, N. Villars, and B. Walker. The drift table: de- signing for ludic engagement. In CHI ’04: CHI ’04 extended abstracts on Human factors in computing systems, pages 885–900, 2004. [48] B. G. Glaser and A. L. Strauss. Discovery of Grounded Theory: Strate- gies for Qualitative Research. Aldine, 1967. [49] E. Goffman. Behavior in Public Places: Notes on the Social Organi- zation of Gatherings. Glencoe: The Free Press, 1966. [50] B. C. E. Graham. A Study of Audience Relationships with Interactive Computer-Based Visual Artworks in Gallery Settings, through Obser- 116 Bibliography vation, Art Practice, and Curation. PhD thesis, The University of Sunderland, 1997. [51] M. Gurevich. Jamspace: designing a collaborative networked music space for novices. In NIME ’06: Proceedings of the 2006 conference on New interfaces for musical expression, pages 118–123, 2006. [52] D. Heller. Aesthetics and interaction design: some preliminary thoughts. interactions, 12(5):48–50, 2005. [53] K. Höök. Evaluating the utility and usability of an adaptive hyperme- dia system. In IUI ’97: Proceedings of the 2nd international conference on Intelligent user interfaces, pages 179–186, 1997. [54] K. Höök, P. Sengers, and G. Andersson. Sense and sensibility: eval- uation and interactive art. In CHI ’03: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 241–248, 2003. [55] J. Huh, M. S. Ackerman, and R. Douglas. The use of aesthetics in hci systems. In CHI ’07: CHI ’07 extended abstracts on Human factors in computing systems, pages 2441–2446, 2007. [56] E. Huhtamo. Trouble at the interface, or the identity crisis of interac- tive art. FRAMEWORK: The Finnish Art Review, 2(1), 2004. [57] T. Iwai. Composition on the table. In SIGGRAPH ’99: ACM SIG- GRAPH 99 Electronic art and animation catalog, page 10, 1999. [58] J. F. Kelley. An iterative design methodology for user-friendly natu- ral language office information applications. ACM Trans. Inf. Syst., 2(1):26–41, 1984. [59] R. L. Knickmeyer and M. Mateas. Preliminary evaluation of the in- teractive drama facade. In CHI ’05: CHI ’05 extended abstracts on Human factors in computing systems, pages 1549–1552, 2005. 117 Bibliography [60] M. W. Krueger. Artificial Reality II. Addison-Wesley, Reading, MA, USA, 1991. [61] M. W. Krueger, T. Gionfriddo, and K. Hinrichsen. Videoplace – an artificial reality. In CHI ’85: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 35–40, 1985. [62] D. V. Lehn, C. Heath, and J. Hindmarsh. Exhibiting interaction: Con- duct and collaboration in museums and galleries. Symbolic Interaction, 24:189–216, 2001. [63] D. Livingstone and E. Miranda. Orb3: adaptive interface design for real time sound synthesis & diffusion within socially mediated spaces. In NIME ’05: Proceedings of the 2005 conference on New interfaces for musical expression, pages 65–69, 2004. [64] D. M. M. Lopes. The ontology of interactive art. Journal of Aesthetic Education, 35:65–81, 2001. [65] F. J. Malina. Differences entre la science et l’art: Quelques reflexions. Leonardo, 1(4):449–455, 1968. [66] T. W. Malone. Heuristics for designing enjoyable user interfaces: lessons from computer games. pages 1–12, 1984. [67] M. Mateas. Expressive AI. Leonardo, 34(2):147–153, 2001. [68] E. Mayo. The human problems of an industrial civilization. MacMillan, New York, NY, USA, 1933. [69] L. Muller and E. Edmonds. Living laboratories: Making and curating interactive art: Copyright restrictions prevent acm from providing the full text for this work. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Art gallery, page 160, 2006. [70] Q. Nguyen, S. Novakowski, J. E. Boyd, C. Jacob, and G. Hushlak. Motion swarms: video interaction for art in complex environments. 118 Bibliography In MULTIMEDIA ’06: Proceedings of the 14th annual ACM interna- tional conference on Multimedia, pages 461–469, 2006. [71] J. Nielsen. Heuristic evaluation. In J. Nielsen and R. L. Mack, editors, Usability Inspection Methods. John Wiley & Sons, New York, NY, USA, 1994. [72] D. Norman. The Design of Everyday Things. Basic Books, New York, NY, USA, 2002. [73] D. Norman. Emotional Design: Why We Love (Or Hate) Everyday Things. Basic Books, New York, NY, USA, 2003. [74] D. A. Norman and S. W. Draper. User Centered System Design: New Perspectives on Human-Computer Interaction. L. Erlbaum Associates, 1986. [75] K. Olsson and T. Kawashima. The king has... In MULTIMEDIA ’05: Proceedings of the 13th annual ACM international conference on Multimedia, pages 1075–1076, 2005. [76] H. M. Parsons. What Happened at Hawthorne?: New evidence sug- gests the Hawthorne effect resulted from operant reinforcement con- tingencies. Science, 183(4128):922–932, 1974. [77] L. Pellarin, N. Böttcher, J. M. Olsen, O. Gregersen, S. Sarafin, and M. Guglielmi. Connecting strangers at a train station. In NIME ’05: Proceedings of the 2005 conference on New interfaces for musical expression, pages 152–155, 2004. [78] S. Penny. From a to d and back again: The emerging aesthetics of interactive art. Leonardo Electronic Almanac, 4(4), 1996. [79] M. G. Petersen, O. S. Iversen, P. G. Krogh, and M. Ludvigsen. Aes- thetic interaction: a pragmatist’s aesthetics of interactive systems. In DIS ’04: Proceedings of the 5th conference on Designing interactive systems, pages 269–276, 2004. 119 Bibliography [80] R. W. Picard. Affective computing. MIT Press, Cambridge, MA, USA, 1997. [81] F. Popper. From Technological to Virtual Art. The MIT Press, Cam- bridge, MA, USA, 2005. [82] J. Preece, Y. Rogers, and H. Sharp. Interaction Design. John Wiley & Sons, Inc., New York, NY, USA, 2002. [83] V. Pulkki. Virtual source positioning using vector base amplitude panning. Journal of the Audio Engineering Society, 45(6):456–466, 1997. [84] D. Rokeby. Transforming mirrors: Subjectivity and control in inter- active media. In S. Penny, editor, Critical Issues in Electronic Media. State University of New York Press, Albany, NY, USA, 1995. [85] D. Rokeby. The construction of experience: interface as content. In Digital illusion: entertaining the future with high technology, pages 27–47. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 1998. [86] R. Rosenthal and K. L. Fode. The problem of experimenter outcome- bias. In D. P. Ray, editor, Series Research in Social Psychology. Na- tional Institute of Social and Behavioral Science, Washington, DC, USA, 1961. [87] E. P. Rozanski and A. R. Haake. The many facets of hci. In CITC4 ’03: Proceedings of the 4th conference on Information technology cur- riculum, pages 180–185, 2003. [88] S. D. Scott, M. Sheelagh, T. Carpendale, and K. M. Inkpen. Territori- ality in collaborative tabletop workspaces. In CSCW ’04: Proceedings of the 2004 ACM conference on Computer supported cooperative work, pages 294–303, 2004. 120 Bibliography [89] P. Sengers and B. Gaver. Staying open to interpretation: engaging multiple meanings in design and evaluation. In DIS ’06: Proceedings of the 6th conference on Designing Interactive systems, pages 99–108, 2006. [90] B. Shneiderman. Designing the User Interface: Strategies for Effec- tive Human-Computer Interaction. Addison-Wesley Publishing Co, Reading, MA, USA, 1987. [91] C. Sommerer and L. Mignonneau. A-volve: a real-time interactive environment. In SIGGRAPH ’94: ACM SIGGRAPH 94 Visual Pro- ceedings, pages 172–173, 1994. [92] L. A. Suchman. Plans and Situated Actions: The Problem of Human- machine Communication. Cambridge University Press, Cambridge, 1987. [93] T. Szakal, C. Groenegress, W. Strauss, and P. Peranovic. Phonic frequencies – shaping networked realities. In Proceedings of Melbourne DAC, the 5th International Digital Arts and Culture Conference, 2003. [94] J. C. Tang. Findings from observational studies of collaborative work. In Computer-supported cooperative work and groupware, pages 11–28. Academic Press Ltd., London, UK, UK, 1991. [95] R. B. Taylor. Human Territorial Functioning: An Empirical Evolu- tionary Perspective on Individual and Small Group Territorial Cog- nitions, Behaviors, and Consequences. Cambridge University Press, New York, NY, USA, 1988. [96] P. L. Tyack and E. H. Miller. Vocal anatomy, acoustic communication and echolocation. In A. R. Hoelzel, editor, Marine Mammal Biol- ogy: An Evolutionary Approach, pages 142–184. Blackwell Publishing, 2002. [97] A. Webb, A. Kerne, E. Koh, P. Joshi, Y. Park, and R. Graeber. Chore- ographic buttons: promoting social interaction through human move- 121 Bibliography ment and clear affordances. In MULTIMEDIA ’06: Proceedings of the 14th annual ACM international conference on Multimedia, pages 451–460, 2006. [98] G. Weinbren. In the ocean of streams of story. Millennium Film Journal, (28), 1995. [99] M. Weiser and J. S. Brown. Designing calm technology. PowerGrid Journal, 1, 1996. [100] D. Wessel and M. Wright. Problems and prospects for intimate musical control of computers. Computer Music Journal, 26:11–22, 2002. [101] K. D. D. Willis. User authorship and creativity within interactivity. In MULTIMEDIA ’06: Proceedings of the 14th annual ACM interna- tional conference on Multimedia, pages 731–735, 2006. [102] T. Winograd. From computing machinery to interaction design. In P. Denning and R. Metcalfe, editors, Beyond Calculation: The Next Fifty Years of Computing, pages 149–162. Springer-Verlag, 1997. 122 Appendix A Study Material A.1 Participant Recruitment Notice User Interface and Experience Study of “Echology”: an Interactive Spatial Sound and Video Artwork I am conducting a study of the interactive sound and video installation, Echology, for my graduate thesis. The purpose of this study is to investigate new methods of evaluating the experience people have with artworks of a similar nature. ***If you are over the age of 19, and you have not already seen Echology, you are invited to this special presentation*** The study will take a total time of 60 minutes and your experience at the Echology installation will be video-taped. Following this, you will be given a questionnaire and led through an interview about your experience. Par- ticipants will receive a payment of $10 for the 60-minute study. 123 Appendix A. Study Material A.2 Orientation Media: Echology Instruction Cards 1. Sounds are controlled by the motion of Beluga whales in the video feed. 2. There can be up to 4 types of sound playing at once, each represented by a different colour of circle. Each type of sound is controlled by motion in a different area of the Beluga whale video feed. 3. Sounds are “spatialized” within a circular area around the Interaction Table so that they appear to be moving. 4. A sound can travel paths between 8 “reflection points” which are rep- resented by blue circles on the Interaction Table screen. 5. The next reflection point on a sound’s path is determined by the “di- rection” of the current reflection point. 6. A reflection point has 8 possible directions: towards any of the other 7 reflection points or towards itself. 7. The direction of a reflection point can be changed by pressing the associated button on the Interaction Table. 124 Appendix A. Study Material A.3 Orientation Media: Echology Introduction Cards Description with no indicator of live or prerecorded video: Echology is an installation piece that uses a video feed of the Bel- uga whales at the Vancouver Aquarium to control sounds played in a large atrium space. The sounds are “spatialized” around a sound system. Participants can “play” with the directions of sound by pressing buttons at an “Interaction Table”. Feedback of sound movement is given to the participants, along with visu- als displaying the Belugas swimming at the aquarium. Description stating video is live: Echology is an installation piece that uses a LIVE video feed of the Beluga whales at the Vancouver Aquarium to control sounds played in a large atrium space. The sounds are “spatialized” around a sound system. Participants can “play” with the di- rections of sound by pressing buttons at an “Interaction Table”. Feedback of sound movement is given to the participants, along with visuals displaying the Belugas swimming at the aquarium. Description stating video is prerecorded: Echology is an installation piece that uses a PRERECORDED video feed of the Beluga whales at the Vancouver Aquarium to control sounds played in a large atrium space. The sounds are “spatialized” around a sound system. Participants can “play” with the directions of sound by pressing buttons at an “Interac- tion Table”. Feedback of sound movement is given to the partic- ipants, along with visuals displaying the Belugas swimming at the aquarium. 125 Appendix A. Study Material A.4 Interview Question Guide When you walked into the Echology room, what were your first impressions? Can you describe a little more what you saw? Are there things that you particularly liked? What about things that you did not like? Did you feel frustrated at any point? Did you feel tired or bored at any point? Can you remember any things that you reflected on that maybe were not directly related to Echology but were thoughts or feelings that came up for you? Were you relating this to any other experience that you have had before? Can you tell me how you think Echology worked? What connection did you see between the music and sound and the whales? Did you see Echology more as an art piece or as a game or neither? Can you talk about what you might have been thinking or feeling about the Belugas? Have you gone to the Vancouver Aquarium before? If you could talk with the Belugas at the aquarium, what would you say to them? Can you think of anyone who would find a work like Echology appealing? Can you think of places or events where it would make sense to install Echology? Do you have any further questions or comments? 126 Appendix A. Study Material A.5 Questionnaire 1. Age: 2. Gender: Male / Female 3. Occupation: 4. How many experiences have you had with interactive artworks? (None, Some, Several, Many) 5. How often do you work with computers? (Rarely, Once a month, Once a week, Daily) 6. How much experience do you have playing video games? (None, Some, A fair amount, A lot) 7. Do you have any musical education? (None, Some, A fair amount, A lot) 8. How often do you go to art galleries? (Never, Rarely, A few times a year, Often) 9. How often do you go to museums? (Never, Rarely, A few times a year, Often) 10. Did you enjoy your experience with Echology? (No, A little bit, Yes, A lot) 11. Did you enjoy the aesthetics (how the interaction space looked) of Echology? (No, A little bit, Yes, A lot) 12. Have you gone to the Vancouver Aquarium before? (No, Yes but a long time ago, Yes, Yes I go often) 13. Please list any 3 words that you can associate with your experience. 14. In one or two sentences, please state what you believe the general artistic concept of Echology to be. 127 Appendix B Study Results B.1 General Data Table B.1: Study participant information: age, assigned alias, gender, and occupation. Session Type: No Instructions, Pre-recorded Told Live or Recorded Age Alias Gender Occupation Y 21 Anne F student: psychology Y 23 Andrea F student Y 20 Ambrose F student Y 22 Abbott M student Y 19 Arthur M student Y 24 Amanda F grad student: nutri- tion Y 26 Audrey F grad student: nutri- tion Session Type: No Instructions, Live Told Live or Recorded Age Alias Gender Occupation N 30 Ben M student Y *** Brian M *** Y 21 Burke M student: mechanical eng 128 Appendix B. Study Results Y 21 Barbara F student Y 20 Brook F student: civil eng Y 36 Brigitte F phd student: dentistry Y *** Beth F student Y 28 Byron M student Session Type: Instructions, Pre-recorded Told Live or Recorded Age Alias Gender Occupation N 31 Carl M grad student: comp sci N 23 Claude M grad student: elec eng N 20 Cory M student Y 25 Chris M student: comp sci Y 21 Charles M student Y 25 Cassandra F grad student: comp sci Session Type: Instructions, Live Told Live or Recorded Age Alias Gender Occupation Y 20 Deborah F student Y 19 Danielle F student: first year eng Y 24 Daria F student: enviro eng Y 25 Denise F admin assistant Y 25 David M student 129 Appendix B. Study Results Table B.2: Echology experience durations recorded in min- utes as time from subject entering room to when experi- menter called. Anne 2.5 (minutes) Andrea 10 Ambrose 6 Abbott 4 Arthur 6.5 Amanda and Audrey 10.5 Ben 8 Brian 7.5 Burke 7 Barbara not recorded Brook and Brigette 10 Beth and Byron 11 Carl 20 Claude 6 Cory 12.5 Chris 18 Charles 9 Cassandra 10 Deborah 16 Danielle and Daria 37 Denise and David 5.5 130 Appendix B. Study Results B.2 Questionnaire Results Table B.3: Questionnaire results: free input responses. Anne (left blank) Interactive voices Andrea exciting, scary, grate- ful I think it is a connection between what people hear and what people do with it. Ambrose calming, unique, peaceful Bridging art and interaction by using visual, auditory, and tactile means. Abbott fun Atmosphere is good, looks like I am underwater. Arthur curious, fun, enjoy The purpose of the Echology is prob- ably to be used in a movie theatre of video games (like an war / an action game). This way, a user would feel as if it happened to him. Amanda (left blank) An interactive game in which people can set sounds or just turn them off. Audrey different, confusing, abstract To provide a way for the art and per- son to interact for a greater, more per- sonal experience. Ben curious, interesting, enjoyable It is an idea of communication be- tween human beings and nature. Brian entertaining, mystical, wonderful A mystical experience where one ex- periences the beauty of the belugas and their interaction with their envi- ronment. Burke behaviour, biology, life It is to investigate and show how an animal (or maybe other living things) behaves with the surroundings. Continued on next page 131 Appendix B. Study Results Table B.3 – continued from previous page Barbara calm, relaxing, blue You can interact with the art, and change it to what you want. Brook spatial, absorb, bounce I think the general artistic concept of Echology is to encourage people to appreciate the “spatial feel” of sound that Echology can bring to people. Brigitte interesting, fun, simple Mimic the lone situation. Beth calming, interactive, unexpected, fascinat- ing As an interactive system, Echology may be to tune the general public to sounds from nature through their senses (touch, listen, sight). Byron mysterious, remote, calm It sounds mysterious. It makes me feel I’m alone but not lonely. Carl curiosity, thought- provoking, beautiful Help people learn how animals com- municate. Connect people’s visual and audio sensations together. Claude comfortable, curious, kind of attractive Use your feeling to control the sounds around you. Cory (left blank) (left blank) Chris unique, innovative, in- teresting Virtual reality and interactive art- work. Charles cool, aesthetic, little confusing when enter the room/before read cards Express sound in a visual way. Make an isolated environment to make the sound more effective than it is, and let the user to concentrate on the sound in that environment. Continued on next page 132 Appendix B. Study Results Table B.3 – continued from previous page Cassandra non-intentional, inter- esting, confused First, it’s better that an Echology art piece would make people involve in the whole setting. (Just like the set- tings in this experiment, I can have some communication with the set- tings). Second, a general echology art piece would better convey some mean- ingful content/idea to the audience. Deborah fun, curious, amazed Using sound as an art, for application in public. Art can also be relaxing and fun. Danielle confuse, fun, interest- ing Something to do with nature. Denise spacelike, underwater, cozy Manipulation of sound. David sound, effects, sub- merged The manipulation/interaction with whales by their way of communicat- ing. 133 Appendix B. Study Results B.3 Interview Transcript Excerpts Table B.4: Participant statements related to Echology as a game and Echology as art. Anne Neither. I think it’s like, those educational stuff, like telling you how the whales communicate with each other and stuff like that. Cause like, you showed me the information about the whales before so I was connecting, and I do see the whales in the background. Andrea I mean, the decoration seems like art, but the table seems like a game. It maybe could have a score, you know, if you catch it (re- ferring to the moving sound balls) you can get a high score? That’s my suggestion or. . . Ambrose Mixture of both and at the same time neither. Do you have reasons for that? Well, games usually have an objective and this one didn’t really have an objective but at the same time, there was a user interface controls and stuff so well, you could make it a game if you wanted to. And as an art piece, umm, well usually, well umm, I personally think that games are a form of art so it works out in that sense and it just seemed really artistically laid out and involved the senses, but then at the same time, it was completely different from most things that I have seen that are games or art. Authur I don’t see it as an art piece that’s for sure. I probably just see the technology side. Amanda It’s not a game. (Audrey) No. . . I think it would probably be more like art. Continued on next page 134 Appendix B. Study Results Table B.4 – continued from previous page Audrey (Asked if Echology reminded them of anything.) Re- mind me of anything. . . Pacman or something. (Both Au- drey and Amanda laugh, Amanda agrees.) Pacman and like, frogger. . . no, not frogger the one that’s like (imitates video game noise and action). Oh pong. Pong yeah (laughs) but it wasn’t really, no, I don’t know. Cause it wasn’t really anything. Cause usually when you have computer things like this there’s like, some sort of point. But like here, it was hard to just sort of. . . just enjoy the balls and we were trying to figure out some sort of point. Like okay, am I trying to get, you know, this there and what am I trying to do with these balls and maybe if you sort of went in with the idea that there was really no point to it then maybe. . . Did you guys go in with the idea that... No it’s like, I was trying to figure out. . . (Amanda: We wanted to know, which one was the point.) Yeah we were really trying to figure out what, what to press when. Cause we’re so used to like the game where it’s like (imitates video game sound), where there’s some sort of pattern to (. . . ) and this sort of didn’t have a pattern cause we really didn’t relate it to the whales at all. Ok so when you walked into the room, did you have any thoughts on ‘‘oh is this supposed to be art?’’ and like...what were your thoughts on that? No I never thought about it as art, like mostly thought it was going to be like, an activity. . . (Amanda: Like a game?) Yeah like a game or something so that’s. . . (Amanda: I thought it was going to be a game) (Audrey agrees) Especially because of the buttons. Ben Umm, I prefer game. Game. . . game like, a game but not purely game, but close. Cause you know ahh, the way you touch the buttons and then gonna change something around, you change the way the whales are going to swim around and you change the sound. . . Continued on next page 135 Appendix B. Study Results Table B.4 – continued from previous page Barbara Umm. . . I think more like a game. Do you have any reasons for that? Umm. . . well I don’t know, for me, cause I can actually participate, like change the direction of the sound and stuff, so I thought it was more like a game then art. Because for me, art is more like sitting there and just looking at or hearing and stuff. Beth Not sure, I’m not sure whether I think of it as art, or chil- dren though – if there was such an interactive station at the aquarium, I’m sure they would think of it as a game cause not only could they use their hands and hear all kinds of dif- ferent sound but if they, umm, they could also not just use their hands and listen, but also look at, be able to view Bel- uga images. Like say, before the Beluga presentation there was a station, they could view how Belugas were like when they were playing around. If that sort of interests them, at least the moving coloured balls would be sort of fascinating I guess. Byron Neither, neither, neither. . . I don’t think it’s art or game. Do you have reasons for that? I think it’s not a game, because it’s not a game, but I think it’s not art either, but it’s definitely not a game. Though, it seems like a game because I keep pushing buttons but I don’t think it’s a game. Did you think it was a game when you first walked into the room? Yeah, at first I, yeah I mean, right after I came into the room and was pushing buttons it seemed like a game, but later, I don’t think it’s a game. (. . . ) Now I would say it is not a game. But yeah, you are right. If you asked me right after I came into the room, then I would most likely think, yeah, it’s a game. Continued on next page 136 Appendix B. Study Results Table B.4 – continued from previous page Carl Umm, maybe it’s art, for me a game is more interactive and more directions and you know what to do, like, what’s your mission in this setting. There’s like, confrontations and stuff. And when I went into this room I don’t know what I’m sup- posed to do, compared with if I want to play a game and there is. . . I get instructions telling you what you are sup- posed to do, what you shouldn’t do. That’s just one kind of game (. . . ) it’s not a game like that. Claude It’s more like a game. Do you have some reasons for that? No, actually, I don’t like to play computer games, but I feel that this is like a game. Maybe you could tell me why it would be a game and not an art piece. Why? Umm, I can tell some reason why it is not a game, because there is not a specific goal that I can reach. Then that is the reason, the reason that isn’t a game. But it seems to me that it is a game, which is by my feeling, but logically, it’s not a game. Cory It’s more like a game to me, I mean, the square, I mean the circles were kind of like pacmen for some reason. Do you have reasons for why it’s not art? Just that, well you can interact with it and then, it changes in your environment and then. . . so it’s more like a game than an art piece where it’s more like, still, and all you do is observe it and then make an analysis. Did you analyze this at all? Well, I analyzed it when I tried to put the sounds with the video and then with the, pressing of the buttons and then changing the movement I guess. Chris Yeah I think it’s more like an art than a game. Umm. . . Yes I think so. (Asked to expand on why.) Umm. . . I think . . . I cannot think of anything right now. Continued on next page 137 Appendix B. Study Results Table B.4 – continued from previous page Charles Artwork. . . not quite. Ahh. . . game? Maybe. Artwork looks very, artificial. . . like art, a piece of art. . . but I prefer to make it like this game Do you think that it could be both? Maybe both, but more close to a game. Cassandra Can you maybe see this as an art piece? Art piece? Yeah. You mean. . . the screen? or. . . Everything in the whole... (thinking) No. . . I would never relate that to. . . Why wouldn’t it be an art piece? So there would be some, at least some contents or, or some meaning, mean- ingful things that can be expressed or displayed by the art piece. So, I would. . . basically that kind of thing would be an art piece But I cannot figure out anything about such a setting so I cannot relate that you know. Deborah Mmm. . . I think it’s more like a game. More like a game? Yeah. . . Do you have some reasons? I’m thinking cause art is something like opera or. . . Denise I saw it actually more like art (. . . ) I think I’d be happy just to look at it without interacting, I mean, the sound is still going around, you don’t have to interact with it to make the sound move I guess. David It was a game, I thought it was a game (. . . ) As a game because there’s buttons and a screen and the moving things in the middle like, so colourful. Yeah cause it’s colourful and the screen and the buttons. Yeah, more like a game, because you can change (. . . ) pushing the buttons, you feel like, oh yeah, this moves this, or yeah it turns out if you press this (. . . ) so you have to figure out. 138 Appendix B. Study Results Table B.5: Participant statements related to understanding of Echology interactivity. Anne I can hear sound travel from hear and there although I can’t really match to the picture I see. I realize I don’t know what the purpose of the balls was. Sometimes the colours become really sharp; sometimes they are more transparent. Did that frustrate you? Umm, it just made me, I just. . . I should’ve, I should’ve checked what Echology was before I came here. The balls. . . how I turn it will change the voices going around me. (. . . ) But I can’t connect them. I guess I need more. . . I haven’t been hearing a lot of music. I have no experience with this stuff. Maybe if I know something before hand than I can catch it, but with no prior knowledge I don’t think I can really. . . Did you see any connection between the music and sound and the whales? I assume there must be a connec- tion but I can’t. . . I’m not even sure if something changes. I know the sound changes, but I’m not sure how and in what direction it changes. Andrea I saw some coloured balls, just some circles. Yeah and they came around with the animal and at first, at the beginning I didn’t find that it related (?) and maybe two minutes later I got it and when I hear something, the screen, they had them displayed. Do you think, did you feel that you figured everything out? No (laughs) No? I don’t think so. Cause sometimes it is too fast. I cannot catch it and actually I, you know, you mentioned that there is no wrong and right answer, I was just playing there and I don’t know which was the best score. Continued on next page 139 Appendix B. Study Results Table B.5 – continued from previous page Did you get frustrated at all? Sometimes. Sometimes? Yeah when the purple and the yellow and the white and this green one (. . . ) when I cannot catch it (. . . ) for example if I want to catch the purple one I make the donut and the centre of it is purple. But it doesn’t move. When I click bottom then it’s just like, goes around again. (. . . ) When the whales come to the screen and I found the sounds get heavier. When I press a button, and if I don’t make it like a donut and it turns around to a different direction and when I click it, the sound will go to another button, that’s why? Ambrose (. . . ) Took a while to realize that it was mainly an audio ex- hibit, and then it clicked that it was really about the direction of the speakers and that this somehow should be controlling them. (Asked if not immediately knowing how things worked caused frustration.) Not really, no. Well the buttons, they control the, there are a few little pacmen-like creatures, circles with wedges in them that you push to make them change directions and push it enough times it will form a donut shape where it will trap the sound. And the wedge in the circle has to do with the direction that the balls are going to bounce in and then the balls bouncing in there have to do with what speaker you hear the sounds in from the ones surrounding you. Continued on next page 140 Appendix B. Study Results Table B.5 – continued from previous page (Asked to describe experience to hypothetical friend) You are surrounded by a bunch of speakers and blue lighting and right in the middle is this little panel that’s round, it’s cylindrical shaped and there’s this little tv screen in the middle and all around it you have a set of buttons and then, pushing each of the buttons controls a few little circles on the screen that kind of look like pacmen and pushing the buttons controls the direction of the circles’ wedges and then they tell you the direction of, there’s some balls that go around and bounce and stuff and then you have some Beluga whales coming in and then while they come in, you have new balls coming in and these balls represent the sound playing on each of the speakers so you can sort of, using this interface, you can bounce the sounds from one circle to another and it actually makes the sounds bounce from one speaker to another. Arthur I thought the button is used to start the sound but appar- ently it’s not, after a while I noticed it. Did you figure out what makes the sound? Actually, I don’t, but I assume it’s a random process. Were you at all frustrated? Ah yeah, I have to say yes, like at the middle of it I don’t really know what the but- ton does so I kept pressing it and sometimes on the monitor screen, the button changes to square and you have a hole in the middle. I was like, oh, what’s this for? Continued on next page 141 Appendix B. Study Results Table B.5 – continued from previous page After some time I see how the button works (. . . ) You have these sounds, the sound is bouncing back and you decide the direction, where the direction of the sound will be go- ing to, and there are three types of sounds – the purple one, the yellow one and the umm. . . I forget but. . . the other one, the purple one is just like surround sound, the yellow one I think is the whale sound and the other one. . . I don’t remember. Do you think you’ve figured out most of the things going on? I think I figured it out. You could hear the movement right? Yeah. Could you relate the movement of the sounds to the movement of the balls on the screen? Yeah. Was it fun? After figuring it out? Yes, but before that it’s kind frustrating, a little bit but not that much. I want to know how it works but, I don’t know how it works and then I just start playing and then once I know a little bit more I start to enjoy it because I know a little bit more of it. There are three circles there moving around, what is that used for? What are they actually? (. . . ) They are differ- ent. . . ? What’s the purpose of this button? Of the button? Yeah I think you got that, basically you can control the direction of the sounds And why do you want to control it? Like, what do you use it for? (. . . ) Why don’t you just like put it like, you wanna play it on THIS side and you just press this button like that so that you can play it, something like that. . . ? Amanda (. . . ) To the speakers. If we let one of the balls in one of the rings, or (?) in this speaker for example and if the ball was in another one it come from other speaker so we kind of related (Audrey: We figured that out.) Continued on next page 142 Appendix B. Study Results Table B.5 – continued from previous page When we pressed the thing, the ball in the computer was turning and then it was a round circle, like a round circle and then when the sound was traveling it would take the sound cause it was trapping the sound. (. . . ) But then if you press it again you will have different sounds like, it will be one. and we didn’t understand that part well, I didn’t understand that part. Like when you had one sound, and then you kept it and then when you release it then they were well like, three different sounds, or three different circles. (Asked to describe experience to hypothetical friend.) There is a circle with some buttons that you have to push and you will hear this sound and if you trap the sound then you won’t hear anything and there are a computer with whales, a video with big whales swimming or whatever and then you just, make sound and release. Audrey Did you see any relationship between the belugas in the video and the sound? No, I didn’t think about that. We were mostly focused on the balls, the little circles. The belugas were sort of just the background. It seems like we never thought about connecting them. (Four different sound signals explained.) Yeah we saw but we didn’t know why sometimes there were more than one and sometimes it was just one. (Sound-Beluga relationship explained.) Oh so it’s related to the. . . I was frustrated for a little bit of it because I just didn’t see the point. Continued on next page 143 Appendix B. Study Results Table B.5 – continued from previous page (Asked to describe experience to hypothetical friend.) I went to do this sort of project thing and we walked into the room and it was kind of space-aged looking and there was a screen on a sort of table and buttons all over and there were these little circles and you can, well they’d move around representing sort of sound and there are speakers all over so you hear all these whale sounds and see all these whales in it. I was trying to figure out what it did. Yeah, it sort of didn’t really do a lot of things it just (?) sound and circles and whales in the background. It’s kind of weird (laughs) like I wasn’t really sure what it was for or anything, but it was something different. Ben On all the buttons I don’t see any difference to those but- tons. But each time I touch it and then I saw the whales swim around somewhere but yeah, I tried to figure out the difference but finally I think, it’s the same, you know, all the buttons are the same. Did you see any connection between the sounds and the Beluga whales? Every time I touch it, it like, changed it a little bit. It looks like communication somewhere but I don’t know what that really mean for whales. But it did change, I try to figure out the connection between but I’m actually not – cause it’s not human being. I mean, understandable, cause it’s their language. Cause I mean, I touch it, I try to, I touch and listen too and then, I didn’t find the connection. I did find that there was something where I touch the button here and then the sound would change a little bit, I did like, try to touch it faster then, maybe a little bit of change over there but I didn’t connect it with the movement. Continued on next page 144 Appendix B. Study Results Table B.5 – continued from previous page I tried to figure out the connection between the button I touch and the sound, but I didn’t figure it out so (. . . ) finally I realized a little bit the connection over there but it was really hard to figure out what the connection actually is. Like if it’s how hard you touch it or how fast you touch it. I mean, I found it could change accordingly but I didn’t figure it out, just very interesting. It was a good experience. Burke Is there anything that you are still confused about? Yes, so many things, like, how could we get the sound to go from Vancouver Aquarium? (. . . ) How can we play around with the buttons and then reflect it to different points. Barbara Do you think that you were able to figure what was going on? At first no, then after I read the instructions and after I play with the buttons several times, I realize that we can change the – through the button you can change the position of the sound. Were you frustrated when you couldn’t figure that out? Umm. . . a little bit. Beth I realized that oh, it seemed to be there were a lot of different kinds of sounds that arose from touching the buttons so that I thought, was quite interesting. Do you see any connection between the sound and the Belugas in the video? Some relation. . . like some sounds were definitely Belugas playing around or frolicking around but others they seemed more like sound effects. Byron There are the coloured balls, through the screen it is a bit easy to change the direction of the balls but if there is no such ball then stop the sound, and it is quite difficult to catch where the sound is from. Continued on next page 145 Appendix B. Study Results Table B.5 – continued from previous page I did think that the movement of the ball, the direction of the movement of the ball was the same as where the sound is from. I think there was direct relationship between where the sound came from and the direction of the balls. I think I could figure it out, I can identify the relationship between the sound and the ball and the change of the cir- cles but if you ask further if I could identify further the re- lationship of further how the system works, I don’t think so. (Beth: Yeah that was pretty much what I got too.) Did anything frustrate you? Yeah, the sound, I mean, I sometimes use my ears to catch the sound and the sound changes very rapidly so I try to push this button and (. . . ) but I can’t catch the movement of the sound. But frustrating as I said, that I cannot catch the, it’s too fast and my reaction I think is a bit slow. I cannot match the movement of the sound. I know the sound is here, I want to press all of these buttons but I couldn’t. Carl Later I found like, when I pressed these buttons, it will change the direction of the reflecting part, the circles. Sometimes when the sound is intense, the whale, the whole body of that whale will show up. But I don’t know during the experiment, sometimes it like, resets, after a while there is low whale and low sound. (. . . ) maybe two buttons, goes to (?) one speaker. (. . . ) just repeated things, just repeated sounds, by switching those buttons you can change the sequence of the sounds (. . . ) Continued on next page 146 Appendix B. Study Results Table B.5 – continued from previous page I haven’t figured out the circles. They have different colours like there is yellow, purple, red and I think (. . . ) and also I don’t know how many whales were in the video interface. I don’t know if circles corresponding to each one of them. There are too many things that I don’t know (. . . ) how it works (. . . ) speakers corresponding to reflection point and change orientation and sound will change and nothing beyond that (. . . ) maybe sounds corresponding to actions of the whales (. . . ) Ok, so not being able to get those connections, did that frustrate you a little bit? Yeah kind of. . . Claude When you touched that, did you expect it to be a touch screen? Yeah. So you read the cards before you tried to press any of the buttons? Yes, and I didn’t know the button is for me to press, I thought they were only the decora- tions. OK, so at this point are you still confused? Yeah, sort of. Were you a little bit frustrated? Umm. . . not really. Did you see the connection between the audio and the video? I tried but it sounds like. . . this circle is kind of connected to the audio. Okay, then what about the whales in the background? The whales in the back- ground? Yeah. I didn’t focus on them (. . . ) Yeah, I saw the whales and the audio, but I didn’t focus on the fish and the audio I mean. Continued on next page 147 Appendix B. Study Results Table B.5 – continued from previous page What’s the background, the fish – it’s actually not related to the audio at all? (. . . ) Cause I thought that the relation is the, position of the fish with the direction of the audio (. . . ) the audio is up and down, up and down, and that is corresponding to the water, not the fish mostly. Do you get what I mean? (. . . ) What the strong feeling to the visitor is the water, not the fish. So the volume of the audio is just like, gives the impression to the visitor that they are in a water environment. Cory I heard the sounds and then I saw, like, all these things, and then, it makes sense after I read the cards. Could you hear the sounds and match them up with the moving the moving balls on the screen? Yes. I did count the speakers and there was six of them and eight dots and I was like okay, it’s missing two but it travels in a circle anyway. Did you see any connection between the sound and the belugas in the video? Well there’s like four sounds and when I look at the sounds and the video it like, matches I guess. I assume like, closer by they make these chirpy noises and when they swim farther away or just suddenly appear or something like that. . . Why it is 6 speakers when there’s like, 8 buttons? (. . . ) I was trying to follow the sounds and was like, okay so when I come to this circle the sound is just like, somewhere in between. Continued on next page 148 Appendix B. Study Results Table B.5 – continued from previous page Chris At the beginning, it took me some time to get used to the en- vironment because I couldn’t differentiate the sound sources. (. . . ) and then I can get used to the space and the sound movement in the space, and then I can tell where the sound is moving, in which direction, and because there was a screen on the table so I can see where the sound is travelling and I can hear it at the same time so the table is very useful and helpful. At first, I don’t know what happens if I push any one of the buttons so I just keep on pushing and pushing and then I figure out that it will change the orientation of each reflection point then after that I know how to use the interactive part. I think the speakers actually control the sound source, and when the sound is travelling, each speaker will change the volume of its sound so it will create the movement of the sound traveling through the space. I think when the Beluga whales is moving, so is the sound. And I think, I don’t think it’s that much closely related but I definitely can see there is a connection. Just wondering, because there are 8 reflection points, but only 6 speakers, how does that. . . ? Charles I see that, as the card says, the ball reflects and the direction, it is going according to your current node. (Asked about noticing connection between Belugas and sound.) It took me a while to figure anything out and the only thing that I could find out is that the balls, the coloured balls becomes more darker when the Belugas come to the screen. Yeah that’s what I found. Continued on next page 149 Appendix B. Study Results Table B.5 – continued from previous page Cassandra Were you frustrated? Frustrated? No (. . . ) Actually con- fused. I don’t know what is going on (. . . ) What are you seeing when you are pressing the buttons? For the first time I didn’t notice anything I just push the buttons and there will be some balls in the screen and maybe after 1 or 2 minutes I just realized that once I push those buttons then there will be a move to an angle. So I noticed that and I tried another several times and then I see when the ball hits, hit on the angle then it will try to change to its direction. That’s the second thing that I perceived. That I tried to push the combinations to see but I don’t know the result. I just don’t know what the direction or meaning of the game. . . Did you hear the sound going in the same direction as the balls on the screen? No, I didn’t notice that (. . . ) Actually I can hear the sound moving and sound different directions but I didn’t try to relate the sound with the direction of the balls. Deborah I saw the little circles, and I don’t know what it is so I just press it, then certain sound came. And I thought, ok when sound comes, I don’t know right. . . but it bounces, like the sound is also in circles, the first time, so like, ok, maybe if I press this it will go to another direction. Were you frustrated at all? Not really. Not really, did you feel confused? Yeah, a bit confused. What type of questions were you asking yourself? Like, what are these buttons for? Then, in the cards it mentioned about the sound or something like that, and I didn’t hear the sound, so I was like, where’s the sound? What connection exists between the buttons and the sound? Continued on next page 150 Appendix B. Study Results Table B.5 – continued from previous page Did you see any, sort of connection between the sound and the Belugas? Yeah, the Beluga is on the screen, the dots are like, colour not blurry, then when the Beluga is really far away, it’s blurry. David Yeah, yeah. I could hear that the sounds were mov- ing. How about you (to Denise) did you hear that as well? (Denise: Yeah.) I had to look at the papers for the little ball bouncing was the food or was the whale itself. 151 Appendix B. Study Results Table B.6: Participant statements related to sensation and enjoyment. Anne I like the lighting (. . . ) the lighting is comfortable. I liked the 3D audio part. The audio part was kind of comforting, I don’t mind staying until I can. It would be nice to sit in the middle and watch a movie. Andrea (. . . ) I found that it is like a mystery. Did you like the sound? No actually (laughs) I find it’s weird. I feel scared when just this purple comes. It’s just, it’s much more weird and scares me. And blue one and yellow one that’s fine. It’s not too heavy. What were the aspects that you liked? When I catch the balls (laughs). It’s like, one of the site of the Star Wars you know? The Star Wars, the movie Ok yeah... like the planet and the (?) table, really like the planet, like control something Like the control centre? Yeah, yeah. Did you enjoy your time? Yeah, although sometimes I feel scared, but yeah, It’s like an adventure I think. Was it scary because you thought that you were being tested or scary because the music was scary? Music and the whole room, the decorations, the lights, I mentioned it’s like a spaceship. Continued on next page 152 Appendix B. Study Results Table B.6 – continued from previous page Ambrose The sounds were quite soothing and nice (. . . ) the intensity of the sound wasn’t all that noticeable, but it was really nice. I found that the design of the soundboard thing had some aesthetics to it as well, visually. The material at the bottom kind of looked like it came from some sort of glacier or something. Are there other terms that you would use to describe the space? Maybe quasi-futuristic. I like that it’s audio based – something new and different because art is usually visual in nature. (talking about sounds) They were soothing I can say I guess, with, slightly punctuated by discord and yeah, but mostly it was unobtrusive. Arthur Oh, it’s like. . . the room itself is so cool because, I don’t know like, the way I see it is, it’s like lots of speakers there it’s like, oh wow this is so cool but, like, I didn’t really see into the table at first. . . (Asked what he liked the most.) Technology (laughs) (. . . ) My first impression when I came into the room was like, wow, it’s like, lot’s of speakers, it’s like you’re in a movie and you’re going to watch big screen tv and lot’s of speakers there and it’s like wow. . . that’s the thing that I liked. Audrey Is there anything that you particularly liked about being in there? Not really. . . (Amanda: I don’t know. Well we want to just, keep the sound out.) Ben It’s a very quiet room, I feel comfortable (. . . ) and a little bit curious cause it’s my first time there. Continued on next page 153 Appendix B. Study Results Table B.6 – continued from previous page The things I liked, it’s possible you can play around it, and also the lights and stuff – not too bright, a little bit dim, I feel comfortable there. The sound. . . it’s not like music, it’s just natural sound, it’s not noise, but it’s not enjoyable. I have to say, it’s not enjoy- able (. . . ) It’s not enjoyable cause for human beings, they like music right? But for that sound it’sI still feel comfortable, but not like say, say I enjoy it. Burke I’m pretty sure that’s the technology you have. But I’m pretty curious how to program that stuff (. . . ) I’m one of the lead (?) sound system, so I’m pretty curious to how we can set up that kind of thing (. . . ) How could you put the code in the computer and it like, works with the 8 speakers? Did the sounds in general make you feel anything? Not really. I mean like, that’s kind of useful, to sound like (makes sound effect) and for me, umm, there’s nothing special. What makes me really, really excited is how can we, like, what type of program. . . (Max/MSP and programming background explained to him.) Is there any particular course that teaches this kind of stuff at UBC? (. . . ) I’ve never worked with this kind of stuff (. . . ) pretty amazing for me. Maybe, this, this is the first time. (. . . ) Almost all my friends here, they are like, musical. So, for me, if I can know stuff like this, at least I can have better communication with them maybe. Cause it’s pretty difficult if they talk about music and I know nothing about recording and stuff but at least if I can learn about this I can better communicate with them and spend time with them. Continued on next page 154 Appendix B. Study Results Table B.6 – continued from previous page Barbara I was impressed by the lights. Then I start to listen to the sound that was coming from all different directions so I felt like I’m actually in the middle of where the sound is coming from. I liked the lights and the sounds, it made me, calm down and stuff. And, I liked the fact that I can change the direction of the sound. (Asked to describe experience to hypothetical friend.) The sounds were like whale sounds, it was pretty cool and I felt like it calms me down and I feel relaxed. I think it would be fun if there was somewhere like that that you can go into when you are stressed out or something, you know, to get relaxed. (. . . ) I think it would be a lot better if I can actually hear the sounds. Like usually when I go to the aquarium I just look at it and that’s it. But if I can actually hear it then I think that I feel like I’m in the ocean or something (. . . ) Could you like, make the whole room with whales like, with pictures and webcam videos and clips to make it feel more like an aquarium? Beth I thought it was rather nice to just be able to experience the thing that was present, and not have a set of instructions, to just be able to do what we want when we were in that kind of setting. Sort of reminded me of, especially with the occasional Bel- uga sounds, my most recent trip to the Vancouver Aquarium where in the atrium they just had all kinds of, I think they had this theatre or something where they just played out, just broadcasted all kinds of whale sounds and when I heard that it kind of just reminded me. Continued on next page 155 Appendix B. Study Results Table B.6 – continued from previous page I think it was rather interesting how most of the sounds were from nature and they were intermingled with sounds from Beluga whales. Byron Then I hear some sound and that kind of sound is very, like some noise, some kind of noise, like the sound from the forest, like you are in the forest, and feel very removed very far away. Sound is very mysterious like in some movie (. . . ) I like the kind of atmosphere. It’s kind of a very mysteri- ous sound and like you are in a forest. You are surrounded by nothing but trees (. . . ) You are in a forest and nobody is (. . . ) only you in the atmosphere and the sound like, it seems that the sound is from very removed area, very removed dis- tance. Like you were in a mountain or in a forest and you hear something and this also very interesting to catch the sound, catch the direction of sound by pushing button. (. . . ) It seems I was dreaming, feeling I was dreaming, I mean sometimes when I am dreaming I have a bit similar feeling. I can conclude my feeling as that, I feel that I’m alone, but I’m not lonely. That is my feeling, I’m alone, but I’m not lonely. And I think that, that music, that sound, can make me calm. I heard some music that seems came from very remote places, very remote place. I thought that the music sound very mys- terious and can leave me feeling calm (. . . ) music is very helpful for you if you are very nervous for your exams, for your finals, you just listen to that kind of music, can make me feel relaxed, not so nervous. Continued on next page 156 Appendix B. Study Results Table B.6 – continued from previous page Carl It’s a natural environment, sea animals, they can only hear these kind of sounds (. . . ) I was separated from the outside world and I was like, in nature and kind of those feelings, kind of. . . peace. Interesting, cause if you play with these buttons and you can actually change the sound, how the system sounds, you can like, if I do this I can affect something. I can enjoy what I did, if I find some particular setting I can get better sound effect, it’s worth while to make this effort to try to figure out (. . . ) maybe there are things for curious people you want to spend more time to play with this kind of work and try to figure out more. Claude (Asked about his first impressions.) Like. . . something. . . mystery about. . . not mys- tery. . .mystical. . . yeah mystical. (Asked to relate experience to another.) Scientific museum or like, ghost house (laughs). Comfortable. . . it is comfortable for me (. . . ) I like to play around the balls on the screen. When the visitor goes into such an environment, they may expect the audio is, to give them an impression of the water, it’s like, you are in the water, you are under the water and the frequency is like that also. I was happy with playing around with this. . . Cory The aquarium sounds, the beluga whale sounds, that’s very comforting I guess. I got bored after a while (. . . ) maybe just like, one minute before I call you? Continued on next page 157 Appendix B. Study Results Table B.6 – continued from previous page I thought it was really interesting how you can put a sound with the video and then you can feel like you are actually in this place by just like, standing there and listening to the sounds going around you. Chris I think the setting is very nice and the room is dark and quiet so the lighting is very comfortable and it kind of gave me the feeling that I am entering an art museum or an art gallery and not just a lab either. It’s very peaceful there. There is nothing like this that I have experienced before. Like a total surrounding environment that you are actually involved in. Charles Kinda cool, like an aquarium inside, and the sound is very (?) home stereo, the sound is very good. I just like to, I feel like, sound like goes around me, and that’s very cool (. . . ) The environment is very fantasy. The coolest thing about this experience is the room, the sounds in the room. Cassandra Sometimes when I tried to press the buttons and it would change the angle, then if it moves 8, or maybe, sometimes several times it will be a full round, then if a ball hits into that button, then it will just, stuck there. So I think that was one of the interesting things I think. I think the whole setting is similar to all the other aquariums that I’ve been to. I’ve never been to the Vancouver aquarium but I’ve been to another aquarium in Baltimore, the settings and the sound there are similar. Continued on next page 158 Appendix B. Study Results Table B.6 – continued from previous page Deborah I just like, playing around with the buttons, just like, I think it’s fun (laughs) I’m playing around. I was curious at first. And then I thought, how come I never see this before, even at the aquarium, and then, well it’s cool I think. Cool buttons (. . . ) and it looks cool. Denise Whales actually swimming around you or something, kind of a visual part of it would be more interesting (. . . ) But I think making it soft, like making it dim like that really makes it more about the sound and where they are coming from. Cause if you were in a bright room and there’s sounds coming from different directions. . . kind of obvious. It actually reminded me of being in the aquarium with the darkness. I kind of liked, you know that room was kind of small. I was thinking it would be a lot bigger, but it actually felt, quite, kind of intimate, you know, dark and kind of creepy and these noises going around you. (Asked to describe experience to hypothetical friend.) I would talk about the dark room and the noises going around, like you can see, directions. . . I don’t really know. I enjoyed it I think, yeah, I would have liked it if the sound went on a bit longer. It made me think of like a, spacecraft or something. Continued on next page 159 Appendix B. Study Results Table B.6 – continued from previous page I think I expected maybe like, pictures of whales going around on the walls or something. And I think probably you’ll see on the video, I think I was kind of like, looking around go- ing, is there anything else? (laughs) (David: yeah) I guess I expected pictures to come with the sounds as they moved maybe? (David: yeah, there was just low visual part.) David It’s kind of cool but you know, I’ve been at Disney World and similar places where they have installations and special effects and stuff and you know you find well mounted things, like walls, cables. You don’t see the speakers, you don’t see the stuff, you see only whales and (. . . ) So, although my ex- pectations were high cause I have already seeing those types of things at the giant places, it was really special because of the set. (Asked what Echology reminded him of.) Whenever you are swimming under the sea, you can here sometimes sounds. (Asked to describe experience to hypothetical friend.) (. . . ) The blue lights made me think I was in the sea, submerged, and the console in the middle, I don’t know, like a cockpit or navigation kind of thing. 160 Appendix B. Study Results Table B.7: Participant statements related to connecting with the Belugas and the live webcam feed. Anne The first thing I notice is the table and then and then I saw, I’m going to see some whales but they’re really, they are just background, very transparent. I can’t really see the whales. The thing that I noticed most was those little pacmen. It was kind of blurry. I didn’t even pay close attention to the whales. Any thoughts you have about the Beluga whales? Any thoughts? I just know they are unique with their voice, with their heads. Andrea I’m wondering that, is that a video of underwater? Yeah, it’s... Is that a video of underwater at the aqua centre? If you could say anything to the whales, is there something you would say? I just wonder how they communicate with each other. And I have a question – the sound is artificial or it’s real? (. . . ) Can I hear the sounds when I visit the Aqua Centre? Did you notice the Belugas or have any thoughts on the video? No cause when they come, I heard the sound. That’s it. Ambrose Asked about the Belugas as content. I don’t know, I guess they could have used any other animal or anything that makes a sound and it could have worked out either way. If you could say anything to the whales, is there something you would say? Hmm. . . hi, how’s it going? Continued on next page 161 Appendix B. Study Results Table B.7 – continued from previous page Did you feel any sort of connection with them in the video? Not really. Any enjoyment of watching the video? A little yeah, it moved a little bit too slowly. It would take a while for the Beluga whales to come and stuff. Arthur (Asked about video in the background.) The video in the background? I don’t find it helpful (. . . ) Yeah, I didn’t really see it. Amanda Then we saw the whales but we, I don’t know if they were the same ones or different whale? Is it the same reel but repeated or like, different reel? (. . . ) We saw the big one and it was sort of like. . . waaahhh (they laugh). It was a big whale. When the sound was traveling we heard this noise (imitates beluga voice). I told you (imitates again, both laugh) (. . . ) Like the beluga was crying or something. Audrey I don’t know, but yeah the video was sort of, you couldn’t really see the whales that well. (Amanda: no.) It was kind of a blurry feel. (Amanda: mmhmm.) Did that frustrate you a little bit? Well yeah, sort of like. . . (Asked about seeing the Belugas at the Vancouver Aquarium.) (. . . ) You know how they have that thing underneath? Like, I’ve seen that and that’s cool. (. . . ) I thought they were cool they were (. . . ) they were really neat cause the ones, when we went, one of them was sticking its tail out and slapping the outside and when we saw them on top we could see the tail and then down at the bottom you could see them sort of like, vertical? (. . . ) You know the tails out on the outside and it’s kind of neat to see that (. . . ) Yeah, I like the beluga whales. They’re probably the best things there honestly. Continued on next page 162 Appendix B. Study Results Table B.7 – continued from previous page We were mostly focused on the balls, the little circles. The belugas were sort of just the background. It seems like we never thought about connecting them. So like, what was the concept? (Echology conception with webcam feed explained.) Oh yeah that’s why. Ben I don’t think it’s very interesting. Cause you know, the screen showing up there, it’s not close to a real thing, it’s just a figure there. I mean, if there’s real whales somewhere, even if it’s just. . . it’s not really thing so. The whole experience is about the whale I would say, I tried to, cause I read article about the whale that said the whale sound is very ahh, very special, cause whales can talk to each other through their sound. (talking about tv program viewed in past) But whales definitely, they can understand each other, and then. I think whales are very good animal, very smart animal, just like a, even smarter than dogs, my understanding of them (. . . ) they communicate well with human beings. If you could say anything to the whales, is there something you would say? I would like to kiss them (laughs). Whales really smooth right? I think whales know they are very friendly to human beings. The only thing I don’t like is just, a lot of time it’s not like, real whales. It’s just figures, still very interesting (. . . ) you touch something and then you hear something. Continued on next page 163 Appendix B. Study Results Table B.7 – continued from previous page Why are the whales never going to. . . like let’s say you have to touch it, touch the button like you touch the whales and there, on the screen, close to you, it allows you to communi- cate with the whales, but these ones, they always stay in the same place. Like in the video? Well, because it’s a live webcam feed so those are real whales. Oh! Oh my god, I don’t realize (chuckles) so if I was confused beforeI’m definitely going to pay more attention to how the whale. . . oh, really, it’s real whales? Yeah, so we don’t have much control over where the whales go... Oh my god, that is very interesting, I would like to do it again (. . . ) It’s really interesting, that interesting, that’s real whales over there, no that side, oh, omigosh. I didn’t re- alize. I just think it’s like a computer game stuff. So what’s behind touching the button? Is it actually like, I can touch the whale? (. . . ) When you touch the button over here, and there are balls over here, the whale will know that you touch it? Did anybody actually know what the whale is taking about? The whale language? Burke Yeah like, since, I don’t know about biology and stuff, that species, I just want to have time maybe looking around Van- couver Aquarium next summer – I’ll go this summer (. . . ) Are you advertising for the Vancouver Aquarium? That’s pretty interesting for me to see something live. Continued on next page 164 Appendix B. Study Results Table B.7 – continued from previous page If you could say anything to the whales, is there something you would say? Maybe like, how long can a whale live like, what does he eat, or something like that. How could they keep it alive for a long time in that kind of place, like, what was the person in charge thinking the first time when he gets the whale and puts it in there. Ok, so you are thinking a little bit about whale captivity? Yeah, maybe not really, but how, not what, but how. Barbara I was thinking about the time I actually went to the Aquar- ium (. . . ) Since the sound thing, it was whales. . . If you could say anything to the whales, is there something you would say? I think I would ask them to come near me so that I can touch them. (Asked about video in background.) I thought that it worked with the sound, but then I was distracted by the circles going around. Beth I think the video seemed to be a little bit static, like it was just images that changes every once in a while. Perhaps because I was too focused on trying to press the buttons to get the sound (. . . ) but when I did pay attention to the background video, I thought, it seems to be just images of, I think there was two Beluga whales? Byron At first, I have thought that that was not the animal that you mentioned, I have thoughts that that thing, I don’t know how to say that in English. Like a beauty fish but like, a beauty woman Oh Mermaid. Yeah, at first I thought that was it. Continued on next page 165 Appendix B. Study Results Table B.7 – continued from previous page Carl There are sounds that are like (. . . ) I think it’s kind of sad. It’s like (. . . ) maybe it feels not that good. Kind of but I don’t know how to like, release those kind of pains they’re maybe feeling, like, try to cheer them up. I don’t know (chuckles). I also don’t know the answer to that question, if this is live or not live, I don’t know, just repeated things, repeated sounds. I think this, is uh, just a sequence of image I don’t think it’s, it’s not as realistic as you like, stand in front of, just separated by a glass and you see whales there and they are swimming and coming out and it’s more realistic. For me it’s just the sequence of images and video. Ok, it seems to me, if I do something, this will affect the whales in the aquarium (. . . ) It seems like you take care, maybe you can give something to the whales. Maybe the whales are trying to get some food like it’s a game where you adjust those orientation of those reflection points maybe it is easier for them to get food then you won’t, sounds won’t be sad and you get cheerful sounds. I was trying to see whether it works (. . . ) after I while I thought it just a sequence of video and sounds get repeated and I change orientation of reflection point. Claude The video in the background, I didn’t focus on the back- ground, I mean on the fish. I didn’t focus on the fishes, just focused on the moving ball and the circles around. The purpose? The purpose just like, you press this to play around with the fish. That is what my guess. Continued on next page 166 Appendix B. Study Results Table B.7 – continued from previous page Cory If you could say anything to the whales, is there something you would say? I would ask how they feel about living in such a place that’s not exactly their environment. (Asked what he thinks a Beluga-whale related message of the artwork might be.) Maybe some- thing with beluga whales, their environment. I guess cause that’s what my thing is so I would say, their lifestyle and their habitat and they’re just trying to say something to humans and then we can’t actually relate to them because we don’t understand them as well as we think we do. Do you know a lot about like whale conservation and whales in captivity and stuff? Yeah. What are your thoughts on that? It’s not good, their not doing a very good job of it. Did you identify with the whales at all? My thoughts on them is like, pretty crazy but. . . I would identify myself as a whale because, I dunno, this environment maybe, like, being put in a different place, in a different home I guess, that sort of thing. That’s what I’m going into actually, macro biology and envi- ronmental sciences. Do you have a particular interest in whales? Yes I do. Chris What were the things that you didn’t like too much? I think it would be the resolution of the screen is not too high so sometimes the image of the whales is a little bit blurry and it’s not so clear. I think my wife would be interested in this because she loves animals Charles If you could say anything to the whales, is there something you would say? Are you bored in there? (we laugh) Yeah, it’s so small place. Continued on next page 167 Appendix B. Study Results Table B.7 – continued from previous page Did you pay more attention to the sounds or the belugas? The sounds cause I was playing with the directions of the sounds I guess. Cassandra (Asked about video in background.) Which one? (. . . ) There is a background video? ...with the beluga whales swimming... Oh yes! So you mean like, on the screen there is some whales, some swimming. Okay, I un- derstand. Did you sort of notice connection between them and the sound? Sometimes yes. Deborah (Asked about the video in background.) Oh, yeah, I didn’t really notice it. I more focused on the sound. At first I don’t really notice the Belugas but then like, ok, there is something swimming around so I look at it (. . . ) Belugas, oh yeah, but at first I didn’t really notice it. Denise I was a bit kind of confused I think because it talked in the instructions about the visual thing but it was actually kind of hard to see the whales on the screen (. . . ) Is there a strong link between the whales and the sound I think, cause the screen quite small. The pictures were small and it was actually quite hard to see the whales so it was a bit disconnected for me, from I think, the whales. It was like, these noises going around. Is the sound the actual sounds that the whales are making at that time? David I imagined swimming around whales. (Asked to describe experience to hypothetical friend.) I would say that it would be a place to familiarize or interact with whales, to know more about whales, and the way they communicate. 168 Appendix B. Study Results Table B.8: Participant statements related to participant ex- pectations and technique. Anne I should’ve checked what Echology was before I came here. Andrea I just have an overview of the room and then I just went to the table and watch the screen and just play (laughs). (Asked to describe experience to hypothetical friend.) I would tell them at the beginning I have no idea and at the end I still have no idea what is the right answer or the wrong answer. I just found, it’s weird, that’s it. Do you think that there needs to be an answer? If it is not a test I don’t think so. It maybe could have a score, you know, if you catch it you can get a high score? That’s my suggestion. Ambrose Are you experiencing any uncertainty or confusion? Yeah a bit of, I’m trying to figure out what those buttons actually do. Not really sure how it has a purpose exactly except maybe to - I know it’s using the Beluga whale sounds because the experiment says so and then you have the occasional whale floating through the screen and, but other than that, I am not really quite sure what it would mean. Did you come up with anything on your own? Not really, maybe a sort of oceanic experience of some sort. (Asked about reading card on table.) Yeah. I went back to look at it a few times (. . . ) will this give me ad- ditional information as to how this exhibit works. (. . . ) By the time I left I had already figured out all I felt I needed to. Continued on next page 169 Appendix B. Study Results Table B.8 – continued from previous page Arthur When I saw the table like, I was confused like, what should I do now? Then I pressed a button there and the sound starts playing. I read the card but in the middle of my reading I just pressed a button. I played around a little bit like, 2, 3 minutes but then, after I know how it works I just call you. I just wanna know how it works, like the sound or whatever, how it works, that’s all, not like the process itself. I’m an electrical engineering student (. . . ) so I probably don’t enjoy art that much. (Asked to describe experience to hypothetical friend.) I went to an experiment and the experiment is like, how you move different sound to different speaker, I’ll probably say that, not like the beluga and Vancouver aquarium stuff. . . I’ll probably say just the technology is like there’s (. . . ) lots of speaker and the sound goes through that speaker and you feel it, you feel the movement. (Had a friend who did the study before him.) It’s funny actually because after I spoke with my friend like, she talks about the art stuff instead of like the technology side and like, now I know ok, she probably don’t like the technology side more than the art side (. . . ) She’s just telling me that there’s like, the movement of sound on this experiment but like, that’s all. Did she tell you how it works? No Did you have expectations? No, it’s like, I know that I’m gonna do this, I don’t really wanna ask her before that. Continued on next page 170 Appendix B. Study Results Table B.8 – continued from previous page Have you done other studies? Yeah a lot of them actu- ally. Did you think that you were just going to sit in front of a computer and... I think I would do that yeah, because all of my studies, they are all like that. Like, one person makes a program and I have to like, play with it or like, see which one does better or something. Lot’s of them have to do with graph, plots, and like, also personality stuff. I like to know how stuff is working so that when I first came into the room instead of enjoying it has an art I just, I figure out what does this do and something like that. Amanda (. . . ) Then we read the instructions and we understand and, it was sort of understandable thing. (Audrey: Yeah we were trying to figure out what the point of the thing was. Were you trying to figure out what the point was as in, why we made it or, what it does? Like, how it works. We want to just keep the sound out. What do you mean out like, not playing? Uh-huh, well we turn all the circles around to the closed circles (?) but then if you press it again you will have different sounds like, it will be one. Like okay, let’s play and see what happens. And we start turning the. . . circles? You know, on the screen and make changes in the position and (?) and then we want to just keep the sound up (Audrey agrees) and play (. . . ) for me at least. Audrey Yeah well, first we were trying to figure out like how pressing the thing would do, and what the little balls were and how it related to the sound. Continued on next page 171 Appendix B. Study Results Table B.8 – continued from previous page (Asked about previous experience with interactive art.) Well yeah I went to one and they had like mechanical things that would move but nothing like, electronic or anything, or sound or anything, visuals (. . . ) No wait, they did have a tv thing once (. . . ) there’s some weird words flash on the screen, but other than that I’ve never had interactive where I am actually touching, just watching at the art gallery. I was frustrated for a little bit of it because I just didn’t see the point (Amanda agrees) Did you try and think of what the point might be? No not really, I just thought ok (Amanda: let’s play) Yeah (both laugh). Are there still questions that you have... Yeah like, I’m not so, this is supposed to be like an art display with sound? Is that what it is? (Talk about interactive art for a while.) It just looked like the point of it was to be an art piece (. . . ) Do you think it would be maybe a bit different if the experience of going to an art, knowing that it was like an art specific thing that. . . abstract. There’s really no just like art I can just go and look at it and try to find something the artist is trying to say. Like if I had come in from that point of view then it probably would have made more sense but because, it was just that by itself in a room and I had never seen or heard anything from it, it was really confusing. (Amanda: and you think that it’s a game) Yeah, you’re expecting some sort of activity. Ben At first I tried to find the camera (. . . ) I’m just curious because you just say there are buttons here so I just touch and try to play around with it so I am just curious (. . . ) that was my first feeling about it. You just play around and you touch it and the whales swim around and different figures show up there. Continued on next page 172 Appendix B. Study Results Table B.8 – continued from previous page I tried each of them, actually, and I tried to figure out what’s the difference between those balls, but I don’t think there’s a big difference there. I also pay attention to the music and actually at the very beginning I didn’t notice the music there. At the very beginning, I feel really curious, not sure what I’m doing there, but I have to read the manual, read the instructions and then decided what I can do. (talking about the sound) . . . at last I just ignore it (. . . ) didn’t pay attention to the sound, I try to focus on the button stuff. I’m very curious about the whole thing and how the whole thing works cause I’m a computer guy so I try to figure out how something works. Burke I walked around and then I looked at the speaker and stuff and then I started reading the instructions on the table and it says like, I should push the button and then I pushed it one by one (. . . ) Barbara (Asked to describe experience to hypothetical friend.) I walked in, I didn’t know what to do at first when I walked and she just left. I felt like, why she going away (laughs). I thought, oh she should give me some instructions but she just left so I started to walk around the room and I saw these lights and I saw this screen and then I start to press the buttons but I couldn’t figure out what it was so I start to read the instructions and then I start to play with the buttons and it worked. Do you have things that you are wondering about? What’s this for? What do you think it is for? For me, it just sounds like, for the one, when you cannot go to the aquarium and want to, or like, when you want to relax. Continued on next page 173 Appendix B. Study Results Table B.8 – continued from previous page Beth Rather than reading through the card carefully I just skimmed through it and got a general idea. I more so paid attention to the coloured balls on the screen then followed the sound even though they were kind of like, like, all the sounds were like, just coming out all around me, I was somehow just paying more attention to the coloured balls. I was more thinking like, how to keep the sound going as well, like, realizing that I made the circles in the perimeter donut shaped they would stop so I would try to make the circles be able to bounce from one end to the other to have kind of two openings. (Asked to describe experience to hypothetical friend.) I did this interactive study today that happened in this darkened room where I could just explore how the sounds that I heard made me do certain things like press buttons or pay attention to the screen, while also having me to try and think about what all that I said before had to do with Beluga sounds. Byron After I came into the room I tried to read the green card to know what we should (. . . ) I try to change the direction of the sound by pressing the buttons. If I hear that sound is from that direction I hit this button. I think by pushing buttons, sometimes I can trick, by pushing buttons sometimes I can trick the sound to happen. Continued on next page 174 Appendix B. Study Results Table B.8 – continued from previous page My experience is that I need to push several buttons to chase the sound. I need to push several buttons and then there can be some kind of sound. But if I just push button one time and then, I need to push several times and then there will be sound. Yeah you can see I always move my hand right. I’m trying to catch the sound here push button and there push button. (Asked to describe experience to hypothetical friend.) You push buttons, you catch sound by pushing these buttons, you can use your ears to catch the sound, the movement of the sound. I think next time when you have other people do this ex- periment, maybe you should have a, not step by step list, but perhaps detailed instruction because I think we at least should have read the green card here at the very beginning and not in the room. Cause I think that maybe if I read the green card here in this room maybe I could perform better. Carl I just wonder what’s around this and like, how it works. Just kind of just curious. I went to see whether these buttons were control buttons to the speakers and if I press one, if one of your speakers would sound. No I just wondering what would the sounds maybe sounds like if I try different settings to see (. . . ) want to hear the sounds, the system response to those settings. Continued on next page 175 Appendix B. Study Results Table B.8 – continued from previous page By curiosity I want to know if I do this, what will happen next at the same time when you hear this sound, sometimes it’s cheerful, sometimes it’s kind of, it’s not good feeling, when I hear those kind of sounds. But maybe by curiosity I just want to, how can I make those cheerful sounds appear again and try to avoid those (?) sounds. Try to figure a way to like, I want better feeling inside. I tried to figure out which speaker is corresponding to. . . because there are 8 of those buttons and only 6 working speakers and I tried to figure out which speaker is corre- sponding to which button (. . . ) Were you hoping that there would be instructions? Yeah, um, a little bit. Like just, something, you press this button and what will happen if you like, press that button. Like, I even tried to turn this button because I didn’t know whether to press or turn to change that direction (. . . ) I tried to figure out the rules, the settings. (Near end of interview) Yeah, I’m still trying to figure out. . . Claude I can play around with many things. When I got into the room, I thought just computer, I can play around here (. . . ) I might walk around before I came to the table. I try just like, move the ball, actually it’s randomly, yeah. I arrange it just randomly. But I try to form some triangle or something like that. The speakers, I thought there should be something on the speaker, not just the table because it’s like a very (?) en- vironment, a very spatial place and what I can play around with is just table. That is why when I stepped in the room, I walked around to see (. . . ) Or don’t show me the speakers, that is okay. Continued on next page 176 Appendix B. Study Results Table B.8 – continued from previous page Cory I’m looking at the screen, trying to follow the sounds. That’s when I read the cards like, the second time (. . . ) I can press these buttons and then I can change the directions and then, make the noise go where I want it to go. Chris So when I first came in, I look around the room and the space and I tried to push the buttons and see how the reaction would be (. . . ) I walk around and try to stand at different positions around the table and I want to see whether there is any difference (. . . ) Charles Before I press any buttons I read the cards so I know I should press any button to see the ball reflect around (. . . ) But I still used maybe like, 30 seconds, to actually manipulate anything (. . . ) I was trying to figure out when the sound actually comes on. Cause sometimes it is very loud and sometimes it fades away and the sound is quite, like, it is not as loud as other times. I tried to make the sound go like a circle and I found that if I like, when I try to change the direction of one node, and if I make the angle too large it will become like a ball with the circle inside and it will trap the sound inside. And I would trap maybe 2, 3 balls inside and then I release them. I also make two nodes face to face and maybe 2 or 3 sound balls go between those two. I just think, I should like, try to find more about this to an- swer your questions better. What kind of questions did you expect that we would... I don’t know, that’s why I read the cards maybe 3 or 4 times, and try to find something like a clue about your questions. Ok, so did you feel that you might be quizzed or tested? Yeah, probably (. . . ) when the colour of the ball becomes darker, you may ask why. Continued on next page 177 Appendix B. Study Results Table B.8 – continued from previous page Cassandra (. . . ) I don’t quite understand that so I try to play on the buttons so at that time I didn’t read the instructions so I tried several times, I guess, 5 minutes or more I tried to press all the buttons and see what’s going on. (. . . ) Then I tried to figure out what is going on and I find the instruction cards there so I tried to read it Did you read any of the cards under it? No I didn’t. (. . . ) So the whole purpose is, I was trying to figure out what I should, the result that I can get from such a game but I still don’t, didn’t figure it out. (Asked to describe experience to hypothetical friend.) It would be a little difficult for me to describe this because I don’t know the goal of such a setting so I would just simply say there is this screen with a lot of buttons around and there’s a (?) around the settings so if you press a button then the angle of each button will change its direction, then you can play with these buttons and change the direction of the balls. I think that’s all, cause I’m not quite clear about the goals. I also tell them that it’s just like a game but I’m not clear about the, about your goals. What’s the purpose of such a setting? Echology concepts explained. So when I press the button and change the direc- tions maybe it’s a little bit angle so the sound will change? I didn’t notice that. Sound control explained. Yeah I also noticed that sometimes like, the red one will be very strong and sometimes it will be dim, the colour will change. Simplicity and playing with whales explained. Ok so just like, people and whales can play with each other? Okay (. . . ) I didn’t realize that when I was playing that. Yeah, probably it seems like a good piece, seems like a good idea (. . . ) I would like to know that beforehand so that I can play with it, so that like, purposefully I can try it. But, if I didn’t know that maybe I can never figure that out. Continued on next page 178 Appendix B. Study Results Table B.8 – continued from previous page Deborah From that description in your consent, I’m not really sure about what I am going to do and when I walked into the room I was like, ok there’s buttons so what should I do? I don’t know. And then I read the cards, the first card, the top one, and I thought ok, I will try to push it and I sort of get it, what I’m doing. Denise It wasn’t really what I was expecting. I think that maybe I felt that there would be more of a visual thing, as well as the sound. (Asked about reading the instruction cards.) I read the cards. I think if we had had a little bit more time with the noises, that would have been a bit better. (Asked about opinion of interactive art.) I think it’s more of a personal preference but I’m just not really into (. . . ) and I guess, it’s kind of frustrating too, if you don’t know, what it’s meant to do or what you can do with it. You can press buttons or do things on the screen and nothing happens or, I think it’s kind of frustrating when you try to interact so I just watch. David I was expecting something dark and blue, because of the aquarium. (Asked about reading the instruction cards.) I was kind of reluctant to read them. How come? I don’t know it was more interesting pushing the buttons and looking at the screen. I was more interested in figuring out what was this about (. . . ) and associating the sounds to the ball bouncing to one side of the circle. Continued on next page 179 Appendix B. Study Results Table B.8 – continued from previous page It would be better if the screen has something more different, more interactive, more interesting than the pacmen turning around (. . . ) Cause you have to wait for the balls to get to the point where the sound is resurrected right? So that’s where you come into play, you have to wait for that so, when there’s no sound you get frustrated, and when the sound is moving around then you can start pushing the buttons. I totally prefer to interact, it makes it easier to understand, and more fun and maybe, I can appreciate art but, whenever I can touch it, and I can do something with it, it makes me more focused. 180 Appendix B. Study Results Table B.9: Participant Statements related to breakdowns. Ben There’s a button here and I’m not sure why it didn’t go on (referring to video) cause I didn’t touch it at that time but it’s still gone (. . . ) cause that’s why I’m curious about it and I try to read the instruction manual to figure out why it’s gone (. . . ) And then I try to touch the game and then it’s back, you see what I mean, you see what I mean? That was confusing? Yeah I try to find the magic or something there. Not confusing, it was just, I was trying to think of if it was designed that way, even when I didn’t touch anything its got no light there. Burke I pushed it one by one (. . . ) nothing happens and then, alright, what was I supposed to be doing and then I walked to the laptop and then saw what was on the laptop but then nothing’s happening and then ok, like, I decided to call you. Were you frustrated? Not really frustrated, but, I was like, just kind of weird, like, I was supposed to play but then, ok, and then I just decided to call you because maybe something is wrong, that’s what I was thinking. Are you at all disappointed that when you first came in it wasn’t working? Disappointed? Hmm. . . no, cause I know that there might be something wrong and I’m pretty sure that you can fix it cause you are like, the person that makes the installation work, I’m not, but then I’m not feeling like I was frustrated or something cause I was thinking like, maybe I did something wrong or maybe I didn’t understand the guideline very well and that’s why I called you. Continued on next page 181 Appendix B. Study Results Table B.9 – continued from previous page What do you think the purpose of Echology is? Maybe it’s like to get to know how a person feels the first time when things are not in proper? And then, but after things are working very well, you might want to investigate how a person maybe plays with the (?) and how curious that person is. . . I’m maybe wrong (laughs). Did it make up for the fact that, well, you sort of understand the webcam technology and it slows down... Oh yeah, I know that, and it depends on the Internet connection. . . if the connection slows down. Barbara Did you call me right after the music had stopped? No, I waited for a while and it wasn’t. . . I thought I did something wrong so I keep pressing buttons, and I thought it wasn’t working, and then I thought that maybe the session ended, like automatically. That’s why I called you. Denise (. . . ) When all the sounds stopped, we were kind of like, why did the sounds stop and what’s it really like, is there a strong link between the whales and the sound I think, cause the screen quite small. I think if we had had a little bit more time with the noises, that would have been a bit better. I thought it was interesting until the sound stopped. And then we were sort of like, in a dark room with these buttons that weren’t doing anything (laughs) and we’re kind of like, what the hell are we supposed to be doing, what the hell is this? It doesn’t do anything. I wondered if it only was, like if it only went for two minutes or something, and then at the end of the two minutes it was done and nothing else happened. Continued on next page 182 Appendix B. Study Results Table B.9 – continued from previous page David When did the whales stop, was that pretty much right after you guys... Like, umm, five minutes it stopped. (Denise: Yeah, they went for a bit.) Yeah cause I got impatient because nothing was happening. There were not any lights. So many things around and noth- ing happened. Ok so at this point here, is there any noise right now or are you guys kind of... Yeah there’s no noise. There’s nothing moving around so there is no sound. And so you just kind of sat there and... . . . and pushed everything. Did you think that if you pushed something that the sounds would start again? (both) Yeah Then did you guys go back to the cards? (both) Yeah. I also wondered if I touched the wrong button or if I moved the things the wrong way. 183 Appendix B. Study Results Table B.10: Participant statements related to collaboration. Andrea I find the table is too large (laughs) and actually I can work around the table but I find if I stand at one point in place it is more convenient to concentrate. Barbara It was hard to reach the other side so I was a little bit frus- trated about that. Beth (Mentions skimming through instructions then) I think it was my partner who first started touching the buttons. So were you guys working together at all to try and? Not really (Byron: No.) I did notice that near the end of my partner trying out, that there seemed to be more sound coming out and he was defi- nitely pressing a lot of buttons. Denise Did you tell him what the cards were saying? Not really, I think we talked a little bit about what, when you were moving around, pressing the buttons and changing the directions did to those circle things. 184 Appendix C BREB Certificate of Approval Figure C.1: Certificate of Approval received from the UBC Behavioural Research Ethics Board (BREB) 185


Citation Scheme:


Usage Statistics

Country Views Downloads
China 10 32
United States 2 1
South Africa 1 0
Macedonia 1 0
City Views Downloads
Beijing 10 0
Washington 2 0
Johannesburg 1 0
Unknown 1 1

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}


Share to:


Related Items