Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Participant experience studies of interactive artworks : an investigation of laboratory-based methods.. Deutscher, Meghan Catherine 2008

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata


ubc_2008_spring_deutscher_meghan.pdf [ 2.22MB ]
JSON: 1.0066354.json
JSON-LD: 1.0066354+ld.json
RDF/XML (Pretty): 1.0066354.xml
RDF/JSON: 1.0066354+rdf.json
Turtle: 1.0066354+rdf-turtle.txt
N-Triples: 1.0066354+rdf-ntriples.txt
Original Record: 1.0066354 +original-record.json
Full Text

Full Text

Participant Experience Studies ofInteractive ArtworksAn Investigation of Laboratory-Based Methods Used toStudy EchologybyMeghan Catherine DeutscherBaSc., The University of Regina, 2003A THESIS SUBMITTED IN PARTIAL FULFILMENT OFTHE REQUIREMENTS FOR THE DEGREE OFMaster of Applied ScienceinThe Faculty of Graduate Studies(Electrical and Computer Engineering)The University Of British Columbia(Vancouver, Canada)April, 2008c Meghan Catherine Deutscher 2008AbstractWe investigate the use of laboratory-based methodology for studying par-ticipant experience of interactive artworks. The investigation is motivatedby two goals: to inform the HCI practitioner of the role of participant ex-perience studies in artwork from the perspective of the artist and to informthe artist of how laboratory-based methodology can contribute to the refine-ment of their techniques and aesthetics. In this thesis three main purposesfor participant experience studies in the artist?s process are derived fromthe roles of artist, art object, and participants in an interactive artwork.Common characteristics of participant experience studies are reviewed, withthree cases unique in their use of more formal methodologies examined indetail. This thesis builds on a foundation set forth by these three cases in aninvestigation of orientation media: media such as text, images, or videodesigned by the artist to convey supplemental information to participantsand thus selectively influence their understanding of different elements inan interactive artwork. Orientation media in the form of instructionscards is used in a study of the interactive sound and video installation piece,Echology. The orientation media is successful in revealing elements of theartwork that, given explicit instructions or not, still cause confusion amongparticipants. A general review of the study methodology is also provided.This includes observations of changes in participant behaviour due to theirroles as subjects in a study and implications these changes have on usingformal methodologies for studying participant experience.iiTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . xiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Problem Scope . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1.1 The Interactive Arts . . . . . . . . . . . . . . . . . . . 31.1.2 Collaborative Sound and Video Installation Art . . . 61.1.3 Participant Experience Studies . . . . . . . . . . . . . 71.1.4 Interactive Arts and HCI . . . . . . . . . . . . . . . . 81.1.5 Differences Between HCI and the Interactive Arts . . 91.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . 111.3 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.4 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Interactive Art and the HCI Process . . . . . . . . . . . . . 162.1 Defining Interactive Art . . . . . . . . . . . . . . . . . . . . . 172.1.1 Defining ?Interactive? . . . . . . . . . . . . . . . . . . 172.1.2 Background in Participatory Art . . . . . . . . . . . . 18iiiTable of Contents2.1.3 The Interactive Art System . . . . . . . . . . . . . . . 192.2 Contributors to the Aesthetics of Interactive Artworks . . . . 222.2.1 Roles of the Art Object . . . . . . . . . . . . . . . . . 232.2.2 Roles of the Participant . . . . . . . . . . . . . . . . . 272.2.3 Roles of the Artist . . . . . . . . . . . . . . . . . . . . 282.3 Participant Experience Studies of Interactive Artworks . . . 292.3.1 Purposes of Participant Experience Studies . . . . . . 292.3.2 Characteristics of Participant Experience Studies . . 302.3.2.1 Goals . . . . . . . . . . . . . . . . . . . . . . 302.3.2.2 Methods . . . . . . . . . . . . . . . . . . . . 322.3.2.3 Settings and Subjects . . . . . . . . . . . . . 332.3.2.4 Form of Results . . . . . . . . . . . . . . . . 332.4 The HCI Process . . . . . . . . . . . . . . . . . . . . . . . . . 352.4.1 User-Centered and Interaction Design . . . . . . . . . 352.4.2 Parallels in HCI and Artistic Processes . . . . . . . . 382.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Formal Participant Experience Studies . . . . . . . . . . . . 403.1 Case One: The Influencing Machine . . . . . . . . . . . . . . 413.1.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.1.2 Artwork . . . . . . . . . . . . . . . . . . . . . . . . . 413.1.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . 423.1.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 433.2 Case Two: Iamascope . . . . . . . . . . . . . . . . . . . . . . 433.2.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.2.2 Artwork . . . . . . . . . . . . . . . . . . . . . . . . . 443.2.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . 443.2.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 463.3 Case Three: Fa?ade . . . . . . . . . . . . . . . . . . . . . . . 463.3.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . 463.3.2 Artwork . . . . . . . . . . . . . . . . . . . . . . . . . 473.3.3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . 473.3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 48ivTable of Contents3.4 Contributing to Existing Study Methodology . . . . . . . . . 523.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 Echology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.1 An Interactive Sound and Video Installation . . . . . . . . . 554.1.1 Echology Summary . . . . . . . . . . . . . . . . . . . 554.1.2 Artistic Motivations . . . . . . . . . . . . . . . . . . . 564.2 Echology Design Guidelines . . . . . . . . . . . . . . . . . . . 564.2.1 Collaboration, Simplicity, and Approachability . . . . 564.2.1.1 Collaboration . . . . . . . . . . . . . . . . . 574.2.1.2 Simplicity . . . . . . . . . . . . . . . . . . . 584.2.1.3 Approachability . . . . . . . . . . . . . . . . 594.2.2 Designing for the Open Media Environment . . . . . 604.3 The Echology System Description . . . . . . . . . . . . . . . 604.3.1 Beluga Video and Motion Capture . . . . . . . . . . . 634.3.2 Soundscape Spatialization . . . . . . . . . . . . . . . 644.3.3 Participant Control of Sound Spatialization . . . . . . 694.3.4 Graphical Feedback . . . . . . . . . . . . . . . . . . . 704.3.5 Phidget Interface . . . . . . . . . . . . . . . . . . . . 714.3.6 Lighting . . . . . . . . . . . . . . . . . . . . . . . . . 714.3.7 Interaction Table . . . . . . . . . . . . . . . . . . . . 724.3.8 Musical Elements . . . . . . . . . . . . . . . . . . . . 744.4 Desired Echology Experience . . . . . . . . . . . . . . . . . . 754.4.1 Participant Experience Summary . . . . . . . . . . . 754.4.2 Connecting with the Belugas . . . . . . . . . . . . . . 764.4.3 Participant and Art Object Roles . . . . . . . . . . . 774.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 785 Studying Participant Experience of Echology . . . . . . . . 795.1 Informal Observations at NIME . . . . . . . . . . . . . . . . 805.1.1 Summary of Observations and Feedback Received . . 805.1.2 Questions Remaining . . . . . . . . . . . . . . . . . . 815.2 Laboratory-Based Experience Study . . . . . . . . . . . . . . 83vTable of Contents5.2.1 Orientation Media . . . . . . . . . . . . . . . . . . . . 845.2.2 Participants . . . . . . . . . . . . . . . . . . . . . . . 845.2.3 Setting . . . . . . . . . . . . . . . . . . . . . . . . . . 855.2.4 Study Procedures . . . . . . . . . . . . . . . . . . . . 865.2.5 Data Collected . . . . . . . . . . . . . . . . . . . . . . 885.3 Study Results . . . . . . . . . . . . . . . . . . . . . . . . . . 895.3.1 Visual and Auditory Enjoyment . . . . . . . . . . . . 895.3.2 Understanding of Echology Interactivity . . . . . . . 905.3.3 Connecting with the Belugas . . . . . . . . . . . . . . 925.3.4 Collaboration . . . . . . . . . . . . . . . . . . . . . . 935.3.5 Participant Behaviour Tendencies . . . . . . . . . . . 935.4 Interpretation of Study Results . . . . . . . . . . . . . . . . . 975.5 Discussion of Study Methodology . . . . . . . . . . . . . . . 995.5.1 Participant Behaviour as a Study Subject . . . . . . . 995.5.2 Orientation Media Review . . . . . . . . . . . . . . . 1025.5.3 Benefit of In-depth Data Collection . . . . . . . . . . 1025.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1036 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1046.1 Thesis Problems . . . . . . . . . . . . . . . . . . . . . . . . . 1046.2 Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . 1056.3 Laboratory-Based Methods in the Artistic Process . . . . . . 1076.4 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 1086.4.1 Study Methods Based on Artwork Characteristics . . 1096.4.2 Aesthetics of Study Result Presentation . . . . . . . . 1106.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112AppendicesA Study Material . . . . . . . . . . . . . . . . . . . . . . . . . . . 123A.1 Participant Recruitment Notice . . . . . . . . . . . . . . . . 123viTable of ContentsA.2 Orientation Media: Echology Instruction Cards . . . . . . . 124A.3 Orientation Media: Echology Introduction Cards . . . . . . . 125A.4 Interview Question Guide . . . . . . . . . . . . . . . . . . . . 126A.5 Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . . 127B Study Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128B.1 General Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 128B.2 Questionnaire Results . . . . . . . . . . . . . . . . . . . . . . 131B.3 Interview Transcript Excerpts . . . . . . . . . . . . . . . . . 134C BREB Certificate of Approval . . . . . . . . . . . . . . . . . . 185viiList of Tables1.1 Differences in Studies Conducted in HCI and in the Interac-tive Arts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.1 Summary of Three Cases of Formal Participant ExperienceStudies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.1 Design Guidelines for Echology as a Collaborative MusicalExperience . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574.2 Echology?s Roles as an Art Object . . . . . . . . . . . . . . . 774.3 Participant Roles in the Echology Art System . . . . . . . . . 78B.1 Study Participant Information . . . . . . . . . . . . . . . . . 128B.2 Echology Experience Durations . . . . . . . . . . . . . . . . . 130B.3 Questionnaire Results: Free Input Responses . . . . . . . . . 131B.4 Participant Statements Related to Echology as a Game andEchology as Art . . . . . . . . . . . . . . . . . . . . . . . . . . 134B.5 Participant Statements Related to Understanding of Echol-ogy Interactivity . . . . . . . . . . . . . . . . . . . . . . . . . 139B.6 Participant Statements Related to Sensation and Enjoyment . 152B.7 Participant Statements Related to Connecting with the Bel-ugas and the Live Webcam Feed . . . . . . . . . . . . . . . . 161B.8 Participant Statements Related to Participant Expectationsand Technique . . . . . . . . . . . . . . . . . . . . . . . . . . 169B.9 Participant Statements Related to Breakdowns . . . . . . . . 181B.10 Participant Statements Related to Collaboration . . . . . . . 184viiiList of Figures1.1 Thesis Scope Diagram . . . . . . . . . . . . . . . . . . . . . . 51.2 Thesis Problem Scope . . . . . . . . . . . . . . . . . . . . . . 141.3 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . 152.1 The Art System . . . . . . . . . . . . . . . . . . . . . . . . . . 202.2 Art Classification Systems . . . . . . . . . . . . . . . . . . . . 212.3 Roles in the Art System . . . . . . . . . . . . . . . . . . . . . 232.4 Parallels in HCI and Artistic Processes . . . . . . . . . . . . . 373.1 Iamascope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453.2 Fa?ade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474.1 Echology Sound and Video Installation . . . . . . . . . . . . . 554.2 Echology System Diagram: Physical Components . . . . . . . 614.3 Echology System Diagram: Inputs and Outputs . . . . . . . . 624.4 Layering of Beluga Webcam View . . . . . . . . . . . . . . . . 644.5 Steps of Video and Motion Processing . . . . . . . . . . . . . 654.6 Two-dimensional Sound Plane . . . . . . . . . . . . . . . . . . 674.7 Eight Reflection Points . . . . . . . . . . . . . . . . . . . . . . 674.8 Reflection Point Direction . . . . . . . . . . . . . . . . . . . . 684.9 Sound Path Example . . . . . . . . . . . . . . . . . . . . . . . 694.10 Interaction Table Graphics . . . . . . . . . . . . . . . . . . . 704.11 Suspended Light . . . . . . . . . . . . . . . . . . . . . . . . . 724.12 Interaction Tabletop . . . . . . . . . . . . . . . . . . . . . . . 734.13 Interaction Table with Screen Removed . . . . . . . . . . . . 735.1 Echology Shown at NIME . . . . . . . . . . . . . . . . . . . . 80ixList of Figures5.2 Study Session Type Matrix . . . . . . . . . . . . . . . . . . . 835.3 Study Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . 86C.1 Certificate of Approval from the Behavioural Research EthicsBoard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185xAcknowledgementsTo my supervisor Sid, for your knowledge and unique perspective. I alwaysleave our discussions inspired, motivated, and seeing the world framed ina new way. This work could not have been completed without your greatpatience and support. To friends at HCT, particularly Tony and Ian, for allyour help and for every word of advice. To roommates Eric and Hendrik,for your knowledge and for taking time to listen to me stress and worryabout my thesis. To best friends Kim and Jemma, I would not have had theconfidence to finish this work without your encouragement. Finally, to mymother Donna, sister Sarah, and brothers Luke and Mark, I am so incrediblyfortunate to have you all as my loving family. Thank you for believing inme.xiDedicationThis is for dad ? In his memory I am able to live life fully and with passion.xiiChapter 1IntroductionAdvancements in digital technologies and modern computing have broughtabout rapid development in new technologies and computing practices. Twofields have emerged in this development: Human Computer Interaction(HCI) and the Interactive Arts. Both of these fields are concerned withthe relationship between people and computing technologies and accord-ingly, there are cases of collaboration between HCI practitioners and artists.Furthermore, there are people who work as both HCI practitioner and artist.Recently this collaboration has become more prominent as computershave become ubiquitous and interactive art has moved into the mainstream.As the study of aesthetic, emotional, and social factors becomes vital tocreating good computing experiences, HCI practitioners seek creative inspi-ration and understanding from the interactive arts. In the interactive arts,artists no longer engage through the ?wow factor? of new technology, butthrough good interface design. Furthermore, an artwork?s success is notbased on what the artist has created alone, but is also highly dependent onthe interaction that develops between the artwork and its audience. Theaudience becomes a participant. Thus to evaluate, understand, and refineart techniques, artists engage in studies of participant experience, oftenappropriating user study methods used by HCI practitioners.This thesis explores participant experience studies of interactiveartworks with two motivations:? To inform the HCI practitioner of the purpose for conducting partici-pant experience studies.? To inform the artist of how formal laboratory-based methodologycan contribute to the refinement of their techniques.1Chapter 1. IntroductionHere ?formal? is used to refer to methods that use planned procedures incontrolled settings in an attempt to maintain consistency among participantexperiences studied and data collected.The work begins with a review of art theory, articulating the roles ofartists, participants and ?art objects?, or the artifacts created by artists, inthe production and life of an artwork. Three main purposes for conductingparticipant experience studies are derived from these roles: to develop un-derstanding of interaction and evaluate art object design, to feedback whatwas learned into new versions of an art object, thus creating art objectsthat indirectly learn and evolve, and to create a record of interaction thatbecomes part of the artwork?s content. Common study characteristics aresummarized, showing that they often use informal methods, relying on theartist?s recollection of observations and participant comments made duringa work?s exhibition. Following is a review of the HCI user-centered or inter-action design process. This process is paralleled with the iterative artisticprocess of design, implementation, and study. Similarities and differencesbetween user studies conducted in HCI and participant experience studiesare noted.The above provides a foundation from which three cases of formal par-ticipant experience studies in the HCI literature can be reviewed in detail.These cases exemplify the benefits of methods for in-depth data collectionand controlled experiment settings. They provide a foundation to whichthis work makes its contributions with an investigation of methodology forlaboratory-based participant experience studies through a study of the in-teractive sound and video installation piece, Echology.Echology?s artistic motivations, design, and desired participant experi-ence are presented and then the design of a laboratory-based participantexperience study using orientation media is detailed. Orientation me-dia is media such as text, images, or video designed by the artist and givento a study participant before or during their experience with an interactiveartwork. The media is designed to convey additional information that willinfluence a participant?s understanding of different artwork elements in orderto distinguish the effects understanding of those elements have on interaction2Chapter 1. Introductionexperience. Experiences of participants given orientation media, in thiscase in the form of instructions and descriptions of Echology?s functions,are compared against a control group given no media. The orientationmedia is successful in revealing elements of the artwork that, given explicitinstructions or not, still cause confusion among participants.Finally, study methods used are reviewed. Observations of changes inparticipant behaviour due to their roles as subjects in a study and implica-tions these changes have on using formal methodologies for studying partic-ipant experience are presented.1.1 Problem Scope1.1.1 The Interactive ArtsThis research considers user study methodology for evaluating and under-standing participant experience of interactive artworks. In Chapter 2 defi-nitions of this art form are discussed more deeply but prior to this, a shortintroduction is due. Generally, interactive art is considered ?art intendedfor the viewer?s direct participation? [3]. When experiencing interactive art,viewers or spectators are no longer only passive audience; they are partici-pants. They become engaged in ?a dialog between the piece and the partic-ipant; specifically, the participant has ?agency? (the ability to act upon) thepiece... ? From the 1960s onward, interactivity in art was pushed forwardby an increased interest in participatory artwork [41] as well as developmentsin electrical and digital technologies.The new aesthetics of interactive art are concerned with the relationshipbetween the participant and the artwork. A quintessential example fromMyron Krueger, the pioneer of interactive art [81], is his 1970 installationVIDEOPLACE [61]. VIDEOPLACE is an artificial reality environment con-sisting of a video camera and other sensors to track a participant?s movementin front of a projection screen. Video displayed on the projection screen re-sponds to participant gestures with over 50 compositions. The compositionsmanipulate participant silhouettes and create graphical worlds around them3Chapter 1. Introductioncontaining virtual objects and organisms. The participant in turn can inter-act with movement of their whole body as well as by directly manipulatingelements of the compositions. In VIDEOPLACE, artwork content is onlyrevealed through participant interaction. Full expression relies on partici-pant understanding of how they can interact and furthermore, engagementin an exploration of the content space.Content may also depend on the participant to the extent that the par-ticipant becomes creator of the content. In ?The king has. . .? (2005) KristerOlsson and Takashi Kawashima explore the types of secrets people are will-ing to disclose given an anonymous communication channel [75]. They posttheir own secrets in public, inviting strangers to return a secret by sending ashort message (SMS) from their mobile phone. Secrets received are printedon wooden blocks with red ink to be displayed in gallery spaces.Engaging the public outside of the gallery space is another commontheme in the interactive arts. Many works are found on the Internet or havebeen taken to the city street, activating participation among a more diverseaudience. Blast Theory?s ?Can You See Me Now?? (2001 - present) is a gameof chase played between a set of runners equipped with handheld computersin a city?s streets, and people participating online from their computer [15].Online participants are virtually dropped in random locations around thecity and the runners use their handhelds to track and chase them in the realworld. Online participants navigate their virtual position and can eavesdropon a stream of audio from the runners. A participant is ?caught? if a runnergets within 5 meters of their virtual position. ?Can You See Me Now??may seem to be crossing a line between art and game but many interactiveartworks do indeed involve participant play and game-like elements. In theseworks, an artist may be concerned with how to create game-like interactionexperiences where participants still critically reflect on the meaning of theartwork.This thesis focuses on developing a general understanding of the contrib-utors to aesthetics in interactive art and accordingly, the purpose of studyinginteraction and participant experience. The above examples illustrate a se-lection of issues concerning participant experiences. They do not cover the4Chapter 1. Introductionbreadth of concepts explored and technologies used in the past 30 to 40 yearsand there is no attempt to provide a classification for these in this research.Instead, theoretical definitions of interactive art are used to understand itsaesthetics and within these, purposes for conducting participant experiencestudies.Research is confined to a study of electronic- and computer-based inter-active artworks. Digital technologies have become somewhat synonymouswith interactive art. However, it is important to note that interactive art-works need not be computer- or electronics-based. Also, not all computeror electronic artworks are interactive. To clarify, the Venn diagram in Fig-ure 1.1 shows where there is an overlapping area of interactive artworks thatare computer or electronic-based or both.computerelectrinteractive artresearch areaFigure 1.1: Venn diagram highlighting the scope of this research in thecross-sections of electronic- and computer-based interactive artwork.5Chapter 1. Introduction1.1.2 Collaborative Sound and Video Installation ArtAn investigation of methodologies for studying participant experience is con-ducted through studies of ?Echology? [37] ? an interactive art piece designedand develop by the author of this thesis in collaboration with Sidney Fels,Sachiyo Takahashi, and Reynald Hoskinson. On top of being a computer-based interactive artwork, Echology is also a collaborative sound art-work and an installation piece. It is important to note these last twoclassifications, as they have a significant impact on study goals specific toEchology.Studies of collaborative sound artworks are concerned with the socialbehaviours of participants simultaneously and cooperatively engaged in themanipulation of sound and musical elements. Behaviours of interest becomeones such as cooperation, competition, territoriality, and ability for strangersto communicate and connect (see for examples [20], [51], [63], [77]).In studies of artwork that allow participants to create or manipulatemusical elements, artists are interested in the balance between supportingthe beginner and at the same time, engaging the virtuoso. There is a heavyfocus on expression as the way in which a participant becomes engaged withthe artwork. Simple, easy to learn interfaces provide limited expressionand more complex interfaces provide many dimensions of expression [100].Blaine and Fels [20] discuss this in their work on collaborative musical ex-periences for novices and have identified many of the qualities to organizemulti-person interactive sound installations. They provide guidelines basedon these qualities to make sound interaction successful. These guidelineswere integrated into the design of Echology and will be discussed in Section4.2.Studies of installation pieces are generally concerned with the audienceexperience of a space or an environment. They are interested in the extentthat the artwork overcomes the space in which it is installed (such as agallery room) and immerses the viewer in the synthesized environment [30],[38]. An installation ?addresses the viewer directly as a literal presencein the space? [19]. This contrasts with artwork providing a specific focal6Chapter 1. Introductionpoint for the audience such as a framed painting or sculpture and possiblytargeting a specific sense, as well as with artwork that does not rely on thecontext of the viewer such as art delivered over the Internet.1.1.3 Participant Experience StudiesInteractive artists frequently study participants interacting with their art-works to develop an understanding of the aesthetics of interaction and torefine their techniques. For example, artist David Rokeby speaks of his exhi-bitions as ?a public research laboratory? where his ?ideas about interactionand experience are tested, affirmed, or shot down? [85]. These participantexperience studies usually include accounts of how people were observed ap-proaching and interacting with a piece. This act of observing makes senseconsidering that the product of an interactive artwork is no longer a static,fixed ?thing? that an audience views or experiences. The product includesthe interaction created between audience and some system and the concernsof the artist involve the aesthetics of this interaction. Thus ?experimenta-tion? [65] cannot be done without the activity of the participant [69].Determinants of success and study goals are difficult to define as theyvary depending on the artwork. They may or may not involve participantperformance, pleasure, understanding, and so on. For now the generalizationthat artists have the ultimate goal of creating experiences that ?engage? theparticipant will be made.Muller and Edmonds summarize the approaches to these studies as oc-curring in two areas: in museums and galleries and in HCI. In the former, thefocus is mainly on educational and interpretive uses of interactive technolo-gies [69]. They do not necessarily consider the significance of the interactionin the context of an artwork, nor do they inform the development of anartist?s techniques. In the latter, there are a number of studies that drawtogether HCI and the interactive arts. HCI methodology is appropriated tostudy and evaluate participant experience of interactive artworks and theresults are presented to both the HCI and the arts communities.In both cases, studies are often based on informal methods and observa-7Chapter 1. Introductiontion of interaction in the artwork?s natural context. The characteristics ofthese studies will be further detailed in Section 2.4. In contrast to commoninformal methods are a number of cases where more formal methodologieshave been used. Of note is research conducted in the Interface Ecology Labat Texas A&M University [4] and in the Creativity and Cognition Studios atthe University of Technology in Sydney, Australia [1]. Three specific studieswill be presented as case studies in Chapter 3 ([29], [54], [59]). These stud-ies have shown that laboratory-based studies and in-depth data collectionmethods are valuable for identifying and documenting bad artwork design,participant behaviours and participant techniques. This research builds ontheir formal methodology by considering methods for studying variations inparticipant understanding of interaction in a controlled setting.1.1.4 Interactive Arts and HCIThere has been a shift in HCI ?from a concern with functional efficiency toa broader interest in the overall user experience; this has brought greaterprominence to affect, engagement and pleasure.? [33]. This shift makessense given the injection of computing practices into our daily life fabric.No longer limited to work-related applications, computing technologies havepenetrated entertainment, social, home, urban, transportation, and otherdaily life spheres. Research in HCI has broadened [87] to consider elementsof user experience such as aesthetics (see for example [9], [45], [52] [55], [79]),affect (see for example [22], [35], [73], [80]), and play ([21], [66]).Consequently, HCI looks to the interactive arts, among other fields, foruser experience knowledge and creative inspiration. A recent CHI workshopon methodology and evaluation shared between HCI and the new media artsdiscusses collaborations as occurring in HCI explorations of ?artful interac-tion, tools to support creative engagement, and constructed user experiencesmeant to elicit creative responses.? [8] In the interactive arts, new interac-tion concepts can be attempted and understanding of relationships betweenhumans and computers beyond those we are familiar with today can be de-veloped [84]. The artist can explore areas not on the standard HCI research8Chapter 1. Introductionagenda and confront challenges for the future of computing practices.Also, while the interactive arts can draw from HCI study methods, HCIpractitioners can draw from artists? study results to develop understandingof how participant or user experience may be investigated. User experiencedesign goals are highly subjective [82] and can involve multiple interpre-tations [89] thus it can be difficult to evaluate to a single, ?good? design.Interactive artists deal directly with multiple interpretations of participants[84] and the manner in which they absorb and use participant interpretationscan be quite informative to the HCI practitioner.1.1.5 Differences Between HCI and the Interactive ArtsArtists and HCI practitioners can use similar methods but there will bedifferences such as study goals, results expected, experiences being studied,and view of errors (see Table 1.1). If HCI practitioners are to look towardsstudies conducted in the interactive arts for understanding and inspiration,an understanding of artistic goals and the significance of methods chosen isneeded to interpret the work.User studies and evaluation in traditional HCI have been generally con-ducted with goals of increasing a user?s comfort and efficiency with a com-puting technology. On the other hand, an artist is more interested in in-creasing a participant?s engagement with an art piece and this may or maynot depend on comfort and efficiency. As discussed above, there has beenincreasing importance placed on engagement in HCI research, but there arestill differences in study purposes in terms of how the outcomes are used andvalued. While HCI evaluation studies are part of an iterative design pro-cess strictly to inform design and redesign, H??k et al. note ?when artistsdo user studies, they are likely to see the user study itself as part of thecommunication through the artwork and another opportunity to shape the?message? of the artwork? [54].These differences in goals translate to differences in study results. InHCI, objective and quantifiable data is highly valued as reliable whereasartists are usually more interested in the subjective components of interac-9Chapter 1. Introductiontion and the insights that can be gleamed. While an HCI practitioner mayalso be interested in subjective elements, qualitative data tends to be av-eraged, generalizing a study?s results to make them repeatable and broadlyapplicable. Artists are less likely to generalize because they are interested inrich narratives of interaction and peculiarities that arise in single instancesof behaviour [33] [54].Table 1.1: Differences between characteristics of user stud-ies conducted by HCI practitioners and those of participantexperience studies conducted by artists.User Studies in HCI Participant ExperienceStudies in the InteractiveArtsGoals Increasing usability; informiterative design.Increasing participant engage-ment; further communicationof the artist?s message to par-ticipants.Results Objective, quantifiable datahighly valued; data averaged.Subjective, qualitative datahighly valued; interested ineach unique case of participantinteraction.Experience Task models; predefinedobjectives and paths to reachthem.Experience models; Experi-ence objectives not necessarilydefined.Errors Errors and negativeexperiences are undesirable;user should have a clearmental model of system.Errors and frustration can beintentional; artists may usetechniques of illusion or ambi-guity.10Chapter 1. IntroductionUser experiences in HCI are usually based on tasks and have predefinedobjectives and paths to reach them [16]. For most artworks, there may onlybe vaguely predefined experience objectives and ways to get there, if anyat all [33]. An artwork could perhaps be considered the proposition of atopic or question and the study is an artist curiously waiting for a varietyof responses and opinions.Finally, in traditional HCI, user errors and negative experiences are tobe avoided and a design goal is for the user to have a clear mental modelof the system they are interacting with. On the other hand, it may beacceptable for a participant of an interactive artwork to encounter errorsand frustration in their experience and artists quite often rely on techniquesof illusion or ambiguity [46] and not clear mental models.1.2 Problem StatementThere are great benefits in collaboration for both sides, but for collaborationto be successful, some of the above conflicts need to be addressed. This workaims to do this by providing a basis for each side to better understand thegoals and methods of the other. More specifically, this thesis addresses threeproblems:1. We do not fully understand the aesthetics of interactive art-works and accordingly, the purposes for conducting partic-ipant experience studies. HCI practitioners may have difficultiesaccepting the qualitative and interpretive nature of an artist?s studymethods. This is likely due to limited understanding of the artisticmotivations behind conducting studies. Yet for collaborations betweenHCI and the arts to occur, a common language and understanding ofartistic intentions and visions needs to be established [26].2. We do not fully understanding how laboratory-based userstudy methodology can be beneficial for the development ofinteractive art experiences. Many artists may question the ne-cessity for conducting participant experience studies at all. Then, if11Chapter 1. Introductionthey do include this in their practice, they may strongly believe thatlaboratory-based studies take a person away from the natural con-text of interaction and as a result, they only use informal methods forstudying participant experience. While there are cases of more formalmethodology used ([54], [29], [59], [97]), there is still room for morecommentary on the benefits and implications of using these methods.A strong role for formal studies conducted in conjunction with informalstudies in the artist?s design process needs to be established.3. We have no investigation of methods to compare experiencesof participants with varying levels of understanding of art-work elements in a controlled laboratory setting. Interactiveartworks require some exploration on part of the participant and theartifacts created by the artist should support this exploration. If aparticipant is unable to develop understanding of functional and ex-pressive elements of an artwork, their experience may be limited. Forexample, on approaching an artwork that uses body gestures as input,participants may not immediately understand how their movementsaffect the artwork. Or they may see the effects of their movementsbut they may be unable to attribute expressive meaning to their ac-tions and the artwork?s reactions. Thus, key concerns of the artist are:does the artwork properly support development of participant under-standing? How much is the experience actually based on ?figuringthe artwork out?? And how does specific understanding of differentartwork functional and expressive elements affect experience? Thesequestions suggest the need for methodology that would allow artiststo selectively influence participant understanding of specific artworkelements in a controlled setting.12Chapter 1. Introduction1.3 GoalsTo address the above problems, the work in this thesis is approached withthe following goals:1. To make the argument that participant studies are not only a neces-sary activity in the artistic process for refining technique. They arealso part of an artwork?s content and contribute to the aesthetics ofinteractive art.2. To review three cases of participant experience studies conducted usingmore formal methodology appropriated from HCI, highlighting thebenefits accrued.3. To conduct and comment on a study of participant experience withthe interactive artwork Echology in a controlled laboratory settingusing pre-experience information or orientation media to selectivelyinfluence participant understanding of different artwork elements.1.4 OverviewThe research area and problems taken on in this thesis are summarized inFigure 1.2 and an overview of work conducted is given in Figure 1.3. Thenext chapter begins by reviewing background information in interactive arttheory and characteristics of participant experience studies. These studiesas part of an iterative artistic process are compared with user studies asan activity in the HCI process. Chapter 3 reviews three specific cases ofparticipant experience studies conducted using more formal methodologyappropriated from HCI. These studies provide a foundation for this thesisto build upon with an investigation of methodology through a study ofthe artwork Echology. Chapter 4 introduces Echology, discussing artisticmotivations, the Echology system and the intended participant experience.Chapter 5 summarizes an initial informal study of Echology then detailsthe design and execution of a laboratory-based study using orientationmedia. Study results are discussed and a review of the study experience is13Chapter 1. Introductionprovided. Chapter 6 concludes with a review of the problems taken on in thiswork and the contributions made. Areas for future work in the developmentof methods for studying participant experience of interactive artworks aresuggested. The appendices include material used in the Echology study andresulting study data.Human Computer InteractionInteractive ArtsParticipant Experience StudiesInformation and InspirationResearch Area Problems IdentifiedRoot Problem: Lack of understanding of the role for participant experience studies in the artistic process.Problem: It is difficult for HCI researchers to interpret and apply results of participant experience studies.Problem: Artist unaware of benefits of laboratory-based studies using more formal scientific methodology. Figure 1.2: Research area and problems identified.14Chapter 1. IntroductionDrawing from interactive art literature, make a case for the role of participant experience studies in interactive art aesthetics and the artist's process. Summarize common characteristics of participant experience studies. Compare artistic process and studies to HCI design process and user studies.(Chapter 2)Present three detailed cases of more formal laboratory-based studies that use HCI methodology.(Chapter 3)Propose, design and conduct laboratory-based study of participant experience with Echology using controlled experiment setting and variability through the use of orientation media.(Chapter 5)Design and develop interactive sound and video artwork "Echology".(Chapter 4) Study participant interaction through informal observations of Echology at NIME '05.(Chapter 5)compareThesis WorkFigure 1.3: Thesis work overview and chapter breakdown.15Chapter 2Interactive Art and the HCIProcessThis chapter begins by defining ?interactive? and interactive art in orderto distinguish it from other art forms. The definitions reveal the interac-tive artwork as no longer only a static ?art object?, but as an ?art system?consisting of the artist, art objects, and participants where the artwork de-velops through dynamic relationships between these art system entities. Itsaesthetics are also no longer only determined by the art object but dependon the artist, participant, and behaviours within the system. To understandthese new aesthetics, the roles of art system members are summarized inSection 2.2. Purposes for conducting participant experience studies are de-rived from these roles. These studies are revealed as an activity necessaryin an artist?s process for the development of their technique, as an enablerof learning, evolving art objects and as a contributor to the aesthetics andcontent of interactive artworks. Common characteristics of these studies arereviewed in Section 2.3. Following is a brief review of the HCI process in Sec-tion 2.4.1 and in Section 2.4.2 parallels between this process and the artisticprocess are considered in order to highlight similarities and differences inuser studies and participant experience studies.16Chapter 2. Interactive Art and the HCI Process2.1 Defining Interactive ArtWhat makes an artwork interactive? ?Interactive? has become a term usedvery loosely [50]. It is a ?buzzword? [56], [64] used to speak of interactivetelevision, interactive web content, interactive information booths and so on,and as such, has lost some strength in its meaning. Furthermore, referringto art specifically, there is the view that all works are interactive in thesense that an artwork presents the creator?s message to an audience andthe audience responds with, at the very least, some internal interpretation[56] [84]. Thus, to distinguish the interactive arts from other art forms andto begin discussion of its unique aesthetics, it is necessary to consider themeaning of ?interactive?, a background history in participatory art, and amodel for interactive artworks that is used for the remainder of this thesis.2.1.1 Defining ?Interactive?The dictionary provides us with the definition:interactive (adjective)1. influencing each other2. (of a computer or other electronic device) allowing a two-way flow of information between it and a user, respondingto the user?s input(Compact Oxford English Dictionary, 2005)Key elements of this definition are ?each other?: implying that interac-tion involves multiple parties, ?influencing?: implying that a party to theinteraction causes some change to the state or actions of the other party,and ?two-way flow of information?.Further perspective of interaction exists in the social sciences in referenceto interaction between people. The term has also come to be used when refer-ring to the relationship between humans and computers or machines where?interaction between people and machines implies mutual intelligibility, orshared understanding?, ([62], Suchman p. 6). This perspective implies that17Chapter 2. Interactive Art and the HCI Processparties of an interaction use some model of each other to attempt to es-tablish a common language: words, actions, protocols, with which they caninteract.Summarizing the above, we have the following elements of interactivity:? It involves two or more parties.? It results in some form of change in the states or actions of all parties.? There is a two-way flow of information.? Parties maintain some mental model of each other.? A common language is established for the exchange of information.The above elements can be realized in relationships between people andinteractive things in a number of ways, showing that there is a spectrumto the interpretations of ?interactive?. In Section 2.1.3 we will see thatthis spectrum has been acknowledged by art theorists in their definitions ofinteractive art.2.1.2 Background in Participatory ArtAs mentioned in Chapter 1, the early interactive arts were pushed forwardby an increased interest in participatory art. In participatory art the spec-tator becomes an ?active spectator? or participant: an audience that doesnot just passively view artwork but plays some kind of active role in shap-ing it. Motivation for enabling the participant stems from art activities inthe first half of the 20th century. In the Duchamp, Dada, and Surrealismgroups, artists pulled away from the traditional arts, making attacks on artinstitutions and the idea of artwork being a precious, fine object [30]. Theychallenge the traditional roles of artists and artwork and sought to ?reinte-grate art with life-praxis? [24] ? to take it out of the gallery and into theaudience?s everyday life. One manifestation of this is giving the audiencea role as creator. For example, Peter B?rger comments on Tristan Tzara?sinstructions for making a Dadaist poem ? ?This represents not only a polem-ical attack on the individual creativity of the artist; the recipe is to be taken18Chapter 2. Interactive Art and the HCI Processquite literally as suggesting a possible activity on the part of the recipient.?[24]Distinguishing between artwork for passive and active spectatorship, RoyAscott creates two categories among which all artwork can be classified: ?de-terministic? and ?behavioural? [8], [11]. In ?deterministic? art, the artworkproduced by the artist is a final product, a ?container of information? fullydetermined by the artist [28]. In ?behavioural? art, the artwork is a ?trig-ger? [12] designed to stimulate unpredictable behaviour. Ascott writes, ?thenecessary conditions of behaviourist art are that the spectator is involvedand that the artwork in some way behaves.? In this way, the final productof an artwork can be unpredictable, shaped by indeterminable individualor social responses fed back into the work. An example of an early ?be-havioural? artwork is Ascott?s piece ?Change Painting? (1959). It consistsof paintings on sliding panels that audience can move to create their owncomposition.2.1.3 The Interactive Art SystemTo better reflect the product of behavioural art as including artifacts cre-ated as well as processes and events that occur after conception, Cornockand Edmonds use the term ?art system? in place of ?artwork? [28]. Asshown in Figure 2.1, the art system incorporates all involved in an artwork:the active spectators or participants, the artist, and art objects. It is theproduct of these members and the behaviour that arises in the relationshipsbetween them that amount to the artwork. The artist no longer communi-cates simply through a static art object but through the design of a systemwhere interaction between system members is key [101].Cornock and Edmonds envision interactive art as a flavour of behaviouralart especially enabled by digital media. Breaking down Ascott?s division,they categorize all art based on a level of interactivity capable in the artobject (see Figure 2.2). The result is four scenarios considering the relation-ships between the artist, the art object, the audience or participant, and theart object?s environment:19Chapter 2. Interactive Art and the HCI ProcessArtworkThe Art SystemArtworkTraditional ArtworkArtistAudienceArtistParticipantArt Objectcreatingviewingcreatingviewing and affectingFigure 2.1: Traditionally the artist creates an artwork which is viewed by anaudience. In the art system the artist creates an art object which is viewedand affected by audience who have become participants. The artwork is aproduct of artist, art object, participants, and behaviours that arise in theirrelationships.Static: The art object does not change. The audience observes, butdoes not affect the art object.Dynamic-Passive: The art object can change over time or be modi-fied by environmental factors such as sound or light. Given that the artistspecifies the art object?s rules for change, the ways in which it changes arepredictable. The audience still passively observes.Dynamic-Interactive: This is the same as the dynamic-passive sce-nario with the added capability for the audience to influence changes in theart object. The audience no longer passively observes and it may be saidthat they are participants. In this scenario the artist still specifies the waysin which the art object may change, but it is unpredictable to the extentthat participant behaviour is unpredictable.Dynamic-Interactive (Varying): This is the same as the dynamic-interactive scenario with an added set of rules or what can be called ?matrix?programmed by the artist that influences the behaviour of the art object.20Chapter 2. Interactive Art and the HCI ProcessDynamicBehaviouralDynamic-PassiveDynamic-Interactive Dynamic-Interactive (Varying) StaticDynamicReciprocalParticipatoryInteractiveDeterministicless interactive more interactiveCornock and Edmonds (1973)Ascott (1967)Cornock (1977)Figure 2.2: Classification of art systems according to Ascott [11], Cornockand Edmonds [28], and Cornock [27]. Adapted from Graham?s summary oftaxonomies for types of interactivity [50]. Categories are listed from left toright according to less to more interactivity.Behaviour in the art system is fed back to the matrix and the art objectchanges according to rules set in the matrix. The matrix also responds byadapting itself, causing further change in the behaviour of the art object,and finally, changing behaviour within the art system. The art system learnsand evolves through this feedback loop.Cornock later updates this categorization to better detail dynamic artsystems (see bottom of Figure 2.2). At a higher level he maintains thedivision between static and dynamic art systems. He then further subdividesorganizationally dynamic art systems ([27] as summarized by [14]):21Chapter 2. Interactive Art and the HCI ProcessDynamic Art Systems: The artifact (or art object) has organizationaldependence on some environment variable(s).Reciprocal Art Systems: The audience is treated as an environmentvariable (voluntarily or involuntarily) that causes the system to pass througha set of states.Participatory Art Systems: The artist defines a matrix of time, place,and set of ideas or constraints. The artwork is composed of the events,including interpersonal reactions of a group of participants to a situation,that take place within the matrix.Interactive Art Systems: There is a mutual exchange between manand machine that approaches the exchange that occurs during conversationbetween people. To achieve this the art system ?should exhibit the proper-ties of a learning system?.These classifications are useful for exemplifying the spectrum of inter-activity as discussed in Section 2.1.1. Cornock?s classification also suggeststhat an ideal for ?truly? interactive art exists. In this extreme, a partici-pant is able to engage in ?real conversation? with an art system where realconversation is ?an evolving, unpredictable exchange of ideas? [50].In the remainder of this thesis the ?art system? model of interactiveartwork is used. The stance that all members of the art system contributeto an artwork?s aesthetics is taken. The categorizations of dynamic artsystems are not discussed further save for the idea of there being ?truly?interactive art systems.2.2 Contributors to the Aesthetics of InteractiveArtworksIn this section the aesthetics of interactive artworks are explored in a discus-sion of the roles of art system members: art object, participant, and artist.The roles are summarized in Figure 2.3 and they set a basis from which thepurposes for participant experience studies can be derived.22Chapter 2. Interactive Art and the HCI ProcessInteractionArt ObjectRoles:Triggering and Guiding Participant InteractionSupporting Participant Interpretation through RepresentationsFacilitating Communication ParticipantRoles:Imaginary ContributionCreative ContributionInvolvement in a Social SpaceArtistRoles:ImplementStudyDesignConceptFigure 2.3: Roles of artist, art object, and participant.2.2.1 Roles of the Art ObjectIn his thesis on ?Participatory Art and Computers? [14], Bell presents the artobject having a physical interface and a program. The physical interfaceconsists of input and output devices related to one or more of the humansenses: sound, vision, touch, smell, and taste. It may also have input devicesfor sensing its environment or elements not associated with the participant.The program processes input signals from the physical interface and controlsinformation sent to the output devices. Both physical interface and program23Chapter 2. Interactive Art and the HCI Processare designed to create some interaction experience envisioned by the artist.Bell provides a classification of design parameters for physical interfaceand program that are considered by the artist to create art objects thatfacilitate envisioned interaction experiences. Three roles for the art ob-ject are derived from a review of these parameters: triggering and guidingparticipant interaction, supporting participant interpretation through rep-resentations, and facilitating communication.Triggering and Guiding Participant InteractionEncountering an interactive artwork for the first time, a participant may beunaware of the possibility to interact. The art object should have ?attrac-tors?, ?things that encourage the audience to take note of the system in thefirst place? [40]. For example, it may be able to detect the presence of anew participant and initiate interaction.Once a participant?s attention has been captured, they may need guid-ance in interaction. Cues can be given through interface elements withstrong affordances [72] as well as with the use of existing interface andprogram conventions. Bell gives the example of the cursor controlled bya mouse. Conventionally, its movement on the screen will follow that ofthe mouse in physical space. Most people will quickly become familiar in-teracting with an art object using this same convention. The art objectmay continue guidance by operating in a way that allows the participant toclearly see their effect on the art object [38].On the other hand, the artist may wish to use novel or unconventional in-terface and program elements, requiring the participant to explore the spaceand learn the art object. The art object may be ambiguous in directives andresponses so that participants ?interpret the situation for themselves?, al-lowing them to ?establish deeper and more personal relations? with themeaning presented [46].Both of the above design choices have their consequences, that of theformer being the participant?s potential boredom and the latter, possibleconfusion and frustration. For participant enjoyment, it can be desirable for24Chapter 2. Interactive Art and the HCI Processthe art object to be balanced between the two extremes [31], [43].Supporting Participant Interpretation Through RepresentationsParticipants develop unique interpretations of the art object shaped by theirindividual backgrounds and expectations. These interpretations are alsosupported by possible representations taken on by the art object: narration,virtual entities, virtual spaces, and mirrors.Art objects with narrative representations present stories in a linear ornonlinear fashion. Grahame Weinbren?s ?The Erl King? (1982 ? 1985) [98]allows a participant to explore two nineteenth century texts on a touch-screen display, discovering connections between the two and controlling thenarrative flow. Representations of virtual entities may present charactersor life forms to the participant. Laurent Mignonneau and Christa Som-merer?s ?A-Volve? (1994 - 1997) [91] explores interaction with artificial lifeand intelligence. Participants can interact with virtual creatures displayedin a pool of water. Art objects with virtual space representations allow aparticipant to explore or enter new worlds, realities, or new spaces createdthrough telecommunications. Finally, art objects that act as mirrors al-low participants to see representations of themselves. Iamascope, discussedfurther in a case presented in Chapter 3, is an artwork that uses a videocamera to capture a participant moving in front of a projected image (seeFigure 3.1). The participant?s image is processed to create kaleidoscope im-agery projected in the image space [44]. Interaction is much about playingwith an abstraction of one?s own image.The above representations may be given to the participant explicitlyor implicitly and with varying amounts of realism, thus requiring varyingamounts of imagination on the part of the participant. The desired resultis that rather than seeing the art object as the physical system it is, theparticipant becomes immersed in the world of the representation.25Chapter 2. Interactive Art and the HCI ProcessFacilitating CommunicationA final role of the art object is to facilitate occasions for communicationamong members of the art system. Bell [14] discusses occasions for commu-nication as occurring between:1. Artist and participant through the art object.2. Artist and art object in a reflective activity on part of the artist.3. Participant and art object ? a reflective activity on part of the partic-ipant and ?allowed? for by the artist.4. Several participants through the art object.5. Participant and one or many autonomous art objects.6. Participant and an art object acting as participant.In cases 1 to 4, intelligent communication is possible because all commu-nicators have some internal world models as well as understanding of theirpartners? world models. This understanding is used to interpret informationreceived from a communication partner and to choose appropriate responses.In cases 5 and 6, the art object must autonomously communicate witha participant and its ?ability to respond appropriately depends on the qual-ity of the perceptual system and the computer?s ability to interpret what itperceives? [60]. Intelligent communication depends on the art object main-taining some model of participants and their behaviours and the possibilityfor this model to evolve through interaction or learn. This resonates withthe ideal of a ?truly? interactive art system spoken of at the end of Section2.1.3 ? one that ?should exhibit the properties of a learning system? [27].The art object can also create the impression of intelligence by having ?in-terpretive affordances? that ?support the interpretations an audience makesabout the operations of an AI system.? [67]26Chapter 2. Interactive Art and the HCI Process2.2.2 Roles of the ParticipantAs the audience becomes participant they are invited to play a more activepart in the artwork. Besides simply going through the physical motionsof interaction the participant has roles of: imaginary contribution, creativecontribution, and involvement in a social space.Imaginary ContributionBell notes that there will be a difference in how a participant perceivesa program to work and how it actually works [14]. He refers to what isperceived by the participant as the ?virtual machine?: an interpretation ofthe actual machine or art object. For most artworks, it is the intention of theartist to have participants perceive this virtual rather than actual machineor art object. While this is supported by the art object?s representations, italso ?relies on imaginary contribution from participants.? [14]Creative ContributionThis role acknowledges the extent that an artwork is dependent on partici-pant behaviour and the motivation for artists to include inducing participantcreativity in their artistic goals and as part of the aesthetic experience [13].The quality of interactivity can be seen as depending ?upon the extent towhich the work of art can encourage both critical reflection and creative en-gagement? [30]. At an extreme, this creativity involves a participant taking?ownership? of the artwork [34].Involvement in a Social SpaceParticipants can be given the role of actors in a social space. The art objectis often constructed as a mediator of communication between people. Thiscan be done indirectly by creating a social situation or ?environment of com-munication possibilities? [49], or directly, by being a channel through whichparticipants may share information with each other, or become involved incollaboration or competition.27Chapter 2. Interactive Art and the HCI Process2.2.3 Roles of the ArtistThe traditional artist role is broken down in the interactive arts. Tradition-ally, the artist is the star and there is high value in art objects made fromthe artist?s own hands [25]. In the interactive arts:?the artist no longer decides everything and projects it as a wholein some definitive and final composition (. . . ) he now initiatesa dialogue, or set of events, which, when taken up by the au-dience, whether in a group or individually, will be shaped intototally unpredictable and indeterminate forms and experiences.?(Ascott, 1968) [12]The artist initiates this dialogue by relinquishing authorial control andcreating relationships [84] through the design of an art object?s interface andprogram. Returning to the art object roles, it is the artist?s role to design artobjects that support participants in an interaction experience, shape partic-ipant interpretation through representations and facilitate communicationbetween members of the art system.At the same time, the artist must be aware of how participant roles arebeing supported. The ?techniques? of the participant are ?non-existent?[78] thus it is the artist?s role to create an interaction space where a partic-ipant may develop the needed techniques. This requires an understandingof participants in the context of interaction, as well as experimentation con-ducted with both art object and participants to test that desired interactionaesthetics are supported.Thus it becomes the artist?s role to include studies of interaction betweenart object and participants in the cycle of design, implementation and studyas depicted in Figure 2.3. Results of these studies are fed back into artobject redesign, future concepts [84] and the refinement of technique.28Chapter 2. Interactive Art and the HCI Process2.3 Participant Experience Studies of InteractiveArtworks?But accepting responsibility is at the heart of interactivity. Re-sponsibility means, literally, the ability to respond. An interac-tion is only possible when two or more people or systems agreeto be sensitive and responsive to each other. The process of de-signing an interaction should also itself be interactive.? (Rokeby,1995) [84]2.3.1 Purposes of Participant Experience StudiesThree purposes for conducting participant experience studies are derivedfrom the roles of art system members:1. Heightened theory development: to develop understanding of interac-tion and to evaluate art object design.2. Design iteration: to feed back what was learned into new versions of anart object, thus creating art objects that indirectly learn and evolve.3. Build content: to create a record of interaction within the artwork,documentation that becomes part of the artwork?s content.In the first purpose, the artist studies participant experience in orderto develop an understanding of participant behaviour and perception in thecontext of interacting with the art object. The art object is evaluated in itsability to support the participant through the desired interaction experience.New understanding is fed back to develop the techniques of the artist.The second purpose is for new understanding developed in studies tobe fed back into redesign of the art object. The art object then indirectlylearns about the participants and its responses are improved. This feedbackrole can perhaps be interpreted as one where studies produce the evolvingand learning art objects that are necessary to support ?truly interactive?art systems as defined by Cornock [27].29Chapter 2. Interactive Art and the HCI ProcessThe third purpose presents participant studies not only as a processneeded to develop and refine artwork and technique but also as part of theartwork?s content itself. To repeat what was stated in Section 2.1.3, it isthe product of art system members and relationships between them thatamount to an artwork [101]. Thus, other than the art object, interactionbehaviour and individual participant experiences can also be viewed as anartist?s output. Participant experience studies document this product andthe manner in which they are conducted also contribute to aesthetics. Thisadds further meaning to the statement from H??k et al. in Section 1.1.5:that studies are another opportunity for the artist to shape the message ofthe artwork [54].2.3.2 Characteristics of Participant Experience StudiesIn this section the common goals, methods, settings and subjects, and formof results of participant experience studies are discussed with the supportof published cases drawn from the HCI and media arts literature ([10], [17],[18], [23], [47], [70], [93], [62], [97]). GoalsIn Section 1.1.3 it is proposed that generally, artists are interested in par-ticipant engagement. Part of an artist?s role is to observe and evaluate howthe art object fulfills its role in supporting participants in their roles andthus, how it supports an engaging experience. Now that roles have beenreviewed, the question of engagement can be translated to more detailedquestions that may be posed about the relationship between the participantand art object, about the participant him or herself and about the socialbehaviours that emerge among multiple participants experiencing an art-work. Examples of questions that an artist may pose when conducting aparticipant experience study are:30Chapter 2. Interactive Art and the HCI ProcessConcerning the relationship between the participant and an art object:? How do participants approach the art object? How do they first ex-plore the interactions presented to them?? Can participants see the relationship between their actions and the artobject?s actions and reactions? Does this relationship satisfy them?? Do participants understand and become engaged with art object rep-resentations?? Is the art object designed for a range of interaction skill and in whatways are participants able to better their skills towards some level ofexpertise?Focusing on participants themselves:? Generally, do they enjoy their experience? How long do they stayengaged and how does engagement end?? In what ways does the artwork foster personal reflection?? Does a participant move beyond performing ?obvious? interactionsdesigned into the art object to a more creative active involvement?? Are participants able to transition into states of critical thinking aboutthe artwork concepts and aesthetics?Concerning social behaviours among multiple participants:? What social behaviours occur among people participating in the art-work either simultaneously or asynchronously as allowed for by an artobject that creates a communication channel?? Does the artwork promote communication among participants?? Does interaction in the artwork environment cause breakdowns in so-cial norms?31Chapter 2. Interactive Art and the HCI ProcessThe results of studies posing these questions may be applied to the eval-uation of an artwork?s technical or artistic success, to the identification ofparticipant behaviour applicable to theories of participant aesthetics andexperience, and to the redesign of future versions of the art object [10]. MethodsArtists commonly study participants through informal observation of theirinteractions with an art object [18]. The artist may choose to not speakwith the participants, only observe. They may offer assistance or expla-nation about the artwork. Supported by the social space created aroundmany interactive artworks, they also might engage in discussion about theartwork with participants, obtaining valuable feedback and interpretations[10]. What is strongly held as important is the view that artworks be studiedin context; when they are being experienced and in their ?natural? setting.This is when meaning of the interaction will develop for participants [69][92]. It is also common for the artists to make a point of not giving par-ticipants instructions [70] or not specifying that the piece is an interactiveartwork [17].Cases taking a more formal approach involve video recording of the art-work space, interviews requested of people after they experience the artwork[17], and questionnaires. A study by Aley et al. uses a ?Wizard of Oz? [58]simulation in an early study to inform the design of their artwork [10].Other exceptions to the informal methods are those of the cases thatwill be reviewed in Chapter 3 and studies of audience interaction in thegallery setting conducted by Beryl Graham in her PhD thesis [50]. Eth-nomethodological approaches have also been used for studying behaviouraround interactive exhibits [62] and for studying artwork in the home [47]and small community environments.32Chapter 2. Interactive Art and the HCI Process2.3.2.3 Settings and SubjectsAs stated in the previous section, it is often held important that artworksare studied in their ?natural? setting. ?Natural? often means an art gallery.Still, as mentioned in Section 2.1.2, the interactive arts have a backgroundin forms that seek to bring art into the audience?s everyday life. Thus,artworks have been studied in other public places such as museums, caf?s,train stations, in the streets [17], [18], and even in people?s homes [47].These diverse settings enable artists to reach a broad range of partici-pants outside of the art communities, often people having no previous ex-perience with interactive artworks [18]. Of particular interest is that broad-ening the audience allows artists to observe groups not so commonly repre-sented within art communities, for example, the elderly and children. Bothare significantly interesting subject groups; the former with experience in atime that computing technology was non-existent or new, and the latter notknowing a life without it.Finally, artworks are often studied at special events and exhibitions oflimited duration. Some studies hold special exhibitions or parties with thespecific purpose of studying an artwork [70], [97]. Installation and mainte-nance of interactive artworks can be difficult as often, only the artist hasknowledge of how to setup, operate, and fix the art objects, which can beprone to bugs and breakdowns. Thus, it can be more difficult to find placesfor permanent installation. Fortunately, temporary exhibitions can be goodfor receiving feedback on an artwork and then having a chance to redesignthe art object before future exhibitions. Form of ResultsThe manner in which results are presented is highly dependent on wherethey are being presented. For published results, a paper?s audience maybe more interested in learning about the art system as a whole, about thedesign and roles of the art objects, or about the playing out of participantroles. Results may then be isolated to a single paragraph near the end ofthe paper or they may be discussed in further degrees of detail.33Chapter 2. Interactive Art and the HCI ProcessResults are almost always of a qualitative nature. Some exceptions arethe measurement of participant demographics, average times spent inter-acting with the artwork, and quantities derived from questionnaires [97].Otherwise they often take the form of general observations in statementssuch as ?we saw many people who tried to. . . ?, ?people often said that. . .?,and so on [70]. The observations can be supported with examples of be-haviour among one or many participants, or with short statements madeby the participants: ?There was one group that. .. ?, ?A woman said thatshe felt. .. ?, etc [97], [17]. More in-depth discussion is often conveyed ina narrative form, telling of participant interaction in general and garnish-ing it with the instances that stood out, being exceptionally surprising orinteresting. Some reports construct interaction stories giving participantspseudonyms and establishing their character to personalize the experiencebeing told [47].Examples of topics that an artist may cover when discussing the resultsof a participant experience study are responsive to the questions posed inSection the relationship between the participant and an art object:? How participants approach the artwork space.? How participants explore the interaction space.? Relationships discovered between actions and art object reactions.? Stages of interaction and how it would normally begin and end.? Metaphors and concepts used to make sense of the art object.? Comments and criticisms.Focusing on the participant him or herself:? Relating the artwork to personal things like previous experiences.? Emotions experienced.? Strategies of interaction taken.34Chapter 2. Interactive Art and the HCI Process? Moments of creativity.? Skill levels obtained.? Satisfaction with personal performance.Concerning social behaviours among multiple participants:? Conversation occurring among strangers.? Social norms and barriers broken.? People engaging in play behaviour.? Interaction between people with some prior relationship.2.4 The HCI ProcessThis section briefly reviews the traditional HCI process of user-centered,or interactive design, focusing on the role of user studies in this process.Section 2.4.2 considers the parallels between the HCI and artistic processesin order to re-examine similarities and differences between user studies andparticipant experience studies.2.4.1 User-Centered and Interaction DesignUser-centered design is broadly a design process in which the user?s needsand context influence design [7]. Goals driving this process are known as?usability goals?. Preece et al. list these as: effectiveness, efficiency, safety,utility, learnability, and memorability [82]. Guidelines and rules for design-ing for these goals are suggested in several sources [71], [74], [90], all havingcommon points such as:? Make the system status visible to the user at all times.? Match between the system and the real world, speaking the users?language and using real world conventions.35Chapter 2. Interactive Art and the HCI Process? Follow existing standards and conventions set for the design space.? etc.. .As mentioned in Section 1.1.4, HCI has broadened from being only con-cerned with usability to having concern for the user experience of a systemor a product as a whole. This is reflected in the proposition of interactiondesign [102] ? which is founded in user-centered design but also considersuser ?experience? goals. Preece et al. list these goals as creating systemsthat are: satisfying, enjoyable, fun, entertaining, helpful, motivating, aes-thetically pleasing, supportive of creativity, rewarding, and emotionally ful-filling.The interaction design process is an iterative one with four general ac-tivities as defined by Preece et al:? Identifying needs and establishing requirements? Developing alternative designs? Building interactive versions of the designs? Evaluating designsThrough iterations of these activities, a system or artifact transitionsfrom low to medium to high-fidelity prototypes and eventually, to a finalproduct.Designers involve users in the process to both inform design requirementsand to evaluate designs implemented. Methods used in the early stages of aproject may be interviews and questionnaires on user backgrounds and workprocesses, focus groups, on-site observations, role playing, walkthroughs,and simulation [7]. When a product begins to reach its final form, evalu-ation of the product?s fulfillment of usability and user experience goals isconducted.36Chapter 2. Interactive Art and the HCI ProcessImplementUser StudiesDesignUser Needs and RequirementsExisting usability and user study heuristicsHCI ProcessImplementParticipant StudiesDesignConceptPart of final artworkArtist ProcessArtistic goalsFigure 2.4: Iterative processes of design, implementation, and studies com-monly used by both HCI practitioners and by artists.37Chapter 2. Interactive Art and the HCI Process2.4.2 Parallels in HCI and Artistic ProcessesThe artistic process depicted in Figure 2.3 can be compared to the HCI pro-cess as discussed in the previous section. Figure 2.4 shows how both artistand HCI practitioner iterate through activities of design, implementation,and evaluation and involve studies with users or participants in these cycles.However, while recommendations for different methods suitable to differentstages of the cycle have developed quite extensively in HCI, recommenda-tions of the same caliber do not exist for methods of studying participantexperience. This work takes the opinion that this is because participantexperience is often studied as an afterthought and, as discussed in Section2.3.2, methods used are often those of informal observation and interview inan artwork?s exhibition setting. Still, as will be illustrated in the next chap-ter by three cases of more formalized approaches to participant experiencestudies, there are benefits to be gained by exploring various methods thatcould be used at different stages of an artwork?s lifetime.If HCI methods are appropriated to study participant experience, it isbeneficial to acknowledge several differences between the HCI process andthe artist process:? The HCI process begins with identifying needs and requirements; theartistic process begins with an experience concept.? User studies evaluate products against existing goals and heuristicscommon across many products; an artist evaluates according to uniqueartistic goals that may or may not be informed by previously estab-lished heuristics.? Participant experience studies do not only inform design, they are alsopart of the final artwork.38Chapter 2. Interactive Art and the HCI Process2.5 SummaryThis chapter has distinguished the interactive arts from other forms by pro-viding background to the meaning of ?interactive? and by presenting theinteractive artwork as an art system consisting of artist, art objects, andparticipants. Roles for each member of the art system are defined and re-veal participant experience studies to be both an activity in an iterativeprocess taken on by the artist and a contributor to the artwork itself. Com-mon characteristics of participant experience studies are reviewed, revealinga tendency to use open and informal methods in the artwork?s natural con-text. Finally, the HCI process is briefly reviewed and compared with theartist?s process. Differences between the processes are highlighted to betterunderstand how HCI methods may be appropriated by artists to fulfill theirartistic needs and goals.39Chapter 3Formal ParticipantExperience StudiesThis chapter presents three cases of participant experience studies conductedusing more formal methods. These cases are chosen for their attempts toestablish more rigorous methodology for studying interaction, prioritize therole of studying participant interaction in the development of their artworks,and communicate their experiences to both the art and the HCI worlds.They set a basis to which this thesis makes its contributions. For each casestudy goals, artwork studied, methodology used and results presented aredetailed. A summary is provided in Table 3.1. The chapter concludes witha discussion of the benefits of formal study methods as exemplified by thecase studies and a proposal of further work taken on in this thesis.40Chapter 3. Formal Participant Experience Studies3.1 Case One: The Influencing Machine3.1.1 GoalsIn their study of the ?Influencing Machine? [54], H??k, Sengers, and An-dersson have the research aim to develop methodology better suited to theinterests of artists. They take the stance that HCI evaluation methods canbe used to better understand and improve interactive systems. But theyalso note conflicts that exist in the differences between artist and HCI prac-titioner perspectives and that HCI methodology may not produce resultsthat are of interest to artists. They investigate a laboratory-based method-ology based on HCI evaluation methods while keeping in mind the needs ofartists. The methodology is used to determine what thoughts their artwork,the Influencing Machine, is capable of provoking in participants. They alsowant to distinguish, in the case that a user becomes frustrated during in-teraction, whether frustration is rooted in not being able to directly controlthe Influencing Machine or in fact, poor design choices.3.1.2 ArtworkThe Influencing Machine is an artwork that explores an element of affectivecomputing: how people relate to a machine that they can influence but notcontrol. Its purpose is to provoke thoughts on a machine?s capability tobe emotional. The work consists of a wooden mailbox around which post-cards are scattered, video projected onto a wall, and a sound system forplaying musical sounds. In the projected video, childlike drawings contin-uously appear, build up together and fade out. The postcards are printedwith other artworks or a color field, and a barcode. The postcards are aninput mechanism and when one is dropped into the mailbox, the InfluencingMachine detects which card has been dropped and, based on an emotionalmodel, plays sounds and alters how images are drawn in the video projec-tion. Users can drop postcards in varying combinations and intervals andthe Influencing Machine will emotionally react. The interpretation of thereaction is open-ended and highly based on the background of the user.41Chapter 3. Formal Participant Experience Studies3.1.3 MethodThe experiment setting is a room of an old house set up to not look like alaboratory or office space. The Influencing Machine is placed on a clothedtable under which the computer is hidden. Participants are brought ingroups of varying size and a co-discovery method [39] is used to capturea participant?s understanding while experiencing the artwork, but withoutforcing them to speak out loud. This allows them to observe group dynamicsaround the artwork and have the experience approach a setting more similarto a non-laboratory setting.Before viewing the artwork short interviews are conducted. Knowingfrom previous experience that it is inadequate to classify users based onlyon basic demographic information (age, gender, education, etc.) they usethese interviews to collect information on perspectives of computer cultureand art installations. Then, participants are told that the machine theywill view is ?about emotions and that they will be posting postcards into amailbox?, that they can do as they please and leave when bored, and thatthey will be video-taped.The study is done in two parts according to previously established guide-lines for evaluating affective interactive systems; the first part should assurethat emotions expressed between the system and the user are understoodby both, the second part checks that this understanding leads to the desiredinteraction effect [53]. In a first study, the amount a participant can influ-ence the artwork with postcard input is constant. In a second study theyrevise the artwork design to speed up the drawings and include a display ofthe machine?s emotional levels as well as musical feedback. Some groups aretested with the display and some without. Groups also use either only theart postcards or only the color field postcards, and the amount of influencecapable is varied.After viewing the artwork, an open-ended interview is conducted wherethe interviewer is careful not to use language that associates metaphors tothe artwork.42Chapter 3. Formal Participant Experience Studies3.1.4 ResultsIn the first study users were first curious and then became frustrated thatthey could not figure out the relationship between the postcards and thedrawings projected. Other observations were that participants found thedrawings too simple or slow, they liked putting things inside the mailbox,there was some success in having participants reflect on the meaning of theartwork, and there was frustration over the lack of control. After revisingthe artwork and conducting the second study, they conclude that the desiredresults were achieved and comment on the influence of a number of the artobject components. They also learn about differences in behaviour amongtheory formers and those that did not form theories about what was goingon.Results are presented as follows. Results from the first study are pre-sented as general observations supported by quotes from the participants.Results from the second study are presented in the following parts: detailedexperience accounts of 3 out of 9 groups studied, followed by a summary ofthe experiences of all groups, summaries of responses to 5 interview ques-tions with key words and quotes highlighted, a summary of metaphors usedby the participants, and finally a discussion of their overall impressions.A highlight in the presentation of their results is the narrative fashion inwhich they detail the experience accounts of the 3 groups. This narrationtells of interaction with the machine, interaction among participants, the-ories formed, strategies used, and participant reflections. The presentationmanner is successful in revealing the experiences in a holistic manner whilestill maintaining intricacies and unique characteristics.3.2 Case Two: Iamascope3.2.1 GoalsThe second example [29] is a user study of the camera and video systemIamascope [44], a piece by artist Sidney Fels. This case is not a study con-ducted by the artist himself but by another group of artists and researchers43Chapter 3. Formal Participant Experience Studieswell known in the field of interactive arts. They are not interested in eval-uating Iamascope directly, but instead use it as a case with which they can?find useful methodology for recording and analyzing the situated experi-ence of interactive art?. They aim to find methodology that fits in the gapbetween methods of in-depth data collection in controlled settings such asthat used in the previous example, and more informal observational researchconducted in real world settings. Using as a basis a framework created byFels from his own observations [43], they wish to determine if their meth-ods of data collection can ?enrich these observations by showing how suchcategories are produced and operate in audience experience.?3.2.2 ArtworkIamascope is an interactive kaleidoscope using a video camera to captureparticipants moving in front of a projected image space (Figure 3.1). Videoof a participant is processed to create kaleidoscope imagery displayed in theprojection space, thus reflecting an abstraction of the participant?s imageback to the participant.Based on his own observations of participants interacting with the art-work [43], Fels developed a framework for relationships that occur betweena person and an interface. The four relationships: response, control, reflec-tion, and belonging, provide different types of pleasurable experience thatcontribute to intimacy achieved between the person and the interface.3.2.3 MethodThe artwork is viewed in an exhibition space, Beta space, in Sydney, Aus-tralia?s PowerHouse museum. They conduct three studies with three sub-jects. To stimulate a normal museum experience, the subjects are invited totour the PowerHouse museum as they please but asked to, at some point,visit Beta space.Video-cued recall [92] is used to capture the situated experience with-out heavy cognitive load for the participant and to help the participantremember details during a session of retrospective reporting. When a sub-44Chapter 3. Formal Participant Experience StudiesFigure 3.1: Iamascope (Fels, 1997).ject arrives at Beta space they are video-taped (along with the artwork) forthe entire duration of their interaction.Afterwards they are taken to a room nearby where they watch the videoof their interaction and are asked to comment on what they had been think-ing. They can move back and forth through the video and pause as theyplease. This is also video-taped so that when sessions are later analyzed,comments can be associated with video events. This session is wrappedup with an interview about experience with Iamascope and the video-cuedrecall method.Data is analyzed using a method based in grounded theory [48]. Verbalrecordings are transcribed and video events coded. With no existing codingscheme for this type of experience a new coding scheme is created, incorpo-rating terms used by the participants. Codes are grouped into categories ofmovement and cognitive states. The latter is further broken into categoriesof assessing system, referring to self, response, and described behaviour. Thecoded data is then compared with Fels? framework and examined for waysin which the data confirms or contradicts categories of the framework.45Chapter 3. Formal Participant Experience Studies3.2.4 ResultsThey report that from the data, they are able to associate classes of move-ment, vocabulary and behaviour with each category of Fels? framework.They are also able to observe transitions among the categories and deter-mine common patterns or ?trajectories? of interaction. These results arediscussed in relation to each category in Fels? framework, including relevantobservations and participant quotes.They also conclude that the participants reacted positively to the video-cued recall method as it helped them more deeply discuss the experiencesthey had. Still, they caution that the verbal data should be consideredan ?interpretation? of the experience because, even though it is supportedwith video, it is produced retrospectively and out of the experience context.A final remark they make about the reliability of the method is that therewere noticeable changes in study participant interaction behaviour from thatobserved in participants not being studied. The study participants all stayedlonger and made sure to explore the artwork in its entirety.3.3 Case Three: Fa? de3.3.1 GoalsA third example is the evaluation of the interactive drama Fa?ade by Knick-meyer and Mateas [59]. They aim to present methodology that can be usedto study aesthetic and experiential aspects of a system rather than taskaccomplishment.To focus their study, they define a successful experience with Fa?ade as?one in which players experience a sense of agency, maintain engagement,and are motivated to replay in order to try different interaction strategies?.They test to see if players of Fa?ade can maintain a certain level of en-gagement in an experience that does not have concrete goals and, due tolimitations in the AI technology used to create Fa?ade?s responses, is proneto interaction failures.46Chapter 3. Formal Participant Experience Studies3.3.2 ArtworkIn Fa?ade a single ?player? sits at a computer and enters a virtual world inwhich he or she may move around, interact with objects, and have typednatural language conversations with AI-controlled characters (see screenshotin Figure 3.2). A participant ?plays? but Fa?ade is not a game in that thereare no clear goals. Their interactions influence the virtual characters andthus, shape the procession of the story. This gives the drama ?replay? valueas different story trajectories can be experienced each time.Figure 3.2: Screenshot of interaction with characters Grace and Trip ininteractive drama Fa?ade (Mateas and Stern, 2005).3.3.3 MethodRetroactive Protocol Analysis [42] is used as follows. The screen is video-taped while a participant plays Fa?ade. After playing, the participantwatches the video taken and is given a script of game play to read. Duringthis time the participant is asked to describe what is going on and what hisor her reactions mean. The participant then plays Fa?ade a second time anda post-experience interview is conducted.The above is all recorded as data for the protocol analysis. To analyzethe data they use an existing coding scheme developed to code gaming ex-periences and expand it with their own categories of Agency, Exploration,47Chapter 3. Formal Participant Experience StudiesStrategy, and Disinterest.3.3.4 ResultsOverall, six out of the eight people who participated in the study said thatthey would like to play Fa?ade again. In accordance with their definition ofa successful experience, they take this as an indicator of Fa?ade?s success.Beyond determining the success of their system, they are able to use thedata to identify standard patterns of play: one where a player explores gamestrategies and actively tries to manipulate certain outcomes and anotherwhere a player spends less time strategizing but seems more immersed inthe game role and tries to act like him or herself. They also identify threetypes of common behaviour in the face of interaction breakdowns: the playertaking advantage of the breakdown, the breakdown actually triggering anaffective state such as sympathy or frustration for the characters, and theplayer, after understanding that there has been a failure with the system,changing strategy. They are able to confirm that with all three behavioursengagement was maintained.Results are discussed using references to examples of the data in graph-ical formats. In one graphical form, the total times spent talking for eachcategory is plotted in a bar chart for each player. In another form, categoriesare plotted over time and from this, transitions between categories can beobserved.48Chapter 3. Formal Participant Experience StudiesTable3.1:AsummaryoftheInfluencingMachine,Iamas-cope,andFa?cadeparticipantexperiencestudies.InfluencingMachineIamascopeFa?cadeStudyGoalsDeterminethoughtspro-vokedbyartwork;distin-guishcausesoffrustration.Observehowcategoriesofanexistingexperienceframeworkareplayedoutintheexperienceofanartwork.Testthatparticipantsstayengagedinteractingwithnocleargoalsandpossibleinteractionfailures;testre-playvalue.NewMeth-odsBettersuitedtointerestsofartists.Torecordandanalyzesituatedexperienceofin-teractiveartusingmethodbetweeninformal,realworldobservationandformallaboratory-basedstudies.Tostudyaestheticandex-perientialaspectsofasys-tem.Continuedonnextpage49Chapter 3. Formal Participant Experience StudiesTable3.1?continuedfrompreviouspageInfluencingMachineIamascopeFa?cadeArtworkArtobjectrepresentingemotingbeing;interactbyinsertingpostcardsintoartobject.Artobjectrepresentingparticipant,mirror;fullbodyinteraction.Artobjectrepresentingcharactersandastory;text-inputdialogueinter-action.SettingRoominahouse.Exhibitionspaceinamu-seum.Notspecified;assumelab-oratory.MethodCo-discovery;videorecord-ing;interviews;methodsforevaluatingaffectivein-teractivesystems.Video-cuedrecall;RetroactiveProtocolAnalysiswithcustomcodingscheme.Video-cuedrecall;RetroactiveProtocolAnalysiswithadaptedcodingscheme;replay.Continuedonnextpage50Chapter 3. Formal Participant Experience StudiesTable3.1?continuedfrompreviouspageInfluencingMachineIamascopeFa?cadeResultsIdentifyartobjectfea-turesthatfrustratepartici-pants;observedifferentbe-havioursamongtheoryfor-mersandnon-theoryform-ers;resultsappliedtore-designofartobject.Associateparticipantbe-haviourswithframework;identifypatternsoftransi-tionamongframeworkcat-egories.Determineartwork?ssuc-cess;identifystandardpat-ternsofplay;identifythreecommonbehavioursinthefaceofinteractionbreak-down.FormofRe-sultsDiscussionofobservationshighlightedwithselectionofquotes;keywordsum-maries;detailednarrativeof3groupexperiences.Discussobservationsandinterviewsaccordingtoframeworkcategories;relatewithexamplesofparticipantbehaviourandquotes.Discussresultsmakingreferencestoaselectionofcategorizedinterviewstatementsdisplayedingraphicalformat.51Chapter 3. Formal Participant Experience Studies3.4 Contributing to Existing Study MethodologyThe three cases presented in this chapter are unique in their formalizedapproaches. Goals for the study are explicitly established beforehand. Theystudy artworks in their natural interaction contexts but under controlledsettings and with participants who are recruited as study subjects. In-depthdata is collected, including records of both participant and art object actionsand the data is analyzed according to protocols and heuristics suitable forthe context of the study.Their approaches exemplify a number of benefits to be achieved for theartist using more formal methodology:? Allowing controlled investigation of different design options for artobjects.? Concluding with suggestions for art object redesign.? Identifying general participant behaviour types and placing them intoexisting or new experiential frameworks for participant aesthetics.? Identifying participant techniques used. For example, those used whiletrying to understand an art system (Influencing Machine) or in the faceof interaction breakdown (Fa?ade).? Acquiring a collection of documented interaction experiences.? Maintaining uniqueness and depth among collected participant expe-riences.They also provide a basis from which further inquiry into the use offormal methodology to study participant experience can be made.This research takes on a problem that was only briefly touched uponin the Iamascope case: that of the effect of participant bias in the contextof a formal study. They made a short remark about noticeable changesin the behaviour of participants of the study in comparison with the be-haviour of participants not studied: studied participants stayed longer and52Chapter 3. Formal Participant Experience Studiesexplored the artwork in its entirety. There is a wealth of psychology and so-ciology research on the effects that experiment settings have on participantbehaviour. The Hawthorne effect [68], [76] and experimenter effects [86] aretwo examples. Generally, it is considered that participants who are awarethat they are ?subjects? of a study will behave differently. This work seeksto observe changes in the behaviour of participants of an interactive artworkstudy and to make suggestions on how these changes should be consideredwhen interpreting study results.A second problem taken on in this research is the exploration of theuse of pre-experience information, or orientation media, administered toparticipants in order to selectively influence their understanding of differentartwork elements. Variability in a controlled study through the adminis-tration of different orientation media to different participant groups mayhelp reveal the importance and effect of participant understanding of vari-ous elements of the art system in the participant?s overall experience of theartwork.3.5 SummaryThis chapter has detailed three cases of participant experience studies con-ducted using more formal methodology. With controlled settings and meth-ods of in-depth data collection the artists and researchers are able to evaluateart object design and develop understanding of participant behaviour whilestill maintaining their artistic goals. These studies provide a foundation forthe further investigation into formal methods for studying participant ex-perience. That is, observing the manner in which a controlled experimentsetting changes participant behaviour and developing methodology to influ-ence participant understanding of artwork elements in order to distinguishthe effects understanding of those elements have on interaction experience.53Chapter 4EchologyThe interactive sound and video installation Echology [37] is used as a casestudy for investigation into laboratory-based methods for studying partici-pant experience. Echology was designed and developed for exhibition in apublic atrium at a university. This chapter first presents the artistic motiva-tions and design guidelines that served as a basis for Echology?s development.Then, a detailed description of the installation as it was implemented forexhibition at the New Interfaces in Musical Expression conference (NIME)and for the subsequent study is presented. Finally, the intended participantexperience of Echology is discussed.54Chapter 4. Echology4.1 An Interactive Sound and Video Installation4.1.1 Echology SummaryEchology (Figure 4.1) is an interactive sound and video installation for par-ticipants to collaboratively play with directional sounds initiated by theplayful movement of Beluga whales in the water captured with a single livewebcam provided by the Vancouver Aquarium [6].Figure 4.1: Echology installed in the Open Media Environment at theUniversity of British Columbia during NIME 2005.This webcam feed provides an effective means to bring the visual ele-ments of the Beluga whales into the piece and to drive a sound space usingvideo processing. The amount of motion resulting from processing the videois mapped to the amplitudes of four sound sources. In these sources, thesounds of the Beluga whales, recorded over 3 years using underwater micro-phones, are remixed and spatialized in a surround sound system. Partici-pants play with the spatialized sounds through a button and display tabletopinterface.55Chapter 4. Echology4.1.2 Artistic MotivationsThe playful, graceful motions of Beluga whales swimming in water create amesmerizing motion space to watch. These highly communicative creaturesuse a variety of vocalizations, physical expressions and physical contact intheir navigation, social interaction and survival [96]. Artistic motivationsstem from an appreciation for watching the Belugas swim, play and com-municate at the Vancouver Aquarium, and wonderment over how watchingthe Belugas through the aquarium?s webcam can be similarly engaging. Wewere inspired to create an interactive soundscape that provides a representa-tion of the space inhabited by Beluga whales and that, through interactionwith sounds, gives participants the feeling that they are communicating orplaying with the Beluga whales. We focused on enjoyment, understandingand engagement as being the key elements of the expressive meaning of thisinteractive installation.4.2 Echology Design Guidelines4.2.1 Collaboration, Simplicity, and ApproachabilityEchology was created with the intention of installing it in a public atriumspace that will be discussed in section 4.2.2. Our vision for the piece was tocreate a focal point in the large and normally empty atrium so that passersbywould come to interact and socialize with other participants. Hence, fromthe beginning we focused on supporting participant collaboration. Blaineand Fels, referring to D?Arcangelo [32], recommend that ?when designingcollaborative musical experiences for first-time players in public places, theamount of time necessary to learn an interface must be minimized, coupledwith achieving a balance between virtuosity and simplicity? [20]. Thus wealso focused on making interaction with Echology simple. Finally, becauseEchology would be placed in a space outside of the museum or gallery wherebehaviour norms are more conducive for exploration, we focused on creatingan approachable space. These three focuses: collaboration, simplicity, andapproachability, were used as guidelines throughout the entire development56Chapter 4. Echologyprocess and are discussed below. CollaborationAs discussed in Section 1.1.2, Blaine and Fels [20] have identified many ofthe qualities to organize collaborative, multi-participant interactive soundinstallations and guidelines based on these qualities to make the sound in-teraction successful. Echology was placed in their classification system andthe corresponding guidelines were integrated into its design. The relevantqualities and recommendations followed are listed in Table 4.1.Table 4.1: Echology?s placement in Blaine and Fels classifi-cation system for collaborative musical experiences [20] andcorresponding guidelines integrated into its design.Quality Echology Design Recommendationcapacity multiplayer,single interfaceUse turn-taking protocols as well ashaving clear relationships between ac-tion and sound with a multi-person,single interface.aptitude novice Prioritize that the sound be engagingat the expense of sound, video Use video to strengthen the relation-ship between action and sound with-out being too distracting.player interaction buttons Participants use 8 buttons arranged ina circle to reflect sounds; each playerhas the same type of interaction sothat they can learn from each other.Continued on next page57Chapter 4. EchologyTable 4.1 ? continued from previous pagemusical range Players control spatial parameters ofmusic; restrict sound controls to helpcreate an engaging, satisfying experi-ence for novices.physical interface buttons Provide Norman-style affordances [50]to engage novices and make it easy forthem to join the group.directed interaction low Attendants explicitly direct novices sothey understand how the interactionworks.learning curve fast Use a direct mapping between gestureand sound to speed up learning.path to expert none Compromise virtuosity for ease-of-useas the sound installation is intendedfor novices.levels of physicality medium Arcade-style buttons encourage mod-erate levels of physical interactionthat ?lay the foundation for devel-oping intimate personal connectionswith other players. . . ? [20]. SimplicityToshio Iwai?s Composition on the Table [57] provided inspiration artisticallyand technically for the sound and interaction method used. Iwai?s work usesa large horizontal projection surface with a light grid on the display. At eachnode of the grid, a button allows the player to change the direction of thearrow associated with the node. A coloured ball of light moves along thelines of the grid and then follows a path decided upon by the direction of the58Chapter 4. Echologyarrow at each node it encounters. When a node is hit, a specific MIDI noteplays. There are four lights moving at different speeds. Thus, with carefulselection of the direction of the arrows, complex loops and rhythms can becreated. Working with multiple players allows very complex sounds thatcannot be made as effectively by a single person, since there are too manylights active at the same time. Yet even though the resulting compositionsare complex, the barrier to interaction is extremely low. A participant hasonly one method of interaction ? the button press, only four moving lights ofvisual feedback to follow, and a straightforward mapping of notes played onlight-node collision events. Along with this simple interaction, Iwai achievessimple and comfortable ambience through the use of minimalist aesthetics.We draw from what Iwai has achieved with Composition on the Tableand keep participant input limited to button presses as well as attempt tomake mappings between the Beluga webcam video, button presses, visualelements, and the soundscape as straightforward as possible. ApproachabilityEchology would be installed in a large, open, and central public atriumwhere the behaviour norm is to pass through. Other than during organizedevents held in the atrium the space is usually empty. We were aware thatpeople would possibly hesitate to enter the interaction space, either becauseof shyness to break a social norm or because of being occupied and nothaving time to stop and look. Our plan to counter this was to create anapproachable space with an air of mystery and allure.To create an inviting space, we constructed the Interaction Table to ap-pear fun and exciting. We did this through our choice of covering it in afluffy white fabric and using large, lighted, blue buttons to entice spectatorsto the table. Our original design included a projection of the Beluga webcamvideo so that those not at the table could observe and appreciate the map-ping between Belugas and sound. At the same time, the visual elements andsound maintained a mysterious ambience of the underwater world, a strangeand unknown place that provokes curiosity in many people.59Chapter 4. Echology4.2.2 Designing for the Open Media EnvironmentEchology was designed to be an interactive multimedia installation piece forexhibition in the Open Media Environment situated in the atrium of a build-ing at the University of British Columbia. The Open Media Environmentincorporates a center stage area and a movable, suspended, circular theatri-cal truss that has eight attached speakers and can support various lightingattachments. The circular truss has a rear projection screen stretched insideit. An initial motivator to create Echology was to provide an artwork thatcould showcase the Open Media Environment?s functionality. The infras-tructure available highly influenced Echology?s system design. Using thisspace also produced deployment issues that had to be taken into consider-ation. The space is in the center of a building housing offices, laboratories,and classrooms. The installation would be active during the daytime onweekdays so we had to assure that sound levels would not disrupt the peo-ple working in the building.4.3 The Echology System DescriptionThe installation consists of a custom made Interaction Table, 8 surroundspeakers, a PC running Max/MSP and Jitter [2], 8 lights and a lightingcontroller, and the live webcam at the Vancouver Aquarium. Figure 4.2depicts the arrangement of these physical components. Figure 4.3 depictsEchology from a systems point of view, indicating its inputs, outputs, andprocessing done in software. It is from this point of view that I will firstbriefly review the components of Echology.Input to the Echology system consists of the Vancouver Aquarium we-bcam [6] and the 8 Interaction Table buttons. These buttons represent thereflection points on the edge of the soundscape (i.e. the eight loudspeakers).The 8 buttons are arranged symmetrically around the perimeter of the ta-ble (Figure 4.12) and are used to control the movement of the sounds in thesonic aquarium.The Interaction Table also houses a monitor to display our visualization60Chapter 4. EchologyFigure 4.2: Echology system diagram: physical components.of the Beluga webcam and sound space for the participants. The displaygives graphical visual feedback to the participants so they can see wherethe sounds are and where they will go. The sound space is created using 8speakers distributed symmetrically just above head level. A light is mountedbelow each speaker and lights up momentarily to indicate when a soundpasses by the speaker to enhance the spatialized sound.All sound, video, and button event processing is done using the Max/MSPgraphical environment for multimedia programming [2] and Jitter, a col-lection of video and graphics objects to be used in Max/MSP. We choseMax/MSP because of its support for managing complex installation systemsinvolving multiple devices and media in real time and the ease of creatingintuitive graphical interfaces for system monitor and control. Max/MSP isalso commonly used in the video and sound art community, so support forinstallation-specific issues was readily available.61Chapter 4. EchologyFigure 4.3: Echology system diagram: inputs, outputs, and processing donein Max/MSP/Jitter.The above system description is of the original installation of Echol-ogy in the Open Media Environment. For the participant experience studythe installation was moved to an empty room in a laboratory building andmodified somewhat because of hardware constraints. At this time only sixspeakers were available so the system?s program was modified to use only6 speakers but to maintain, in the geometry of spatialized sound, the 8 re-flection points arranged symmetrically around the edge of the soundscape.The new space was also unsuitable for hanging lights without the necessarywiring becoming dangerous so the 8 lights were not used. Instead, 4 lightswere placed around the room as ambient lighting.62Chapter 4. Echology4.3.1 Beluga Video and Motion CaptureThe Vancouver Aquarium has four Belugas in the tank shown through thewebcam. From the webcam, a variety of different Beluga activities canbe observed. They have various swimming patterns, sometimes play withbubble rings, do headstands, bob their tails up in the air and perform trainedmovements during the aquarium show times. The Echology installationprocesses the live webcam feed to capture these Beluga motions and usesthem to control sound in the Echology soundscape.Motion types of the whales change given the different areas of the cam-era?s view. For example, the top portion of the view contains the tank?swater surface. The Belugas spend most of their time near this surface andoften stay fixed in one place, bobbing up and down; this activity is partic-ularly high during the training shows that occur every 1.5 to 2 hours whenthe aquarium is open. We used this idea of different camera view areas todivide the view into four ?layers? where the motion detected in each con-trols one sound signal. These four layers are evenly spaced from top tobottom as shown in Figure 4.4 . The layers were given labels of ?Splash?,?Play?, ?Swim?, and ?Deep? for layers 1 to 4 respectively. The 4 soundsignals associated with these four layers are composed to be aestheticallyand conceptually characteristic of Beluga motions within their given layer.For example, we have a playful motif for the sound associated with the Playlayer. The nature of the musical elements will be discussed further in Section4.3.8.63Chapter 4. EchologyFigure 4.4: Layering of the Beluga webcam view.Beluga motion in each layer is mapped to the amplitude of the asso-ciated sound signals. Motion is calculated by subtracting grayscale pixelvalues in consecutive frames of the Beluga webcam feed. If the differencebetween 2 frames is greater than a threshold value, the intensity of motion(or magnitude of difference) is used to modify the sound?s amplitude. Toavoid detecting the small, rapid motions of light ripples in the water, theframe?s brightness, contrast, and saturation are adjusted and the frame ispassed through a blurring filter. For further smoothing over consecutiveframes, motion values are averaged over a sliding window of 13 frames; theaverages are then normalized and sent to the sound controller. This entireprocess is shown in Figure Soundscape SpatializationBefore spatialization, the sound controller scales the amplitude of four soundsignals by the associated values received from the Beluga motion capture;the more motion there is in a video layer the louder the associated soundsignal is. If there is no motion, the sound is not heard. Often the Belugasmove in the webcam view for only a short duration of time. This makessounds disappear too quickly from the soundscape to be able to interactwith it. To remedy this we added a 10 second sustain to keep the sounds64Chapter 4. EchologyConvert to binary. If pixel value > threshold, pixel = 255, else pixel = 0.Cut frame into layers Splash, Play, Swim, Deep.Previous Video FrameAdjust contrast, brightness and saturation.Perform convolution blur with cross-shaped kernel.Video FrameConvert to grayscale.Calculate absolute difference between current and previous frames.For each layer: calculate mean pixel value. Scale to value between 0 and 100.For each layer: calculated weighted average mean value over 13 frame window.Send 4 average values to Sound ControllerFigure 4.5: Steps of Video and Motion Processing of Beluga Webcam FeedFramesplaying after they pass a threshold volume level.The four sound signals are spatialized around the installation space usinga spatial sound engine based on the Max/MSP plugin for Vector-BasedAmplitude Panning (VBAP), developed by Ville Pukki [83]. VBAP providessound spatialization using an arbitrary specification of speaker position intwo or three dimensions. This is an essential feature for installations suchas Echology that require a high degree of control over sound positions andare set up in disparate environments using fixed infrastructure.Amplitude panning involves sending the same sound signal to a numberof different loudspeakers, with amplitudes based on the loudspeaker posi-tions and the location of the intended sound signal. Using VBAP allows65Chapter 4. Echologyus to easily fit the loudspeaker setup to the space our installation will in-habit. With the VBAP plugin, the speaker placements can be changed onthe fly in a modular fashion that does not affect any of our other softwarecomponents.This method of spatialization is used in a panner that accepts up to eightchannels of input sound signals. Each channel can exist in a unique sound-scape position at the same time. We use four of these, one for each soundsignal. A panning controller calculates and maintains x and y coordinatesfor each sound signal. These coordinates are sent to the panner, as well asto the graphics and lighting controllers discussed below.By default, sound is spatialized along the perimeter of the installationspace defined by a circle made by the symmetrical placement of the speak-ers (Figure 4.6). We also define 8 locations positioned symmetrically onthis circle as ?reflection points? (Figure 4.7). Each reflection point has a?direction? that is set to direct to one of the other 7 reflection points. Asa sound signal passes a reflection point, it continues along the path of thatreflection point?s current direction. This way, a sound signal can travelalong any path between two reflection points, and thus, on paths within a2-dimensional plane bounded by the spatialization perimeter. A reflectionpoint can also direct to itself and in this case, a sound that arrives at thepoint becomes caught until the point?s direction is changed again. This isshown as the rightmost state of reflection point B in Figure 4.8. The de-fault position for each reflection point is to direct towards its neighboringreflection point so that sounds travel in a circle around the soundscape asin Figure 4.7.66Chapter 4. EchologyDefault sound pathFigure 4.6: Two-dimensional sound plane above the Interaction Table de-fined between ring of 8 speakers.A reflection pointFigure 4.7: Eight reflection points placed at equally spaced positions aroundthe sound plane. A reflection point is represented by a circle with a pie sliceremoved. The pie slice indicates the point?s direction of reflection.67Chapter 4. Echologya0 D E FSound travels to:BA BA CDEFGHFigure 4.8: The eight directions of reflection point B. This diagram showsthe orientation of B as it directs towards reflection points C to A. In therightmost, doughnut shape depiction of B it is directing towards itself.68Chapter 4. Echologya1BCDEFGHFigure 4.9: In this example of a sound path between reflection points, pointsA, C, and E have directions to C, E, and A respectively. Sound will travelin a path between A, C, and E.This is illustrated with the example shown in Figure 4.9. The circleswith pie-shaped cutouts represent the reflection points. When a sound hitsreflection point A, it reflects towards reflection point C. When it hits reflec-tion point C, it reflects to point E. When it reaches point E, it returns backto point A. This pattern continues until the direction of reflection point A,C, or E is changed.4.3.3 Participant Control of Sound SpatializationEight buttons on the Interaction Table correspond to the 8 reflection points.The panning controller accepts button press via a Phidget interface andwhen a button is pressed, the direction of reflection for the associated reflec-tion point is adjusted. Each press of the button rotates the reflection pointdirection by one in a counter-clockwise direction including pointing to itself.69Chapter 4. EchologyThus, by pressing the 8 Interaction Table buttons, participants can changethe directions of sound spatialization in the soundscape.4.3.4 Graphical FeedbackParticipants receive feedback of current reflection point directions and spa-tial positions of the sound signals through graphics overlaid on the web-cam video shown in the Interaction Table display (Figure 4.10). The pan-ning controller sends x and y coordinates for each sound signal and buttonpress events to the graphics controller which generates graphics using Jit-ter?s OpenGL capabilities. The graphics, as detailed below, are intentionallykept simple so as not to detract from the live Beluga webcam feed in thebackground.Figure 4.10: Interaction Table display of webcam feed with graphical feed-back overlay.As introduced in Section 4.3.2 and shown in Figures 4.7, 4.8, and 4.9,reflection points are depicted by the row of eight circles with a pie slice70Chapter 4. Echologyremoved. Sound reflects in the direction of the pie slices wide end. Areflection point directed to itself is depicted by a doughnut shape, indicatingthat it will ?catch? sounds.Four circles of different colours move about the Interaction Table display.Each circle directly represents the location and amplitude of the sound sig-nal associated with one of the webcam video layers. The following colourmapping is used to differentiate the signals: Splash is purple, Play is yellow,Swim is green, and Deep is blue. The amplitude is mapped onto the alphavalue of the circles, so that they fade in and out as the level of activity ofthe Belugas increases or decreases in the corresponding layer of the tank.When there is no activity and therefore no sound, the circle disappears.4.3.5 Phidget InterfaceA Phidget [5] interface is used to detect button presses on the InteractionTable and to send an output signal to the lighting controller. The Phidget8/8/8 is an inexpensive digital acquisition board that provide simple mech-anisms to connect and control various sensors and actuators to a computervia USB. It allows for 8 analog inputs and 8 analog outputs. We chose tobuild Echology with this board as it is easy to use and a C/C++ API makesfor easy integration of its operations into other software applications.To access the input and output channels, a Max/MSP external Phidgetobject was written. A button press causes the object to send a signal inthe panner controller software module, changing the direction of a reflectionpoint. This same software module sends a signal to the Phidget outputs toturn on lights when a sound hits a reflection point.4.3.6 LightingEight lights are suspended from the truss (as shown in Figure 4.11) at po-sitions corresponding to the positions of associated reflection point on thesoundscape perimeter. The panning controller sends the x and y coordinatesof the four sound signals to a lighting controller which turns the 8 lights onand off through the Phidget interface. The Phidget interface output is con-71Chapter 4. EchologyFigure 4.11: Light hanging from truss speaker.nected to a lighting controller board (a Light-o-Rama MC-TB08 8 ChannelTriac Board) that allows eight 110V light bulb sockets to be switched undercomputer control. Thus, when a sound hits a reflection point, the lightingcontroller momentarily lights the associated reflection point.4.3.7 Interaction TableThe functions of the Interaction Table buttons and display have been dis-cussed in the previous sections, but it is important to note the design ofthe table itself. The Interaction Table (Figures 4.12 and 4.13) is the focalpoint of Echology. It is placed in the center of the installation space and isdesigned to provide the aesthetics of an aquarium atmosphere and to createand inviting, collaborative space. Specific requirements of the table are todisplay video on the tabletop, to have eight buttons arranged symmetricallyaround the video display, and to have space to contain all wires, circuitry,and ideally, the PC controlling Echology.The tabletop is circular with sufficient diameter to allow a larger numberof participants to comfortably stand around it and see what is going on. Wechose large, glowing buttons to draw people towards the table and entice72Chapter 4. EchologyFigure 4.12: Interaction Tabletop.Figure 4.13: Interaction Table with screen removed.pressing. The buttons, purchased from a supplier of arcade machine parts,Happ, are light sea blue, 3 inches in diameter and illuminated inside witha small DC lamp. When a button is pressed, the light turns off to provide73Chapter 4. Echologyimmediate, direct, visual feedback. The large buttons also fulfilled the designrequirement of having an interface robust enough to withstand sustained useby the general public.The tabletop is a custom-designed, wooden board with eight circularholes for the buttons and one larger square cutout for the display. Four?L?-shaped blocks of wood affixed around the display cutout hold an LCDmonitor in place. The blocks can be loosened and rotated to the side so thatthe monitor may be easily placed in and taken out. Four small clamps areused to fix the tabletop onto the table?s base.Rather than use a traditional four-legged table form and curtains to hidehardware placed under the table, a cylindrical-shaped base is used for a softerappearance. An oil drum provides this shape; it is also a sturdy base and cancontain the Phidget boards and wires needed for button and lighting control.A hole drilled at the base of the drum allows power, ethernet, video, USBcabling, and wiring for the lighting controller to be fed out. The originalplan was to also place the PC inside the drum but because of overheatingissues this was not done. Instead, during NIME the PC was hidden underthe atrium stage and for the study it was placed some distance away fromthe interaction space.Finally, the table base and top is covered in a fluffy white fabric for analien, underwater aesthetic. The tabletop itself has a clear plastic cover togive the impression of a Beluga?s skin. It also has the functional propertyof protecting the monitor surface.4.3.8 Musical ElementsTo enrich the soundscape musically, various sound sources including actualBeluga sounds and composed segments were selected. Although each soundsource had to be elementally simple enough to achieve successful interactionand spatial perception, we also wanted the soundscape to be rich enough togive the audience the feel of playing with the Belugas underwater.Two categories of sounds are used in Echology. The first consists ofshort samples of Beluga whale voices received from the Vancouver aquarium74Chapter 4. Echologyresearch team. The second category is a number of composed synthesizertracks meant to create an aquatic atmosphere and provide continuity andcontext to the soundscape.The sound signals of the four layers in Echology?s soundscape are asfollows:1. ?Splash? contains a synthesizer motif creating an aquatic atmospherewhen a whale moves across the top of the aquarium.2. ?Play? loops several short samples of Beluga voices that are randomlytriggered by the Beluga movements.3. ?Swim? consists of additional Beluga voice samples accompanied withthe drone of synthesizer sound.4. ?Deep? contains low frequency mass to create a drone.We selected high frequency Beluga voices since they provide better cuesfor spatial perception. Atmospheric background synthesizer sounds are de-signed to give continuity to the entire soundscape. In general, there isenough variation in the soundscape to be engaging and interesting becauseof the variety of motion of whales in the aquarium.4.4 Desired Echology Experience4.4.1 Participant Experience SummaryEchology allows participants to experience a sonic aquarium while feelinglinked to the live, organic and mesmerizing movements of the Beluga whales.Participants can play with the spatialized sounds initiated by Beluga whalemotion and watch the webcam feed in the table?s display. Using the buttons,participants control the path of sound overhead, or if desired, they can justwatch and listen to the sounds move along their predetermined course. Themelon of the Beluga (the fleshy part on the top of their head) inspires this useof redirected sound. We intentionally made the interaction simple, so that75Chapter 4. Echologyparticipants need only focus on the spatial aspects of the sound rather thanthe tonal qualities. The imagery of Beluga whales swimming blends with thesound spatialization and visualization to make a rich, playful, and enjoyablemediascape. The whales? play results in movement patterns that are fairlyrepeatable, but unpredictable and organic. A participant can listen to thesoundscape from anywhere inside the atrium; however, the ?sweet-spot? isat the Interaction Table located in the centre.4.4.2 Connecting with the BelugasUsing a webcam feed of live Belugas raises interesting ethical questions wehave tried to address in the artwork. Though they are in an aquarium envi-ronment, the Belugas are wild animals that cannot be fully controlled. Theyare there for us to learn from them, but demand proper respect. For thisreason, we gave participants only partial control over the sounds. Soundsfade in and out according to the activity of the Belugas in each layer; if thereis no Beluga activity at all for a certain amount of time, the installation willbecome silent until the Belugas return. Only when there is a significant levelof activity in all layers will all the sounds be present simultaneously. Thisaspect of the interaction encourages participants to reflect on the Belugawhales as living creatures seen in real time. As such, participants are notable to demand performance from the whales, but must patiently wait forthem to swim into view.76Chapter 4. Echology4.4.3 Participant and Art Object RolesTo create the above described experience, it is desirable for the Echologyart object and participants to play out their roles as detailed in Tables 4.2and 4.3.Table 4.2: Desired manner with which Echology plays out itsroles as an art object.Participant Interaction Trigger Trigger interaction with an approachablespace and large buttons that afford beingpressed.Interaction Guidance Support interaction with limited button presscontrol and simplistic mappings between web-cam feed, button presses, visual elements, andthe soundscape.Representation Create an atmosphere representative of thespace inhabited by Beluga whales and presentthe Belugas as real living creatures that par-ticipants may admire and connect with.Communication Facilitated Create a social space for collaboration andcommunication among participants; give par-ticipants the sense that they may play andcommunicate with the Beluga whales.77Chapter 4. EchologyTable 4.3: Desired manner with which participants of theEchology artwork play out their roles.Basic Interaction Understanding relationship between buttonpresses, graphical feedback, and the sound-scape; being able to distinguish between dif-ferent sound channels and the positions ofsounds being spatialized.Imaginary Contribution Realizing that the Belugas are those living atthe Vancouver aquarium; knowing that thevideo is live and aquiring a sense of connectionto the Belugas.Creative Contribution Experimenting with different patterns ofsound spatialization.Involvement in a Social Space Creating a conversation space around the In-teraction Table, hanging out, communicatingwith strangers, and collaborating to createpatterns of sound spatialization.4.5 SummaryThis chapter has discussed Echology, an interactive sound and video in-stallation designed for collaboration, simplicity, and approachability. TheEchology system as designed and implemented for presentation in the OpenMedia Environment was detailed. The intended interaction experience andexpectations for art object (the Echology system) and participant roles arepresented. This frames motivations behind the participant experience stud-ies of Echology discussed in the next Chapter.78Chapter 5Studying ParticipantExperience of EchologyThis chapter summarizes informal observations of participant interactionwith Echology during its first public presentation at a conference. Thisexperience confirmed that initially established expectations for participantexperience as discussed in Section 4.4 were mostly achieved. Yet still, ques-tions remained about factors that may have influenced participant engage-ment and the levels of participant understanding that had been achieved.Section 5.1.2 poses these questions and then Section 5.2 details the designof a laboratory-based study conducted to more rigorously investigate them.The study methods used are based on those of the cases reviewed in Chap-ter 3 with the added use of orientation media for comparative analysis.Section 5.3 presents study results by first summarizing resulting participantexperiences and then providing interpretations and implications of these re-sults from two perspectives: an artist?s and an HCI practitioner?s. Finally,Section 5.4 reflects on the study experience itself; presenting insight into thelaboratory setting?s effect on participant behaviour and then reviewing theuse of orientation media and formal methodology in general.79Chapter 5. Studying Participant Experience of Echology5.1 Informal Observations at NIMEEchology was exhibited for the first time in a public setting during the 5thInternational Conference on New Interfaces for Musical Expression (NIME?05) in Vancouver, Canada [36]. It ran for one day, during which timesubstantial feedback from conference attendees and members of the gen-eral public was received. More than one hundred people were informallyobserved interacting with Echology. The next section summarizes generalobservations and interpretations made of them. The following section raisesquestions that remained after digesting the NIME observations.Figure 5.1: Echology visitors watching Belugas at NIME.5.1.1 Summary of Observations and Feedback ReceivedThere was a prominent pattern in visits to the installation space. A partici-pant would walk up to the table, glance at the display, and then look at theoverhead lights and speakers. He or she would then focus on the display andeither press a button to receive a reaction, or watch what other people weredoing. Most people asked questions about the installation, how it worked,how it was made, and so on. After discussing the installation itself, theconversation often opened up to other topics or participants would spend afew quiet moments by themselves, enjoying the Belugas and soundscape.We achieved the quality of approachability by creating an inviting space.Participants were immediately drawn to the Interaction Table upon walking80Chapter 5. Studying Participant Experience of Echologyinto the installation space. They did not hesitate to try pressing the buttons.Moments of silence when there were no whales did not feel awkwardamongst participants at the table. They would patiently wait in a mannersimilar to a guest at the aquarium quietly waiting for an aquatic creature toswim into a more visible area. They were comfortable with being silent fora few minutes or spent time conversing about the Belugas, easily breakingthe silence when desired. This allowed some participants to stand aroundthe table and chat for lengths of time up to 20 or 30 minutes. Thus thepiece worked very well as a ?calm? technology [99].Also, without explicit instruction participants chose a button to stand infront of, and subsequent use of that button was solely by that participant.That button became their ?territory?. Territoriality has been consistentlyobserved in research of tabletop displays for computer-supported collabora-tive work [88] [94] and an effect of it is that it helps people mediate theirsocial interaction through the very act of laying claim to a space [95].For some, the connection between the Belugas and the mode of interac-tion was not well understood. The abstraction between how Belugas com-municate, their actions, and what participants could do was not obvious tothem. Usually an attendant had to explicitly state that the Beluga webcamimages were indeed live and not prerecorded. It is difficult to determine theimportance of the audience understanding that the Belugas were live andhow much that understanding changes the metaphor. Because of the livenature of the Beluga camera, some people wanted to take the next step ?to be able to communicate directly with the Belugas. Many people asked,?can the Belugas hear the music too??5.1.2 Questions RemainingThe above experience proved Echology to be approachable, a provoker ofcuriosity and exploration in people, as well as collaborative in that it sup-ported a social space for communication among participants and joint playwith the Belugas. At the same time, we were not convinced that our goal ofcreating a simple and easy to learn interface had been achieved. Questions81Chapter 5. Studying Participant Experience of Echologyremaining were:1. If an artist were not available to explain how interaction and Echologyworked, would the art object alone be able to support participant inter-action and understanding? Furthermore, how much was engagementbased on figuring the system out rather than enjoying the soundscapeand visuals, collaboration with other participants, or feeling connectedwith the Beluga whales?2. How much was enjoyment based on the Beluga video being from a livewebcam feed? Would it matter if it was pre-recorded?3. How did interacting with Echology collaboratively differ from experi-encing the piece alone?This motivated a need to investigate Echology in a more in-depth man-ner than what could be conducted at NIME, or in any other exhibitionsetting. With a more formalized approach in a controlled laboratory settingparticipant experiences could be investigated in greater depth. Variationsin experience could be administered as media given to the participants toorientate them towards understanding of different functional and conceptualaspects of the system. Conducting a study in a controlled laboratory settingand using orientation media as described in Section 3.4, we sought to ob-serve differences in interpretation and understanding between participantswith and without:1. An explanation of interaction with Echology to address the first ques-tion.2. Knowledge that the Beluga webcam feed is live to address the secondquestion.3. Another person to experience Echology with to address the third ques-tion.82Chapter 5. Studying Participant Experience of Echology5.2 Laboratory-Based Experience StudyA study with 8 session types was designed to observe the effects of theabove three points. The session types were based on combinations of threevariables: whether the participant received instructions or not, whether theparticipant was told the Beluga feed is live or prerecorded or not and whetherthe participant viewed the live webcam feed or a prerecorded version (Fig-ure 5.2). Furthermore, sessions were conducted with either a single partici-pant at a time or with participants in pairs.Figure 5.2: Eight session type study matrix based on variables: (a) whetherparticipant receives instructions or not, (b) whether the participant is toldthe Beluga feed is live or prerecorded or not, and (c) whether the participantviews the live webcam feed or a prerecorded version.All participants experienced Echology without the presence of the ex-perimenter so that there would be minimal chance to ask questions and re-ceive information influencing their interpretation. Also, an effort was madeto maintain consistency in dialogue used and information given across allsessions. This care was taken from when the experimenter first met theparticipants to during a final interview, at which point conversation couldbecome more open.83Chapter 5. Studying Participant Experience of Echology5.2.1 Orientation MediaTwo classes of orientation media were created. The first class addressedthe question of how a person?s understanding of interaction affects theirexperience of the artistic concepts and aesthetics. Here orientation mediain the form of a set of cards with instructions as shown in Appendix A.2 wasused. Participants in the ?instructions? session types were given these cardsto reference while viewing Echology. Participants of the ?no instructions?sessions were not.The second class of orientation media consisted of a short descriptionof Echology written on a card placed on the Interaction Table. One of threepossible cards (with descriptions as shown in Appendix A.3) were placed forevery participant, the descriptions on each card being almost identical withthe exception of one part specifying one of the following:1. Echology uses a LIVE video feed of the Beluga whales.2. Echology uses a PRERECORDED video feed of the Beluga whales.3. Echology uses a video feed of the Beluga whales.Echology was accordingly presented with the live feed or a prerecordedversion as specified in the first two cases, or in the third case, either.5.2.2 ParticipantsA total of 26 paid participants, 14 women and 12 men, were recruited viaan Internet-based experiment recruiting system used within the University.People were invited to participate in a ?User Interface and Experience Studyof ?Echology?: an Interactive Spatial Sound and Video Artwork?. The entirerecruitment advertisement is shown in Appendix A.1.Ages ranged from 19 to 36 with an average age of 23.7. All Participantsexcept for one person were students at the University of British Columbia,the majority in the engineering and computer science departments, andused computers on a daily basis. Half of the participants had no experiencewith interactive artworks while among the other half with some experience,84Chapter 5. Studying Participant Experience of Echologyonly 3 people had had several experiences. When asked about video gameexperience and musical education, most participants indicated having someor more while only 2 people had no experience playing video games and 6people, no musical education. Participant data listed according to sessiontype can be found in Appendix B.1.Sixteen people participated in the study alone while 10 people partici-pated in pairs resulting in 5 groups of 2 people. In all 5 groups, participantshad not previously met their partner.5.2.3 SettingThe study was conducted in a usability lab on campus. Two rooms wereused: a small room inside of the usability lab where participants were met,orientated, and later interviewed, and a larger, mostly empty room, outsideof the usability lab area inside which Echology was installed. The relation-ship between the two rooms is shown in Figure 5.3.The small room was sparsely equipped with a desk, 3 chairs, a monitorfor viewing video, and a laptop used for audio recording of the interviews.For the installation space, we had considered showing Echology in the OpenMedia Environment, which neighbored the building housing the usabilitylab, but decided against this for a number of reasons. One being to avoidparticipants feeling that they would be watched by people passing by in apublic space and another being that the studies would be conducted over thecourse of more than a week but for security reasons, Echology would need tobe taken down every evening. Instead Echology was placed in a secure room,empty save for some scaffolding and a few large screen displays. Echologywas installed as shown in Figure 5.3. A video camera was placed in theroom to record participant interaction.Notable differences between the laboratory and the NIME installationsare that, first of all, there were only 6 speakers available for use in thelaboratory installation. The eight reflection points were still arranged sym-metrically around the soundscape but instead of each point being in thesame position as a speaker, most lied between speakers. A second difference85Chapter 5. Studying Participant Experience of Echology~ 25 metres between roomsTesting roomCamFigure 5.3: Study setting: Echology installed in room on left-hand side,interviews conducted in room on right-hand sidewas that during the NIME installation, the lights did not seem very usefulfor providing feedback of the sound paths. Instead their blinking appeareddifficult to follow, almost random, and distracting. They were not used forfeedback in the laboratory installation. Four of the eight lights were placedaround the space and kept on the entire time to create an aquarium-likeatmosphere.5.2.4 Study ProceduresOrientationParticipants were met in the usability lab and given a short introductionto the study. They were told that they would be taken to another roomto see an interactive art installation piece, that their experience would berecorded on video, and that afterwards, they would be interviewed abouttheir experience. It was stressed to them that this was not a ?task? and theywere not being ?tested? on performance or understanding. They were thengiven orientation media dependent on which session type they belongedto. All participants received one of the Echology introduction cards andsome received the set of instruction cards.86Chapter 5. Studying Participant Experience of EchologyViewingAfter the orientation, participants were taken to view Echology. They weretold that they were free to touch and play and could spend as much oras little time in the room as they wanted. To avoid the close presenceof the experimenter and concern about when to return, participants weregiven a walkie-talkie. They were to use it to notify the experimenter whenfinished viewing Echology. After verifying the participants understood howto use the walkie-talkies, they were taken to the installation room. Toavoid initial reactions directed to the experimenter, the experimenter didnot follow participants as they entered the room.InterviewAfter receiving notification that participants were finished viewing Echol-ogy, the experimenter came and took them back to the initial experimentroom. They were seated in front of a computer monitor from which theycould watch the video just taken of their experience. While watching thevideo, an open-interview guided by questions listed in Appendix A.4 wasconducted. Following the interview, participants answered a short question-naire (Appendix A.5).Video-Cued RecallThe study was initially designed to use the video-cued recall method. Partic-ipants would be able to view a recording of their interaction with Echologyduring the interview. During pilot studies it was found that the video didlittle to help participants recall specific moments during their experience.This was explained for by the low range of participant physical interactionand the repetitiveness of the art object?s states. Concluding that video-cuedrecall would not be very helpful, further studies were conducted without themethod. Still, video was prepared for viewing in the event that it couldsupport the interview.87Chapter 5. Studying Participant Experience of Echology5.2.5 Data CollectedThe following forms of data were collected:? video footage of participants at the Interaction Table? transcribed audio recordings of interviews? questionnaire results including general demographic information, re-sponses to multiple choice questions, and free writing responsesInterview transcripts were analyzed to pinpoint areas of participant mis-understanding and frustration in order to make comparisons between sub-jects with instructions and those without. It was also desirable to determinethe different levels of participant engagement as well as interpretations andmetaphors used to describe the experience. After reading the transcripts fora first time, a classification for statements of interest was developed. Duringsubsequent readings, statements were placed among the following categories:? Echology as a game and Echology as art includes statements re-lated to discussion held with all participants on their views of whetherEchology was more a game or more art, as well as random statementsmaking reference to Echology being a game or an artwork.? Understanding of Echology Interactivity includes statements re-vealing participant mental models of interactivity and Echology?s func-tions, as well as expressions of confusion and frustration or confidencein understanding.? Sensation and Enjoyment includes statements related to Echol-ogy?s atmosphere, musical elements, metaphors attributed to the art-work and general expressions of enjoyment.? Connection with the Belugas and the Live Webcam Feed in-cludes statements about the Beluga whales and the webcam feed.88Chapter 5. Studying Participant Experience of Echology? Participant Expectations and Technique includes statements re-lated to participant expectations of the experience and the purposesand paths they create in interaction.? Breakdowns includes statements made during discussion of break-downs that occurred during a small number of participant experiences.? Collaboration includes statements related to collaboration.All participants were given a pseudonym for easy identification. Thesenames and relevant participant information can be found in Appendix B.1.Questionnaire results are listed in Appendix B.2 and statements from theinterview transcripts are listed in Appendix B.3 according to the aboveclassification system.5.3 Study ResultsTo address the questions posed in Section 5.1.2, study results are discussedconsidering the following facets of participant experience: visual and audi-tory enjoyment, understanding of Echology interactivity, connecting withthe Belugas, collaboration, and participant tendencies.5.3.1 Visual and Auditory EnjoymentEchology was successful in creating an immersive, aquarium-like atmospherethrough musical composition, lighting, and the Interaction Table and spacedesign. Participant impressions were revealed in responses to interview ques-tions on their first impressions of Echology, what they liked and did not likeabout the experience, and what they may have related the experience to (Ap-pendix B.3, Table B.6). They were also apparent in questionnaire responsesto the request for 3 words that they could relate to their experience (Ap-pendix B.2, Table B.3). Below are keywords extracted from the responsesto summarize these impressions:? comfort, calmness, relaxing, and peaceful (Anne, Carl, Beth and By-ron, Ben, Barbara, Ambrose, and Claude)89Chapter 5. Studying Participant Experience of Echology? the unknown, curious, mystical, mysterious (Brian, Chris, Carl, An-drea, Byron, Ben, Claude, and Arthur)? immersion (Chris: ?like a total surrounding environment that you areactually involved in?, Barbara: feels in the middle of where soundfrom, Cory: ?feel like you are actually in this place?)? aquariums and underwater (Andrea, Charles, Denise and David, Cory,Cassandra)? a different space or world all together (as opposed to the laboratoryroom) (Carl: feels separation from outside world, Andrea: like starwars in spacecraft, Byron: ?environment is very fantasy?, in a for-est, feels removed and far away, Ambrose: ?quasi-futuristic?, Claude:ghost house, Denise: space-like)5.3.2 Understanding of Echology InteractivityInterview statements expressing correct and incorrect beliefs of how thesystem worked, questions remaining, and feelings of confusion or frustration(Appendix B.3, Table B.5) were considered to assess participant understand-ing of Echology interactivity. From these statements general understandingof the following aspects of Echology was extracted for each participant: thecontrol of reflection points with the buttons, the movement of graphicalballs in relation to the reflection points, the relationship between the soundand the graphical balls, the movement of sound in the soundscape, and therelationship between Beluga whale movements and sound.Changing reflection point direction with buttonsAll participants understood immediately or quickly that they could interactby pressing the buttons. Almost all participants saw that when a buttonwas pressed, the associated graphical reflection point would change and thatpressing a number of times would cycle the point through its seven reflectingdirections and donut-shaped phase. They understood that this was the90Chapter 5. Studying Participant Experience of Echologyextent of their ?direct? control. Five subjects expressed some belief thatpressing a button could trigger a sound (Arthur, Beth, Byron, Deborah), orthat its control was extended to changing the characteristics of the sound(Ben).Reflection point direction defining paths of graphical ballsAll participants understood that reflection point directions controlled thepath taken by the graphical balls. They also saw that if a reflection pointwere pointing to itself, it would catch the sound. This last point causedsome indirect frustration for two subjects (Andrea, Byron) as it gave themthe impression that an objective of Echology was to catch the balls. Aftercatching balls in a reflection point and then pressing the button once more,balls were released to continue their original movement. They were leftwondering how they could really catch the sound.Movement of sound in soundscape and graphical ballrepresentationAlmost all participants could hear the movement of sound around the sound-scape and understand that the graphical balls represented this moving sound.Confusion arose in the fact that there were multiple balls with differentcolours and their appearance and transparency varied (Anne, Arthur, Aman-da, Carl). A few participants were able to distinguish that the differentcoloured balls appeared when different types of sound played, but otherswere not able to distinguish the separate sources of sound as well.Relationship between Beluga whale movements and soundA generally satisfactory level of participant understanding broke down whenit came to understanding how the movement of Beluga whales controlled thevolume of sound. Only a few participants perceived the sound being relatedto the Belugas (Andrea, Ambrose, Cory, Chris, Charles, Deborah).91Chapter 5. Studying Participant Experience of Echology5.3.3 Connecting with the BelugasFeelings of connectedness with the Belugas were probed during the inter-views by asking directly what was thought or felt about the Belugas, andindirectly by posing the question: ?If you could talk with the Belugas atthe aquarium, what would you say to them?? (Appendix B.3, Table B.7)A questionnaire question asking participants about what they believed thegeneral artistic concept of Echology to be also revealed to what extent theBelugas left an impression on them (Appendix B.2, Table B.3).The resulting number of participants who were left with impressionablethoughts on the Belugas was quite low. Some noted that they did not payattention to the Beluga video, as they were concentrating on the sound orgraphics (Andrea, Deborah, Ben, Barbara, Claude, and Arthur). Otherparticipants were annoyed by the video?s slow frame rate and low resolution(Anne, Chris, Carl, Beth, Ambrose, and Denise), with two participants (Carland Beth) remarking that they were unengaged by the video as it appearedto be a ?repeated sequence of images?.There were no remarkable differences in attitudes towards the Belugasbetween participants told the feed was live and those not. Still, there weretwo cases of particularly strong reactions to the live feed that are of interest.In Ben?s case, he became aware of the feed being live only after his expe-rience, during the interview. He was quite amazed by this and it changedhis perception of the artwork as a whole: ?It?s really interesting, that in-teresting. .. that?s real whales over there (. .. ) omigosh, I didn?t realize. Ijust thought it?s like a computer game.. . ? In Burke?s case, he was alreadyaware that the video feed was live but this was further reinforced when thevideo froze due to network problems. Interestingly, he was fairly acceptingof the breakdown and said that he understood how the webcam feed mightfreeze up. He later mentioned wanting to go to the aquarium sometime tosee the Belugas with his own eyes.Ways in which other participants connected with the whales were byimagining that they were swimming with them (David), wondering if thewhales are bored in their habitat (Charles), wanting to touch the whales92Chapter 5. Studying Participant Experience of Echology(Barbara), and wondering if the real whales could be fed through the artwork(Carl).5.3.4 CollaborationBehaviour observed in the five cases of paired participants was different fromthe relaxed, informal social space observed during NIME. Viewing the videorecording of their experiences, the pairs seemed to not mind each other?scompany and there was some joking around and laughter, but for the mostpart, they concentrated on their ?task? and most discussion centered onunderstanding how Echology worked. This focus on ?task? is discussed whenconsidering participant behaviours as study subjects in Section 5.4.1. Therealso seemed to be less collaboration in creating patterns of sound movement,apparent in behaviours such as one person pressing many buttons while theother only watched, or a participant reaching across the table to press abutton directly in front of the other participant instead of communicatingthat they would like the button to be pressed.The atmosphere of the study setting likely had high influence on collabo-rative behaviour and may have actually contributed to creating a less social,personal space. Many participants expressed, in a positive way, feelings ofsolitude and loneliness. Even among the pairs, one participant expressed:?I feel that I?m alone, but I?m not lonely? (Byron). The room in whichthey viewed Echology was dark and quite isolating in comparison to theopen and public setting of the Open Media Environment during NIME. Thespace created experiences that were highly individual and introspective.5.3.5 Participant Behaviour TendenciesParticipant behaviour was shaped by tendencies for the participant to inter-act in modes that surfaced in discussions of expectations, actions taken, andenjoyment (Tables B.6 and B.8 in Appendix B.3). These modes are play,seeking goals, directions, and performance indicators, figuring things out,and passive spectatorship. Tendencies towards one or more of these modesinfluenced the degree and manner with which participant sought control in93Chapter 5. Studying Participant Experience of Echologythe artwork and consequently, what engaged them.Tendency to PlayA participant with a tendency towards play sought and received enjoymentfrom activity: acting on something and receiving a reaction. Their sense ofcontrol was in actions available and the corresponding changes in the system:?.. . if you play with these buttons and you can actually change the sound,how the system sounds, you can like, if I do this I can affect something. Ican enjoy what I did, if I find some particular setting I can get better soundeffect? (Carl). In Echology, this was the action of pressing buttons to causechanges in the reflection points, thus influencing the motion of sound in thesoundscape. A participant seeing the opportunity to control sound by creat-ing patterns in its movement or by catching it could amplify this amount ofcontrol. Still, for simplicity, control in Echology is limited and a participantwith no other tendencies could lose interest quickly. David said that he likedpushing buttons and looking at the screen but, ?it would be better if thescreen has something more different, more interactive, more interesting thanthe pacmen turning around (. . .) Cause you have to wait for the balls toget to the point where the sound is resurrected right? (. . . ) When there?sno sound you get frustrated.? Another participant (Carl) wondered why hedidn?t have more control over the sound or over the Belugas: ?. . .you adjustthe orientation of those reflection points maybe it is easier for them to getfood then you won?t, sounds won?t be sad and you get cheerful sounds?.Tendency to Seek Goals, Directions, and Performance IndicatorsFor participants with this tendency, lack of direction or purpose imposed asense of lack of control and thus, reduced enjoyment. Audrey, who experi-enced Echology with Amanda, said, ?. . .it was hard to just sort of. . . justenjoy the balls and we were trying to figure out some sort of point.? Sheexpressed frustration not being able to see this point. This tendency isstrongly influenced by the context of the study. One participant (Cassan-dra) sought direction because of the study setting and, not knowing what to94Chapter 5. Studying Participant Experience of Echologydo, she felt confused. Other participants viewed Echology as more a gamebut to be completely a game, they felt there should be rules or goals. Fourparticipants (Anne, Beth, Carl, David) related to Echology as an interactiveexhibit that could be used for education on the Beluga whales. Here expec-tations were in the form of learning goals and Echology, lacking informativecontent, did not fulfill their expectations.Tendency to Figure OutParticipants with this tendency found enjoyment in understanding the artobject. They felt control in understanding either how they could interactand how the art object would respond or in understanding how the art objectworked from a technological perspective. An example of the latter is Burkewho was interested in the programming technology for sound spatializationand Arthur, ?I like to know how stuff is working so that when I first cameinto the room instead of enjoying it as art I just, I figure out what does thisdo. .. ? For these participants, it was not so important to have goals builtinto Echology because they created their own goal to figure the system out.Steps towards understanding resulted in satisfaction and enjoyment. ?I wantto know how it works but, I don?t know how it works and then I just startplaying and then once I know a little bit more I start to enjoy it because Iknow a little bit more of it.? (Arthur) On the downside, participants limitingthemselves to this satisfaction would possibly find no further enjoyment ininteraction after figuring the system out.Tendency to be a Passive SpectatorThese participants took enjoyment in what could be sensed immediately, vi-suals and sound, and less in the interaction. One person (Anne) mentionedpassive enjoyment explicitly: ?It would be nice to sit in the middle andwatch a movie?, and another (Denise) hinted at a preference for passive ap-preciation: ?(speaking about interactive art) . . . it?s kind of frustrating too,if you don?t know what it?s meant to do or what you can do with it. You canpress buttons or do things on the screen and nothing happens or, I think it?s95Chapter 5. Studying Participant Experience of Echologykind of frustrating when you try to interact so I just watch.? While partic-ipants could find enjoyment in Echology?s atmosphere and soundscape, theartwork?s total expression would not be experienced without appreciationfor the interactive elements. This could especially cause disappointment forsomeone actively searching for Echology?s expressive elements as an artwork.It is suspected these tendencies were strongly influenced by a view ofwhether Echology was more an artwork, or more a game. This was inves-tigated by directly posing the question ?Did you see Echology more as anart piece or as a game or neither?? Responses are summarized in AppendixB.3, Table B.4. Ten participants felt it was more a game, 4, more art, 3 re-sponded neither and 2 felt it was a mixture of both. Many participants whofelt Echology was more a game said so with some apprehension. It seemedlike a game because of the Interaction Table buttons and screen and becauseyou could change things. Still, it could not be a game completely as it lackeddirections and goals. A reason for considering Echology as an artwork wasthat it was ?artistically laid out and involved the senses? (Ambrose), butparticipants could not say it was art with immediate confidence, possiblybecause they could not relate it to art that they were familiar with, thisbeing something that you passively look or listen to (Barbara), to ?observeit and then make an analysis? (Cory), and that has ?meaningful things thatcan be expressed or displayed? (Cassandra).The strong leaning towards Echology being a game indicates that par-ticipants likely had expectations of game-like qualities for their experience.Upon entering the room, they knew they would be able to play and inter-act, but they probably also expected challenges, rules to figure out, goals,or measures of performance like a score. As an artwork, Echology lacked inthese thus some participants felt confusion or that something was missing.As summarized in Section, many enjoyed the atmosphere and sound-scape ? elements that could be taken in passively. They considered these the?artistic? elements of Echology. But to fully enjoy the expressive meaningof Echology, participants would have to transcend notions of traditional artwhere expression is through a fixed art object.96Chapter 5. Studying Participant Experience of EchologyOther influences to behaviour tendencies were the study settings and thefact that participants were subjects in a study, as well as biases rooted in toparticipant backgrounds. This is discussed further in Section 5.5?s review ofthe study methodology.5.4 Interpretation of Study ResultsThe results discussed in Section 5.2 reveal facets of participant enjoyment inthe Echology experience as well as areas where there was a lack of participantunderstanding. These results can inform future versions of Echology but themanner in which they do depends on how they are interpreted.This section highlights the key points taken from the results for interpre-tation. They are not immediately labeled ?problems? as whether or not itis a problem depends on the interpretation. Each point is interpreted fromboth the traditional HCI viewpoint and the interactive artist?s viewpointand suggestions for changes or considerations in future versions of Echologyare made.Result: (Section 5.3.2) Not a single participant made a clear relationshipbetween the types of sound heard and movement in different areas of thewebcam feed. Furthermore, some were confused by the use of four differentcoloured graphical balls to represent the sound signals. Many could distin-guish that there were different ?types? of sound, but were not able to relateit to anything they saw.Suggestion: From an HCI viewpoint, indication of the webcam feed imagelayers, as well as of the relationships between layers, associated sound, andcoloured ball representation should be provided. Indication could be given,for example, by a periodic graphical display of layer divisions with text labelsand colour coding. From an artist?s viewpoint, it is suggested that some levelof ambiguity is maintained. Participant understanding of the relationshipsbetween layers, sounds and balls will not necessarily promote connectednesswith the Belugas and the sense of wonder and curiosity intended for. If97Chapter 5. Studying Participant Experience of Echologythese relationships are clear participants may lack space in which they canquestion and interpret the actions of the Belugas (in the artwork). Still,participants should be given some cue that the Belugas are controlling partof the installation. This suggests tuning the video processing and soundcontrol to make the relationship between Beluga movement and sound moreobvious.Result: (Section 5.3.3) Six subjects expressed that they did not pay at-tention to the Beluga video because they were focused on the sound andgraphics. The Belugas were often referred to as being ?in the background?and as a result, connection with the Belugas became only a ?background?aspect of Echology.Suggestion: From both viewpoints, the design of the graphical feedbackoverlay on the Beluga webcam content should be reconsidered. The Belugavideo could be brought into the foreground by not overlaying the graphics,or by highlighting and bringing attention to the webcam feed in some way.It may be tempting to use a higher quality video feed but from an artist?sviewpoint, low video quality is more strongly associated with live video thusit is suggested that even if higher quality were available, it should still notbe used.Result: (Sections 5.3.3 and 5.3.5) Knowledge that the Belugas were livehad a great effect on a participant?s interest in and connection to the Belugasand made the participant more accepting of the limited amount of controlthey had on the sound.Suggestion: From an HCI viewpoint, cues that the Beluga webcam feedis live should be provided. Subtle cues could be provided by, for exam-ple, showing the updated date and time generated by the webcam feed, ordisplaying information specific to the Belugas at the Vancouver Aquariumsuch as their given names: Imaq, Kavna, Aurora, and Qila. From the artist?sviewpoint, while agreeing with the HCI viewpoint, it is still suggested that98Chapter 5. Studying Participant Experience of Echologysome care be taken in the manner in which cues are provided. Explicitcues like the ones suggested above might promote the view of Echology asan educational interactive exhibit and increase participant expectation ofinformation provided. Subtler cues could be provided by, for example, re-designing the Interaction Table or installation space to be a replica of thespace where Belugas are watched at the Vancouver aquarium.Result: (Section 5.3.5) Echology did not give enough control to fully en-gage a participant with a tendency for play. It also did not support partici-pants seeking purpose in interaction.Suggestion: More engaging control of sound should be provided. Controlover variables of sound other than its motion could be given to the par-ticipant, allowing for a richer space of play and expression. For example,participants could have some control over the type of sound, or generationof sound that is then modified by the Beluga whales. Or they could have fullcontrol over a set of sounds, while the Belugas would control another set.Play between participant and Beluga would occur in the mixture of thesesounds. From an HCI viewpoint, purpose in interaction could be designed inthe form of connections with the Belugas: through interaction participantscould be generating virtual food for the whales or, as reward for reachingsome end goal, participants could control a virtual character swimming onscreen with the Belugas. From an artist?s viewpoint, providing a specificpurpose to Echology could limit interaction paths and interpretations toones directed towards achieving the goal, thus reducing participant creativ-ity and reflection.5.5 Discussion of Study Methodology5.5.1 Participant Behaviour as a Study SubjectAs discussed in Section 3.4, there is general acknowledgment that experimentsettings have an effect on participant behaviour. In this study it was found99Chapter 5. Studying Participant Experience of Echologythat the root of much study setting-influenced behaviour was a participant?sdesire to ?perform? well. Even though it was stressed that there were noexpectations or right and wrong answers, a number of participants still maderemarks indicating some sense of obligation to do well. Byron, receiving noinstructions, suggested that instructions would have been good so that hecould ?perform better?. Charles read the instructions he received 3 or 4times, telling me that he was trying to find ?something like a clue aboutyour questions?.Participant behaviour was further influenced by belief that the studywould be a usability or user study. Many participants had previously par-ticipated in one or more user studies conducted by various HCI practitionerswho use the same Internet-based recruiting system. These participants hadexpectations of how user studies were conducted based on study designscommonly used. As Arthur remarked, ?. . .all of my studies, they are all likethat. Like, one person makes a program and I have to like, play with it orlike, see which one does better or something. Lot?s of them have to do withgraph, plots, and like, also personality stuff.?Finally, it should be acknowledge that participant background likelyplayed a strong role in resulting behaviour. As mentioned in Section 5.2.2,the majority of participants were students in the engineering and computerscience departments. Some of the behaviours discussed in this section, suchas focusing on the functional aspects of an artwork, could be explained forby the predisposition of these participants to be interested in technologyand how things work.Considering the above influences, prominent types of behaviour that wereobserved are as follows:Participants focused on functional aspects of the artwork over itscontent: Many study subjects indicated their main goal being to figureout how things worked. The advantage of this was that they were likely toattempt exploring all aspects of interaction and it could be easy to observewhat could not be figured out. The disadvantage was that participants spentless time reflecting on artistic qualities of the work.100Chapter 5. Studying Participant Experience of EchologyParticipants were prepared to express what they did and did notunderstand, as well as their personal level of comfort: This be-haviour was advantageous for learning how people approached and tried tofigure the artwork out but on the other hand, they were not prepared toexpress opinion on or critique Echology as an artwork.Participants expected a task, clear instructions for that task, andpossibly some measure of performance: Some participants expressedconfusion about what they were supposed to do with Echology once left inthe room and not about Echology itself. Anne said she ?should?ve checkedwhat Echology was before? coming so that she could have a better sense ofwhat she was supposed to do. Used to user study designs that lead themthrough tasks step by step, they felt uncomfortable having to self-direct theirexperience. This made it somewhat difficult to distinguish a participant?sconfusion or discomfort as caused by the artwork design, or by what wasexpected from them in the study.Participants were more likely to work through a breakdown: Ifsomething appeared to not be working properly, reasons attributed were: theart object ? in which case participants were confident that the experimentercould fix the problem, something they did wrong ? in which case they mayhave felt obligation to work through the breakdown, and as part of the studydesign ? in which case they may have thought their behaviour in the face of abreakdown was under observation. The latter was a tactic used in the Fa?adestudy. However, care must be taken not to consider a participant?s ability towork through a breakthrough as overly positive because in a regular setting,participants do not have the same performance obligations and toleranceand would likely quickly end their experience.Participants in groups maintained individual agendas: As men-tioned in Section, collaborative behaviour observed in the laboratorystudy was remarkably different from that observed at NIME. NIME was apublic space and a sociable event. The laboratory study was closed-off and101Chapter 5. Studying Participant Experience of Echologythe participants, taking on the role of study subjects, were focused on theirown performance. This suggests that the laboratory-based method was notsuitable for studying informal collaboration and socialization around theartwork. Though if the study had been designed in a manner assigning eachparticipant a role to play within a group agenda, it is possible that socialdynamics could be better observed.5.5.2 Orientation Media ReviewTo review, orientation media was used to distinguish the effects thatunderstanding how the system works as well as knowledge of the Belugawebcam feed being live as opposed to pre-recorded had on a person?s overallexperience. Though after reviewing the data, there were no distinguishableeffects across groups with and without either form of orientation media.In all session types participants had varying levels of enjoyment and observedconnection with the Belugas. It is considered that this is because the extentthat a participant will be receptive to the orientation media is dependenton that participant?s tendencies to want to figure things out and to expectinstructions and goals.The orientation media did help reveal elements of the system for whichparticipants without instructions could eventually formulate mental modelsof and elements that were confusing regardless of whether or not a partic-ipant received prior instructions (such as the relationship between Belugamotion and sound).5.5.3 Benefit of In-depth Data CollectionAdding to benefits exemplified by the case studies in Chapter 3, the Echologystudy demonstrates how a protocol for categorization of data to analyzeexperience can be developed (Section 5.2.5). This protocol is an expressionof the intended aesthetic pursuits in Echology created by making repeatedapproaches to the rich data that was collected. It would not have beenpossible to develop the protocol from the informal observations made duringNIME.102Chapter 5. Studying Participant Experience of EchologyThese protocols are important for framing and justifying interpretationsmade of study results. They are similar to the heuristics that inform userstudies but are often arrived at only after study results are reviewed. Excep-tions are cases like the Iamascope and Fa?ade studies that use pre-existingprotocols.Another benefit was achieved through the tendency for participants asstudy subjects to make a large effort to figure the system out. In-depth datacollection resulted in multiple accounts of approaches taken by participantsto understand the artwork and the metaphors and mental models they usedin the process.5.6 SummaryThis chapter began with a summary of informal observations of participantinteraction with Echology during its first public presentation at a confer-ence. To address questions remaining after this event a laboratory-basedstudy was designed and executed as detailed in Section 5.2. The results ofthis study were summarized and interpreted in Section 5.3 and Section 5.4discussed the study methodology and resulting experience itself. Observa-tions of changes in participant behaviour due to their roles as subjects in astudy and implications these changes have on using formal methodologiesfor studying participant experience are discussed. The orientation mediais shown to be successful in revealing elements of the artwork that, givenexplicit instructions or not, still cause confusion among participants.103Chapter 6ConclusionIn this chapter the problems introduced in Chapter 1 are restated and con-tributions made in this thesis work towards addressing these problems aredescribed. The chapter concludes with a discussion of future work suggestedfor the development of methodology for studying participant experience ofinteractive art.6.1 Thesis ProblemsChapter 1 stated three problems to be addressed in this thesis:1. We do not fully understand the aesthetics of interactive art-works and accordingly, the purposes for conducting partic-ipant experience studies. HCI practitioners may have difficultiesaccepting the qualitative and interpretive nature of an artist?s studymethods. This is likely due to limited understanding of the artisticmotivations behind conducting studies. Yet for collaborations betweenHCI and the arts to occur, a common language and understanding ofartistic intentions and visions needs to be established [26].2. We do not fully understanding how laboratory-based userstudy methodology can be beneficial for the development ofinteractive art experiences. Interactive artists may believe thatlaboratory-based studies take a person away from the natural contextof interaction and as a result, they tend to use informal methods forstudying participant experience. While there are cases of more formalmethodology used ([54], [29], [59], [97]), there is still room for morecommentary on the benefits and implications of using these methods.104Chapter 6. Conclusion3. We have no investigation of methods to compare experiencesof participants with varying levels of understanding of art-work elements in a controlled laboratory setting. Interactiveartworks require some exploration on part of the participant and theartifacts created by the artist should support this exploration. If a par-ticipant is unable to develop understanding of functional and expres-sive elements of an artwork, their experience may be limited. Thus,key concerns of the artist are: does the artwork properly support de-velopment of participant understanding? How much is the experienceactually based on understanding development or, ?figuring the art-work out?? And how does specific understanding of different artworkfunctional and expressive elements affect experience? These questionssuggest the need for methodology that would allow artists to selec-tively influence participant understanding of specific artwork elementsin a controlled setting.6.2 Thesis ContributionsThis thesis can be summarized by three contributions made by this work:A relationship between the Interactive Arts and HumanComputer Interaction (HCI) is identified.Art theory is reviewed, demonstrating the roles of art object and participantand the artist?s role as designer of art objects that can support participantsin their role. Purposes for participant experience studies in the artist?s pro-cess are derived from these roles. Knowledge acquired through studies isused to refine techniques and interaction aesthetics. In the iterative activ-ity of design, study, and re-design, an art object learns and evolves in anindirect sense through interaction with participants. Finally, study activitydocuments interaction thus is itself part of an artwork?s content.The artist?s process is compared with the HCI practitioner?s user-centereddesign process to see how HCI user study methods may be appropriated by105Chapter 6. Conclusionartists to study participant experience. Having established deeper under-standing for the purpose of participant experience studies, the differences ingoals and desired outcomes between artists and HCI practitioners becomemore clear.Echology: an interactive artwork and new interface for musicalexpression is created.Echology is an interactive installation piece for participants to play withspatialized sound in collaboration with fellow participants and with the Bel-uga whales at the Vancouver Aquarium. Observations of interaction withEchology during a public presentation confirm that its design is successful inbeing an approachable artwork that supports participant collaboration anda social space around the Interaction Table. Further studies in the labora-tory confirm that, aside from some ambiguity in the relationship betweenBeluga whale motion and sound, it effectively allows participants to hear,visualize and control sound spatialization.Further understanding of HCI user study methods applied tostudying participant experience of interactive artworks isdeveloped.Building on research of more formal, laboratory-based methodology used tostudy participant experience, the use of orientation media to selectivelyinfluence a participant?s understanding of artwork elements in a controlledstudy setting is proposed, practiced, and reviewed. Part of the interactiveart experience is figuring out how one may interact. This game-like qualitycan add to enjoyment and engagement. At the same time, if the challengeof figuring the system out is too great, if the participant is left confused andunable to formulate interpretations, the experience can be un-enjoyable.Using orientation media in the form of instructions for the interactiveartwork, it was possible to compare two groups of participant understandingprocesses. In one group participants approached the artwork with no priorknowledge of how it works and what to do. In the other group participants106Chapter 6. Conclusionwere given prior knowledge of interaction: how they can act and how andwhy the system will react. What emerged from interviews with participantswas knowledge of:1. The elements of the system for which participants without instructionscould eventually formulate mental models.2. Which elements were confusing regardless of whether or not a partic-ipant received prior instructions.This work also investigates behaviour influenced by a participant?s roleas a subject in an experimental setting and considers its implications forstudy design and interpretation. In Section 3.3 it was noted that laboratory-based studies could not be conducted without some change in participantbehaviour due to the study settings but at that time, the possible changeshad not been well observed. Five behaviours that are likely results of thestudy settings are observed in the study of Echology and the implicationsfor results interpretation and future study design are noted.6.3 Laboratory-Based Methods in the ArtisticProcessThis work presents one case of participant experience studies of a specificartwork. The study design, results, and interpretations are specific to goalsof Echology and to the artists? pursuits whereas other forms of study designmay be more appropriate to other artworks. It is hoped that the takeawayof this thesis is that studies conducted outside of an artwork?s ?natural?context, in controlled settings, with methods of in-depth data collection,and with participants brought in as study subjects, are in general seen as animportant activity in the development of interactive artworks. Also, whilethis work is motivated by a focus on more formal study methods, it is notintended for these methods to be considered ?better? than informal ones.Rather, it is desirable that both formal and informal methods are seen as107Chapter 6. Conclusionuseful for studying different aspects of participant experience and at differentperiods of design iteration.In the case of the Echology studies, the informal study presented in Sec-tion 5.1 allowed for observations of participant socialization, collaborationand natural behaviour in the intended artwork context. It also provided theopportunity to receive valuable feedback concerning participant reflectionson the artistic expression of the artwork. This study was useful for under-standing how participants would approach and interact with Echology butless useful for confirming their levels of understanding the piece functionallyand expressively. On the other hand, the laboratory-based study detailedin the rest of Chapter 5 was useful for collecting rich data on participantunderstanding of interaction, the processes used to obtain this understand,as well as metaphors and descriptions that were assigned to the artwork.It is from this study that recommendations for Echology redesign becameapparent, more so than after the informal observations. So to return to thedesign processes discussed in Section 2.4, similarly to different types of userstudies being suitable for different phases of design iteration, it is recom-mended that more formal methods, as well as informal methods be usedduring an artwork?s lifetime.6.4 Future WorkIt is foreseeable that as computing technologies and practices continue torapidly change, artists will respond with work commenting on the effectsthat these new technologies have on daily life. They will also continueto engage the audience in active participation, challenging them to inter-act with art objects that push the boundaries in technology and interfaceconcepts. Thus, studying interaction will remain an important activity forunderstanding the relationships between participants and the art, for feed-back in an iterative art object design process, and to record part of theartwork?s content ? the interaction. Two avenues for future work in thedevelopment of methodologies for studying participant experience are dis-cussed in this section: a framework to recommend study methodology based108Chapter 6. Conclusionon artwork characteristics and exploration into the aesthetics of study resultpresentation.6.4.1 Study Methods Based on Artwork CharacteristicsThis thesis did not provide much attention to the suitability of certain studymethodologies as related to specific characteristics of interactive artworks.Still, in the experience studying Echology some of the initially planned meth-ods did not work out, likely because of characteristics of Echology as anartwork.For example, behaviour between paired participants was remarkably dif-ferent from that observed between participants at the NIME installation(see Sections 5.3.4 and 5.5.1). It was concluded that laboratory-based stud-ies involving participants who are aware that they are study subjects maynot be suitable for studying informal collaboration and socialization aroundan artwork. Another example is that initially, the study was designed to usethe video-cued recall method to support participant discussion of their expe-rience. Though during pilot studies, participants did not refer to the video.On reflection, this may have been because all of the interactive, visual, andauditory elements of Echology that may appear in the video are fairly sim-plistic and repetitive, offering little in the way of experience narrative to cuememories.These experiences point towards a need to better understand what studymethods and designs are suitable for which types or characteristics of inter-active artwork. This work could involve using an existing artwork classifi-cation system such as Bell?s [14]) and making recommendations for studydesign based on characteristics of an artwork such as the level at which aparticipant physically interacts and the extent to which the artwork hasgame-like qualities.109Chapter 6. Conclusion6.4.2 Aesthetics of Study Result PresentationIn the reviews of participant experience studies of Chapter 2 and 3, themanners in which study results are presented were discussed. Although notexamined in detail, this is actually a very important topic in the consider-ation of interactive art aesthetics. Considering the purpose for participantexperience studies as documenting interaction in an art system and that thisdocumentation becomes part of the artwork?s content, the manner in whichresults are presented ultimately affects interpretations of the artwork. Thus,it is recommended that future research explore the characteristics of resultsand the aesthetics of their presentation.6.5 ConclusionIn this thesis purposes for studying participant experience of interactiveartworks in the development of interactive art techniques and aesthetics,as well as in the creation of a final artwork were proposed. Three casesusing formal methods appropriated from HCI highlighted the benefits ofcontrolled settings and in-depth data collection. Building from these cases,a method using orientation media was proposed and practiced in a studyof Echology.The study design presented is just one such design that can be used toevaluate and deepen understanding of an artwork. Through its processes,a particular experience was being created for people, an experience withinwhich they would experience the artwork. The study was not only an activitythat allowed development in understanding of interaction with Echology,but was also another creation allowing for expression between artist andparticipants, another part of the artwork that is Echology.110List of ArtworksAscott, R. (1959). ?Change Painting?.Blast Theory. (2001 ? ). ?Can You See Me Now??.Fels, S. and Mase, K. (1997 ? ). ?Iamascope?.Iwai, T. (1999). ?Composition on the Table?.Krueger, M. (1970). ?Videoplace?.Mateas, M. and Stern, A. (2005 ? 2006). ?Fa?ade?.Olsson, K. and Kawashima, T. (2005). ?The king has. . . ?.Sengers, P., Liesendahl, R., Magar, W., and Seibert, C. (2003). ?The Influ-encing Machine?.Sommerer, C. and Mignonneau, L. (1994 ? 1997). ?A-Volve?.Tzara, T. ?How to make a Dadaist poem?.Weinbren, G. (1982 ? 1985). ?The Erl King?.111Bibliography[1] Creativity and cognition studios website.[2] Cycling 74 max/msp product site.[3] Glossary of terms from education @ turnerhouse., Accessed Jan-uary 2008.[4] Interface ecology lab website.[5] Phidgets Inc.[6] Vancouver aquarium beluga cam.[7] C. Abras, D. Maloney-Krichmar, and J. Preece. User-centered design.In W. Bainbridge, editor, Encyclopedia of Human-Computer Interac-tion. Thousand Oaks: Sage Publications, (in press).[8] P. D. Adamczyk, K. Hamilton, M. B. Twidale, and B. P. Bailey. HCIand new media arts: methodology and evaluation. In CHI ?07: CHI?07 extended abstracts on Human factors in computing systems, pages2813?2816, 2007.[9] L. Alben. Quality of experience: defining the criteria for effectiveinteraction design. interactions, 3(3):11?15, 1996.[10] E. Aley, T. Cooper, R. Graeber, A. Kerne, K. Overby, and Z. O.Toups. Censor chair: exploring censorship and social presence through112Bibliographypsychophysiological sensing. In MULTIMEDIA ?05: Proceedings ofthe 13th annual ACM international conference on Multimedia, pages922?929, 2005.[11] R. Ascott. Behaviourist art and the cybernetic vision (part II). Cy-bernetica, 10(1):25?56, 1967.[12] R. Ascott. The cybernetic stance: My process and purpose. Leonardo,1(2):105?112, 1968.[13] R. Ascott. Telematic Embrace: Visionary Theories of Art, Technology,and Consciousness. University of California Press, Berkeley, CA, USA,2003.[14] S. Bell. Participatory Art and Computers: Identifying, Analysing andComposing the Characteristics of Works of Participatory Art that UseComputer Technology. PhD thesis, Loughborough University, 1991.[15] S. Benford, A. Crabtree, M. Flintham, A. Drozd, R. Anastasi, M. Pax-ton, N. Tandavanitj, M. Adams, and J. Row-Farr. Can you see menow? ACM Trans. Comput.-Hum. Interact., 13(1):100?133, 2006.[16] D. Benyon. The role of task analysis in systems design. Interact.Comput., 4(1):102?123, 1992.[17] Z. Bilda, E. Edmonds, and D. Turnbull. Interactive experience in apublic context. In C&C ?07: Proceedings of the 6th ACM SIGCHIconference on Creativity & cognition, pages 243?244, 2007.[18] D. Birchfield, K. Phillips, A. Kidan?, and D. Lorig. Interactive publicsound art: a case study. In NIME ?06: Proceedings of the 2006 con-ference on New interfaces for musical expression, pages 43?48, 2006.[19] C. Bishop. Installation Art: a critical history. Tate Publishing, Lon-don, 2005.[20] T. Blaine and S. S. Fels. Collaborative musical experiences for novices.Journal of New Music Research, 32(4):411?428, 2003.113Bibliography[21] M. A. Blythe, K. Overbeeke, A. F. Monk, and P. C. Wright, editors.Funology: from usability to enjoyment. Kluwer Academic Publishers,Norwell, MA, USA, 2004.[22] K. Boehner, R. DePaula, P. Dourish, and P. Sengers. Affect: frominformation to interaction. In CC ?05: Proceedings of the 4th decennialconference on Critical computing, pages 59?68, 2005.[23] J. E. Boyd, G. Hushlak, and C. J. Jacob. Swarmart: interactive artfrom swarm intelligence. In MULTIMEDIA ?04: Proceedings of the12th annual ACM international conference on Multimedia, pages 628?635, 2004.[24] P. B?rger. Theory of the Avant-Garde. University of Minnesota Press,Minneapolis, MI, USA, 1984.[25] J. Burnham. The aesthetics of intelligent systems. In E. F. Fry, editor,On the Future of Art, pages 95?122. The Viking Press, New York, NY,USA, 1970.[26] L. Candy and E. Edmonds. Modeling co-creativity in art and tech-nology. In C&C ?02: Proceedings of the 4th conference on Creativity& cognition, pages 134?141, 2002.[27] S. Cornock. Art and interaction. 1977 monograph from LeicesterPolytechnic, UK.[28] S. Cornock and E. Edmonds. The creative process where the artist isamplified or superseded by the computer. Leonardo, 6(1):11?16, 1973.[29] B. Costello, L. Muller, S. Amitani, and E. Edmonds. Understandingthe experience of interactive art: Iamascope in beta space. In IE2005:Proceedings of the second Australasian conference on Interactive en-tertainment, pages 49?56, 2005.[30] G. Coulter-Smith. Deconstructing Installation Art: Fine Art and Me-dia Art, 1986-2006., 2006. On-line book.114Bibliography[31] M. Csikszentmihalyi. Flow: The Psychology of Optimal Experience.Harper Perennial, New York, NY, USA, 1991.[32] G. D?Arcangelo. Creating contexts of creativity: musical compositionwith modular components. In NIME ?01: Proceedings of the 2001conference on New interfaces for musical expression, pages 1?4, 2001.[33] S. B. Davis. Its art, but is it hci? ? testing the boundaries. In ACMSIGCHI?05 Workshop on Graduate Education in Human-ComputerInteraction, 2005. Invited paper for workshop on education.[34] R. Debatty. Interview with Milk and Tales. we make money not art:,Oct 2007. Blog entry.[35] P. Desmet and E. Dijkhuis. A wheelchair can be fun: a case of emotion-driven design. In DPPI ?03: Proceedings of the 2003 internationalconference on Designing pleasurable products and interfaces, pages 22?27, 2003.[36] M. Deutscher, R. Hoskinson, S. Takahashi, and S. Fels. Echology. InNIME ?05: Proceedings of the 2005 conference on New Interfaces forMusical Expression, page 274, May 2005.[37] M. Deutscher, R. Hoskinson, S. Takashashi, and S. Fels. Echology:an interactive spatial sound and video artwork. In MULTIMEDIA?05: Proceedings of the 13th annual ACM international conference onMultimedia, pages 937?945, 2005.[38] T. Dove. Theater without actors: Immersion and response in instal-lation. Leonardo, 27(4):281?287, 1994.[39] J. S. Dumas and J. C. Redish. A Practical Guide to Usability Testing.Ablex Pub. Corp., 1993.[40] E. Edmonds, L. Muller, and M. Connell. On creative engagement.Visual Communication, 5(3):307?322, 2006.115Bibliography[41] E. Edmonds, G. Turner, and L. Candy. Approaches to interactive artsystems. In GRAPHITE ?04: Proceedings of the 2nd internationalconference on Computer graphics and interactive techniques in Aus-tralasia and South East Asia, pages 113?117, 2004.[42] K. A. Ericsson and H. A. Simon. Protocol Analysis: Verbal Reports asData. The MIT Press, Cambridge, MA, USA, 1993.[43] S. S. Fels. Intimacy and embodiment: Implications for art and tech-nology. In Proceedings of the ACM Conference on Multimedia, pages13?16, 2000.[44] S. S. Fels and K. Mase. Iamascope: A graphical musical instrument.Computers and Graphics, 2(23):277?286, 1999.[45] J. Fogarty, J. Forlizzi, and S. E. Hudson. Aesthetic information col-lages: generating decorative displays that contain information. InUIST ?01: Proceedings of the 14th annual ACM symposium on Userinterface software and technology, pages 141?150, 2001.[46] W. W. Gaver, J. Beaver, and S. Benford. Ambiguity as a resource fordesign. In CHI ?03: Proceedings of the SIGCHI conference on Humanfactors in computing systems, pages 233?240, 2003.[47] W. W. Gaver, J. Bowers, A. Boucher, H. Gellerson, S. Pennington,A. Schmidt, A. Steed, N. Villars, and B. Walker. The drift table: de-signing for ludic engagement. In CHI ?04: CHI ?04 extended abstractson Human factors in computing systems, pages 885?900, 2004.[48] B. G. Glaser and A. L. Strauss. Discovery of Grounded Theory: Strate-gies for Qualitative Research. Aldine, 1967.[49] E. Goffman. Behavior in Public Places: Notes on the Social Organi-zation of Gatherings. Glencoe: The Free Press, 1966.[50] B. C. E. Graham. A Study of Audience Relationships with InteractiveComputer-Based Visual Artworks in Gallery Settings, through Obser-116Bibliographyvation, Art Practice, and Curation. PhD thesis, The University ofSunderland, 1997.[51] M. Gurevich. Jamspace: designing a collaborative networked musicspace for novices. In NIME ?06: Proceedings of the 2006 conferenceon New interfaces for musical expression, pages 118?123, 2006.[52] D. Heller. Aesthetics and interaction design: some preliminarythoughts. interactions, 12(5):48?50, 2005.[53] K. H??k. Evaluating the utility and usability of an adaptive hyperme-dia system. In IUI ?97: Proceedings of the 2nd international conferenceon Intelligent user interfaces, pages 179?186, 1997.[54] K. H??k, P. Sengers, and G. Andersson. Sense and sensibility: eval-uation and interactive art. In CHI ?03: Proceedings of the SIGCHIconference on Human factors in computing systems, pages 241?248,2003.[55] J. Huh, M. S. Ackerman, and R. Douglas. The use of aesthetics in hcisystems. In CHI ?07: CHI ?07 extended abstracts on Human factorsin computing systems, pages 2441?2446, 2007.[56] E. Huhtamo. Trouble at the interface, or the identity crisis of interac-tive art. FRAMEWORK: The Finnish Art Review, 2(1), 2004.[57] T. Iwai. Composition on the table. In SIGGRAPH ?99: ACM SIG-GRAPH 99 Electronic art and animation catalog, page 10, 1999.[58] J. F. Kelley. An iterative design methodology for user-friendly natu-ral language office information applications. ACM Trans. Inf. Syst.,2(1):26?41, 1984.[59] R. L. Knickmeyer and M. Mateas. Preliminary evaluation of the in-teractive drama facade. In CHI ?05: CHI ?05 extended abstracts onHuman factors in computing systems, pages 1549?1552, 2005.117Bibliography[60] M. W. Krueger. Artificial Reality II. Addison-Wesley, Reading, MA,USA, 1991.[61] M. W. Krueger, T. Gionfriddo, and K. Hinrichsen. Videoplace ? anartificial reality. In CHI ?85: Proceedings of the SIGCHI conferenceon Human factors in computing systems, pages 35?40, 1985.[62] D. V. Lehn, C. Heath, and J. Hindmarsh. Exhibiting interaction: Con-duct and collaboration in museums and galleries. Symbolic Interaction,24:189?216, 2001.[63] D. Livingstone and E. Miranda. Orb3: adaptive interface design forreal time sound synthesis & diffusion within socially mediated spaces.In NIME ?05: Proceedings of the 2005 conference on New interfacesfor musical expression, pages 65?69, 2004.[64] D. M. M. Lopes. The ontology of interactive art. Journal of AestheticEducation, 35:65?81, 2001.[65] F. J. Malina. Differences entre la science et l?art: Quelques reflexions.Leonardo, 1(4):449?455, 1968.[66] T. W. Malone. Heuristics for designing enjoyable user interfaces:lessons from computer games. pages 1?12, 1984.[67] M. Mateas. Expressive AI. Leonardo, 34(2):147?153, 2001.[68] E. Mayo. The human problems of an industrial civilization. MacMillan,New York, NY, USA, 1933.[69] L. Muller and E. Edmonds. Living laboratories: Making and curatinginteractive art: Copyright restrictions prevent acm from providing thefull text for this work. In SIGGRAPH ?06: ACM SIGGRAPH 2006Art gallery, page 160, 2006.[70] Q. Nguyen, S. Novakowski, J. E. Boyd, C. Jacob, and G. Hushlak.Motion swarms: video interaction for art in complex environments.118BibliographyIn MULTIMEDIA ?06: Proceedings of the 14th annual ACM interna-tional conference on Multimedia, pages 461?469, 2006.[71] J. Nielsen. Heuristic evaluation. In J. Nielsen and R. L. Mack, editors,Usability Inspection Methods. John Wiley & Sons, New York, NY,USA, 1994.[72] D. Norman. The Design of Everyday Things. Basic Books, New York,NY, USA, 2002.[73] D. Norman. Emotional Design: Why We Love (Or Hate) EverydayThings. Basic Books, New York, NY, USA, 2003.[74] D. A. Norman and S. W. Draper. User Centered System Design: NewPerspectives on Human-Computer Interaction. L. Erlbaum Associates,1986.[75] K. Olsson and T. Kawashima. The king has... In MULTIMEDIA?05: Proceedings of the 13th annual ACM international conference onMultimedia, pages 1075?1076, 2005.[76] H. M. Parsons. What Happened at Hawthorne?: New evidence sug-gests the Hawthorne effect resulted from operant reinforcement con-tingencies. Science, 183(4128):922?932, 1974.[77] L. Pellarin, N. B?ttcher, J. M. Olsen, O. Gregersen, S. Sarafin, andM. Guglielmi. Connecting strangers at a train station. In NIME?05: Proceedings of the 2005 conference on New interfaces for musicalexpression, pages 152?155, 2004.[78] S. Penny. From a to d and back again: The emerging aesthetics ofinteractive art. Leonardo Electronic Almanac, 4(4), 1996.[79] M. G. Petersen, O. S. Iversen, P. G. Krogh, and M. Ludvigsen. Aes-thetic interaction: a pragmatist?s aesthetics of interactive systems. InDIS ?04: Proceedings of the 5th conference on Designing interactivesystems, pages 269?276, 2004.119Bibliography[80] R. W. Picard. Affective computing. MIT Press, Cambridge, MA, USA,1997.[81] F. Popper. From Technological to Virtual Art. The MIT Press, Cam-bridge, MA, USA, 2005.[82] J. Preece, Y. Rogers, and H. Sharp. Interaction Design. John Wiley& Sons, Inc., New York, NY, USA, 2002.[83] V. Pulkki. Virtual source positioning using vector base amplitudepanning. Journal of the Audio Engineering Society, 45(6):456?466,1997.[84] D. Rokeby. Transforming mirrors: Subjectivity and control in inter-active media. In S. Penny, editor, Critical Issues in Electronic Media.State University of New York Press, Albany, NY, USA, 1995.[85] D. Rokeby. The construction of experience: interface as content. InDigital illusion: entertaining the future with high technology, pages27?47. ACM Press/Addison-Wesley Publishing Co., New York, NY,USA, 1998.[86] R. Rosenthal and K. L. Fode. The problem of experimenter outcome-bias. In D. P. Ray, editor, Series Research in Social Psychology. Na-tional Institute of Social and Behavioral Science, Washington, DC,USA, 1961.[87] E. P. Rozanski and A. R. Haake. The many facets of hci. In CITC4?03: Proceedings of the 4th conference on Information technology cur-riculum, pages 180?185, 2003.[88] S. D. Scott, M. Sheelagh, T. Carpendale, and K. M. Inkpen. Territori-ality in collaborative tabletop workspaces. In CSCW ?04: Proceedingsof the 2004 ACM conference on Computer supported cooperative work,pages 294?303, 2004.120Bibliography[89] P. Sengers and B. Gaver. Staying open to interpretation: engagingmultiple meanings in design and evaluation. In DIS ?06: Proceedingsof the 6th conference on Designing Interactive systems, pages 99?108,2006.[90] B. Shneiderman. Designing the User Interface: Strategies for Effec-tive Human-Computer Interaction. Addison-Wesley Publishing Co,Reading, MA, USA, 1987.[91] C. Sommerer and L. Mignonneau. A-volve: a real-time interactiveenvironment. In SIGGRAPH ?94: ACM SIGGRAPH 94 Visual Pro-ceedings, pages 172?173, 1994.[92] L. A. Suchman. Plans and Situated Actions: The Problem of Human-machine Communication. Cambridge University Press, Cambridge,1987.[93] T. Szakal, C. Groenegress, W. Strauss, and P. Peranovic. Phonicfrequencies ? shaping networked realities. In Proceedings of MelbourneDAC, the 5th International Digital Arts and Culture Conference, 2003.[94] J. C. Tang. Findings from observational studies of collaborative work.In Computer-supported cooperative work and groupware, pages 11?28.Academic Press Ltd., London, UK, UK, 1991.[95] R. B. Taylor. Human Territorial Functioning: An Empirical Evolu-tionary Perspective on Individual and Small Group Territorial Cog-nitions, Behaviors, and Consequences. Cambridge University Press,New York, NY, USA, 1988.[96] P. L. Tyack and E. H. Miller. Vocal anatomy, acoustic communicationand echolocation. In A. R. Hoelzel, editor, Marine Mammal Biol-ogy: An Evolutionary Approach, pages 142?184. Blackwell Publishing,2002.[97] A. Webb, A. Kerne, E. Koh, P. Joshi, Y. Park, and R. Graeber. Chore-ographic buttons: promoting social interaction through human move-121Bibliographyment and clear affordances. In MULTIMEDIA ?06: Proceedings ofthe 14th annual ACM international conference on Multimedia, pages451?460, 2006.[98] G. Weinbren. In the ocean of streams of story. Millennium FilmJournal, (28), 1995.[99] M. Weiser and J. S. Brown. Designing calm technology. PowerGridJournal, 1, 1996.[100] D. Wessel and M. Wright. Problems and prospects for intimate musicalcontrol of computers. Computer Music Journal, 26:11?22, 2002.[101] K. D. D. Willis. User authorship and creativity within interactivity.In MULTIMEDIA ?06: Proceedings of the 14th annual ACM interna-tional conference on Multimedia, pages 731?735, 2006.[102] T. Winograd. From computing machinery to interaction design. InP. Denning and R. Metcalfe, editors, Beyond Calculation: The NextFifty Years of Computing, pages 149?162. Springer-Verlag, 1997.122Appendix AStudy MaterialA.1 Participant Recruitment NoticeUser Interface and Experience Study of ?Echology?:an Interactive Spatial Sound and Video ArtworkI am conducting a study of the interactive sound and video installation,Echology, for my graduate thesis. The purpose of this study is to investigatenew methods of evaluating the experience people have with artworks of asimilar nature.***If you are over the age of 19, and you have not already seen Echology,you are invited to this special presentation***The study will take a total time of 60 minutes and your experience at theEchology installation will be video-taped. Following this, you will be givena questionnaire and led through an interview about your experience. Par-ticipants will receive a payment of $10 for the 60-minute study.123Appendix A. Study MaterialA.2 Orientation Media: Echology InstructionCards1. Sounds are controlled by the motion of Beluga whales in the videofeed.2. There can be up to 4 types of sound playing at once, each representedby a different colour of circle. Each type of sound is controlled bymotion in a different area of the Beluga whale video feed.3. Sounds are ?spatialized? within a circular area around the InteractionTable so that they appear to be moving.4. A sound can travel paths between 8 ?reflection points? which are rep-resented by blue circles on the Interaction Table screen.5. The next reflection point on a sound?s path is determined by the ?di-rection? of the current reflection point.6. A reflection point has 8 possible directions: towards any of the other7 reflection points or towards itself.7. The direction of a reflection point can be changed by pressing theassociated button on the Interaction Table.124Appendix A. Study MaterialA.3 Orientation Media: Echology IntroductionCardsDescription with no indicator of live or prerecorded video:Echology is an installation piece that uses a video feed of the Bel-uga whales at the Vancouver Aquarium to control sounds playedin a large atrium space. The sounds are ?spatialized? arounda sound system. Participants can ?play? with the directions ofsound by pressing buttons at an ?Interaction Table?. Feedbackof sound movement is given to the participants, along with visu-als displaying the Belugas swimming at the aquarium.Description stating video is live:Echology is an installation piece that uses a LIVE video feed ofthe Beluga whales at the Vancouver Aquarium to control soundsplayed in a large atrium space. The sounds are ?spatialized?around a sound system. Participants can ?play? with the di-rections of sound by pressing buttons at an ?Interaction Table?.Feedback of sound movement is given to the participants, alongwith visuals displaying the Belugas swimming at the aquarium.Description stating video is prerecorded:Echology is an installation piece that uses a PRERECORDEDvideo feed of the Beluga whales at the Vancouver Aquarium tocontrol sounds played in a large atrium space. The sounds are?spatialized? around a sound system. Participants can ?play?with the directions of sound by pressing buttons at an ?Interac-tion Table?. Feedback of sound movement is given to the partic-ipants, along with visuals displaying the Belugas swimming atthe aquarium.125Appendix A. Study MaterialA.4 Interview Question GuideWhen you walked into the Echology room, what were your first impressions?Can you describe a little more what you saw?Are there things that you particularly liked?What about things that you did not like?Did you feel frustrated at any point?Did you feel tired or bored at any point?Can you remember any things that you reflected on that maybe were notdirectly related to Echology but were thoughts or feelings that came up foryou?Were you relating this to any other experience that you have had before?Can you tell me how you think Echology worked?What connection did you see between the music and sound and the whales?Did you see Echology more as an art piece or as a game or neither?Can you talk about what you might have been thinking or feeling about theBelugas?Have you gone to the Vancouver Aquarium before?If you could talk with the Belugas at the aquarium, what would you say tothem?Can you think of anyone who would find a work like Echology appealing?Can you think of places or events where it would make sense to installEchology?Do you have any further questions or comments?126Appendix A. Study MaterialA.5 Questionnaire1. Age:2. Gender: Male / Female3. Occupation:4. How many experiences have you had with interactive artworks?(None, Some, Several, Many)5. How often do you work with computers?(Rarely, Once a month, Once a week, Daily)6. How much experience do you have playing video games?(None, Some, A fair amount, A lot)7. Do you have any musical education?(None, Some, A fair amount, A lot)8. How often do you go to art galleries?(Never, Rarely, A few times a year, Often)9. How often do you go to museums?(Never, Rarely, A few times a year, Often)10. Did you enjoy your experience with Echology?(No, A little bit, Yes, A lot)11. Did you enjoy the aesthetics (how the interaction space looked) ofEchology?(No, A little bit, Yes, A lot)12. Have you gone to the Vancouver Aquarium before?(No, Yes but a long time ago, Yes, Yes I go often)13. Please list any 3 words that you can associate with your experience.14. In one or two sentences, please state what you believe the generalartistic concept of Echology to be.127Appendix BStudy ResultsB.1 General DataTable B.1: Study participant information: age, assignedalias, gender, and occupation.Session Type: No Instructions, Pre-recordedTold Live orRecordedAge Alias Gender OccupationY 21 Anne F student: psychologyY 23 Andrea F studentY 20 Ambrose F studentY 22 Abbott M studentY 19 Arthur M studentY 24 Amanda F grad student: nutri-tionY 26 Audrey F grad student: nutri-tionSession Type: No Instructions, LiveTold Live orRecordedAge Alias Gender OccupationN 30 Ben M studentY *** Brian M ***Y 21 Burke M student: mechanicaleng128Appendix B. Study ResultsY 21 Barbara F studentY 20 Brook F student: civil engY 36 Brigitte F phd student: dentistryY *** Beth F studentY 28 Byron M studentSession Type: Instructions, Pre-recordedTold Live orRecordedAge Alias Gender OccupationN 31 Carl M grad student: comp sciN 23 Claude M grad student: elec engN 20 Cory M studentY 25 Chris M student: comp sciY 21 Charles M studentY 25 Cassandra F grad student: comp sciSession Type: Instructions, LiveTold Live orRecordedAge Alias Gender OccupationY 20 Deborah F studentY 19 Danielle F student: first year engY 24 Daria F student: enviro engY 25 Denise F admin assistantY 25 David M student129Appendix B. Study ResultsTable B.2: Echology experience durations recorded in min-utes as time from subject entering room to when experi-menter called.Anne 2.5 (minutes)Andrea 10Ambrose 6Abbott 4Arthur 6.5Amanda and Audrey 10.5Ben 8Brian 7.5Burke 7Barbara not recordedBrook and Brigette 10Beth and Byron 11Carl 20Claude 6Cory 12.5Chris 18Charles 9Cassandra 10Deborah 16Danielle and Daria 37Denise and David 5.5130Appendix B. Study ResultsB.2 Questionnaire ResultsTable B.3: Questionnaire results: free input responses.Anne (left blank) Interactive voicesAndrea exciting, scary, grate-fulI think it is a connection between whatpeople hear and what people do withit.Ambrose calming, unique,peacefulBridging art and interaction by usingvisual, auditory, and tactile means.Abbott fun Atmosphere is good, looks like I amunderwater.Arthur curious, fun, enjoy The purpose of the Echology is prob-ably to be used in a movie theatre ofvideo games (like an war / an actiongame). This way, a user would feel asif it happened to him.Amanda (left blank) An interactive game in which peoplecan set sounds or just turn them off.Audrey different, confusing,abstractTo provide a way for the art and per-son to interact for a greater, more per-sonal experience.Ben curious, interesting,enjoyableIt is an idea of communication be-tween human beings and nature.Brian entertaining, mystical,wonderfulA mystical experience where one ex-periences the beauty of the belugasand their interaction with their envi-ronment.Burke behaviour, biology, life It is to investigate and show how ananimal (or maybe other living things)behaves with the surroundings.Continued on next page131Appendix B. Study ResultsTable B.3 ? continued from previous pageBarbara calm, relaxing, blue You can interact with the art, andchange it to what you want.Brook spatial, absorb,bounceI think the general artistic conceptof Echology is to encourage people toappreciate the ?spatial feel? of soundthat Echology can bring to people.Brigitte interesting, fun, simple Mimic the lone situation.Beth calming, interactive,unexpected, fascinat-ingAs an interactive system, Echologymay be to tune the general publicto sounds from nature through theirsenses (touch, listen, sight).Byron mysterious, remote,calmIt sounds mysterious. It makes me feelI?m alone but not lonely.Carl curiosity, thought-provoking, beautifulHelp people learn how animals com-municate. Connect people?s visualand audio sensations together.Claude comfortable, curious,kind of attractiveUse your feeling to control the soundsaround you.Cory (left blank) (left blank)Chris unique, innovative, in-terestingVirtual reality and interactive art-work.Charles cool, aesthetic, littleconfusing when enterthe room/before readcardsExpress sound in a visual way. Makean isolated environment to make thesound more effective than it is, and letthe user to concentrate on the soundin that environment.Continued on next page132Appendix B. Study ResultsTable B.3 ? continued from previous pageCassandra non-intentional, inter-esting, confusedFirst, it?s better that an Echology artpiece would make people involve inthe whole setting. (Just like the set-tings in this experiment, I can havesome communication with the set-tings). Second, a general echology artpiece would better convey some mean-ingful content/idea to the audience.Deborah fun, curious, amazed Using sound as an art, for applicationin public. Art can also be relaxing andfun.Danielle confuse, fun, interest-ingSomething to do with nature.Denise spacelike, underwater,cozyManipulation of sound.David sound, effects, sub-mergedThe manipulation/interaction withwhales by their way of communicat-ing.133Appendix B. Study ResultsB.3 Interview Transcript ExcerptsTable B.4: Participant statements related to Echology as agame and Echology as art.Anne Neither. I think it?s like, those educational stuff, like tellingyou how the whales communicate with each other and stufflike that. Cause like, you showed me the information aboutthe whales before so I was connecting, and I do see the whalesin the background.Andrea I mean, the decoration seems like art, but the table seemslike a game.It maybe could have a score, you know, if you catch it (re-ferring to the moving sound balls) you can get a high score?That?s my suggestion or.. .Ambrose Mixture of both and at the same time neither. Do you havereasons for that? Well, games usually have an objectiveand this one didn?t really have an objective but at the sametime, there was a user interface controls and stuff so well, youcould make it a game if you wanted to. And as an art piece,umm, well usually, well umm, I personally think that gamesare a form of art so it works out in that sense and it justseemed really artistically laid out and involved the senses,but then at the same time, it was completely different frommost things that I have seen that are games or art.Authur I don?t see it as an art piece that?s for sure. I probably justsee the technology side.Amanda It?s not a game.(Audrey) No. . . I think it would probably be more like art.Continued on next page134Appendix B. Study ResultsTable B.4 ? continued from previous pageAudrey (Asked if Echology reminded them of anything.) Re-mind me of anything. .. Pacman or something. (Both Au-drey and Amanda laugh, Amanda agrees.) Pacman and like,frogger.. .no, not frogger the one that?s like (imitates videogame noise and action). Oh pong. Pong yeah (laughs) butit wasn?t really, no, I don?t know. Cause it wasn?t reallyanything. Cause usually when you have computer things likethis there?s like, some sort of point. But like here, it was hardto just sort of. . . just enjoy the balls and we were trying tofigure out some sort of point. Like okay, am I trying to get,you know, this there and what am I trying to do with theseballs and maybe if you sort of went in with the idea thatthere was really no point to it then maybe.. .Did you guysgo in with the idea that... No it?s like, I was trying tofigure out. .. (Amanda: We wanted to know, which one wasthe point.) Yeah we were really trying to figure out what,what to press when. Cause we?re so used to like the gamewhere it?s like (imitates video game sound), where there?ssome sort of pattern to (. .. ) and this sort of didn?t have apattern cause we really didn?t relate it to the whales at all.Ok so when you walked into the room, did you haveany thoughts on ??oh is this supposed to be art???and like...what were your thoughts on that? No Inever thought about it as art, like mostly thought it wasgoing to be like, an activity. . .(Amanda: Like a game?) Yeahlike a game or something so that?s. . .(Amanda: I thought itwas going to be a game) (Audrey agrees) Especially becauseof the buttons.Ben Umm, I prefer game. Game. . . game like, a game but notpurely game, but close. Cause you know ahh, the way youtouch the buttons and then gonna change something around,you change the way the whales are going to swim around andyou change the sound. . .Continued on next page135Appendix B. Study ResultsTable B.4 ? continued from previous pageBarbara Umm. .. I think more like a game. Do you have anyreasons for that? Umm. . . well I don?t know, for me,cause I can actually participate, like change the directionof the sound and stuff, so I thought it was more like a gamethen art. Because for me, art is more like sitting there andjust looking at or hearing and stuff.Beth Not sure, I?m not sure whether I think of it as art, or chil-dren though ? if there was such an interactive station at theaquarium, I?m sure they would think of it as a game causenot only could they use their hands and hear all kinds of dif-ferent sound but if they, umm, they could also not just usetheir hands and listen, but also look at, be able to view Bel-uga images. Like say, before the Beluga presentation therewas a station, they could view how Belugas were like whenthey were playing around. If that sort of interests them, atleast the moving coloured balls would be sort of fascinatingI guess.Byron Neither, neither, neither.. . I don?t think it?s art or game.Do you have reasons for that? I think it?s not a game,because it?s not a game, but I think it?s not art either,but it?s definitely not a game. Though, it seems like agame because I keep pushing buttons but I don?t think it?sa game. Did you think it was a game when you firstwalked into the room? Yeah, at first I, yeah I mean, rightafter I came into the room and was pushing buttons it seemedlike a game, but later, I don?t think it?s a game. (. .. ) NowI would say it is not a game. But yeah, you are right. If youasked me right after I came into the room, then I would mostlikely think, yeah, it?s a game.Continued on next page136Appendix B. Study ResultsTable B.4 ? continued from previous pageCarl Umm, maybe it?s art, for me a game is more interactive andmore directions and you know what to do, like, what?s yourmission in this setting. There?s like, confrontations and stuff.And when I went into this room I don?t know what I?m sup-posed to do, compared with if I want to play a game andthere is.. .I get instructions telling you what you are sup-posed to do, what you shouldn?t do. That?s just one kind ofgame (. .. ) it?s not a game like that.Claude It?s more like a game. Do you have some reasons forthat? No, actually, I don?t like to play computer games, butI feel that this is like a game. Maybe you could tell mewhy it would be a game and not an art piece. Why?Umm, I can tell some reason why it is not a game, becausethere is not a specific goal that I can reach. Then that is thereason, the reason that isn?t a game. But it seems to me thatit is a game, which is by my feeling, but logically, it?s not agame.Cory It?s more like a game to me, I mean, the square, I meanthe circles were kind of like pacmen for some reason. Do youhave reasons for why it?s not art? Just that, well youcan interact with it and then, it changes in your environmentand then. . . so it?s more like a game than an art piece whereit?s more like, still, and all you do is observe it and thenmake an analysis. Did you analyze this at all? Well, Ianalyzed it when I tried to put the sounds with the video andthen with the, pressing of the buttons and then changing themovement I guess.Chris Yeah I think it?s more like an art than a game. Umm. .. YesI think so. (Asked to expand on why.) Umm. . . I think.. . I cannot think of anything right now.Continued on next page137Appendix B. Study ResultsTable B.4 ? continued from previous pageCharles Artwork. . . not quite. Ahh. . . game? Maybe. Artwork looksvery, artificial. .. like art, a piece of art. . .but I prefer to makeit like this game Do you think that it could be both?Maybe both, but more close to a game.Cassandra Can you maybe see this as an art piece? Art piece?Yeah. You mean. . . the screen? or. . . Everything in thewhole... (thinking) No. . . I would never relate that to. ..Why wouldn?t it be an art piece? So there would besome, at least some contents or, or some meaning, mean-ingful things that can be expressed or displayed by the artpiece. So, I would. . .basically that kind of thing would bean art piece But I cannot figure out anything about such asetting so I cannot relate that you know.Deborah Mmm. .. I think it?s more like a game. More like a game?Yeah.. . Do you have some reasons? I?m thinking causeart is something like opera or.. .Denise I saw it actually more like art (. . . ) I think I?d be happy justto look at it without interacting, I mean, the sound is stillgoing around, you don?t have to interact with it to make thesound move I guess.David It was a game, I thought it was a game (. . . ) As a gamebecause there?s buttons and a screen and the moving thingsin the middle like, so colourful. Yeah cause it?s colourful andthe screen and the buttons. Yeah, more like a game, becauseyou can change (. . . ) pushing the buttons, you feel like, ohyeah, this moves this, or yeah it turns out if you press this(. .. ) so you have to figure out.138Appendix B. Study ResultsTable B.5: Participant statements related to understandingof Echology interactivity.Anne I can hear sound travel from hear and there although I can?treally match to the picture I see.I realize I don?t know what the purpose of the balls was.Sometimes the colours become really sharp; sometimes theyare more transparent. Did that frustrate you? Umm, itjust made me, I just. . . I should?ve, I should?ve checked whatEchology was before I came here.The balls. . . how I turn it will change the voices going aroundme. (. . . ) But I can?t connect them. I guess I need more.. .Ihaven?t been hearing a lot of music. I have no experiencewith this stuff. Maybe if I know something before hand thanI can catch it, but with no prior knowledge I don?t think Ican really.. .Did you see any connection between the music andsound and the whales? I assume there must be a connec-tion but I can?t. .. I?m not even sure if something changes. Iknow the sound changes, but I?m not sure how and in whatdirection it changes.Andrea I saw some coloured balls, just some circles. Yeah and theycame around with the animal and at first, at the beginning Ididn?t find that it related (?) and maybe two minutes later Igot it and when I hear something, the screen, they had themdisplayed.Do you think, did you feel that you figuredeverything out? No (laughs) No? I don?t think so.Cause sometimes it is too fast. I cannot catch it andactually I, you know, you mentioned that there is no wrongand right answer, I was just playing there and I don?t knowwhich was the best score.Continued on next page139Appendix B. Study ResultsTable B.5 ? continued from previous pageDid you get frustrated at all? Sometimes.Sometimes? Yeah when the purple and the yellow andthe white and this green one (. . . ) when I cannot catch it(. .. ) for example if I want to catch the purple one I makethe donut and the centre of it is purple. But it doesn?t move.When I click bottom then it?s just like, goes around again.(. .. ) When the whales come to the screen and I found thesounds get heavier.When I press a button, and if I don?t make it like a donutand it turns around to a different direction and when I clickit, the sound will go to another button, that?s why?Ambrose (. .. ) Took a while to realize that it was mainly an audio ex-hibit, and then it clicked that it was really about the directionof the speakers and that this somehow should be controllingthem.(Asked if not immediately knowing how thingsworked caused frustration.) Not really, no.Well the buttons, they control the, there are a few littlepacmen-like creatures, circles with wedges in them that youpush to make them change directions and push it enoughtimes it will form a donut shape where it will trap the sound.And the wedge in the circle has to do with the direction thatthe balls are going to bounce in and then the balls bouncingin there have to do with what speaker you hear the soundsin from the ones surrounding you.Continued on next page140Appendix B. Study ResultsTable B.5 ? continued from previous page(Asked to describe experience to hypotheticalfriend) You are surrounded by a bunch of speakers andblue lighting and right in the middle is this little panelthat?s round, it?s cylindrical shaped and there?s this littletv screen in the middle and all around it you have a set ofbuttons and then, pushing each of the buttons controls afew little circles on the screen that kind of look like pacmenand pushing the buttons controls the direction of the circles?wedges and then they tell you the direction of, there?s someballs that go around and bounce and stuff and then you havesome Beluga whales coming in and then while they comein, you have new balls coming in and these balls representthe sound playing on each of the speakers so you can sortof, using this interface, you can bounce the sounds from onecircle to another and it actually makes the sounds bouncefrom one speaker to another.Arthur I thought the button is used to start the sound but appar-ently it?s not, after a while I noticed it. Did you figure outwhat makes the sound? Actually, I don?t, but I assume it?sa random process.Were you at all frustrated? Ah yeah, I have to say yes,like at the middle of it I don?t really know what the but-ton does so I kept pressing it and sometimes on the monitorscreen, the button changes to square and you have a hole inthe middle. I was like, oh, what?s this for?Continued on next page141Appendix B. Study ResultsTable B.5 ? continued from previous pageAfter some time I see how the button works (. .. ) You havethese sounds, the sound is bouncing back and you decidethe direction, where the direction of the sound will be go-ing to, and there are three types of sounds ? the purpleone, the yellow one and the umm. . . I forget but. . . the otherone, the purple one is just like surround sound, the yellowone I think is the whale sound and the other one. .. I don?tremember. Do you think you?ve figured out most ofthe things going on? I think I figured it out.You could hear the movement right? Yeah. Could yourelate the movement of the sounds to the movementof the balls on the screen? Yeah.Was it fun? After figuring it out? Yes, but before that it?skind frustrating, a little bit but not that much. I want toknow how it works but, I don?t know how it works and thenI just start playing and then once I know a little bit more Istart to enjoy it because I know a little bit more of it.There are three circles there moving around, what is thatused for? What are they actually? (. . .) They are differ-ent. . .?What?s the purpose of this button? Of the button? YeahI think you got that, basically you can controlthe direction of the sounds And why do you want tocontrol it? Like, what do you use it for? (. . .) Why don?tyou just like put it like, you wanna play it on THIS side andyou just press this button like that so that you can play it,something like that. .. ?Amanda (. .. ) To the speakers. If we let one of the balls in one ofthe rings, or (?) in this speaker for example and if the ballwas in another one it come from other speaker so we kind ofrelated (Audrey: We figured that out.)Continued on next page142Appendix B. Study ResultsTable B.5 ? continued from previous pageWhen we pressed the thing, the ball in the computer wasturning and then it was a round circle, like a round circleand then when the sound was traveling it would take thesound cause it was trapping the sound.(. .. ) But then if you press it again you will have differentsounds like, it will be one. and we didn?t understand thatpart well, I didn?t understand that part. Like when you hadone sound, and then you kept it and then when you releaseit then they were well like, three different sounds, or threedifferent circles.(Asked to describe experience to hypotheticalfriend.) There is a circle with some buttons that youhave to push and you will hear this sound and if you trapthe sound then you won?t hear anything and there are acomputer with whales, a video with big whales swimming orwhatever and then you just, make sound and release.Audrey Did you see any relationship between the belugasin the video and the sound? No, I didn?t think aboutthat. We were mostly focused on the balls, the little circles.The belugas were sort of just the background. It seems likewe never thought about connecting them.(Four different sound signals explained.) Yeah wesaw but we didn?t know why sometimes there were morethan one and sometimes it was just one. (Sound-Belugarelationship explained.) Oh so it?s related to the. ..I was frustrated for a little bit of it because I just didn?t seethe point.Continued on next page143Appendix B. Study ResultsTable B.5 ? continued from previous page(Asked to describe experience to hypotheticalfriend.) I went to do this sort of project thing and wewalked into the room and it was kind of space-aged lookingand there was a screen on a sort of table and buttons allover and there were these little circles and you can, wellthey?d move around representing sort of sound and there arespeakers all over so you hear all these whale sounds and seeall these whales in it. I was trying to figure out what it did.Yeah, it sort of didn?t really do a lot of things it just (?)sound and circles and whales in the background. It?s kindof weird (laughs) like I wasn?t really sure what it was for oranything, but it was something different.Ben On all the buttons I don?t see any difference to those but-tons. But each time I touch it and then I saw the whalesswim around somewhere but yeah, I tried to figure out thedifference but finally I think, it?s the same, you know, all thebuttons are the same.Did you see any connection between the sounds andthe Beluga whales? Every time I touch it, it like, changedit a little bit. It looks like communication somewhere butI don?t know what that really mean for whales. But itdid change, I try to figure out the connection between butI?m actually not ? cause it?s not human being. I mean,understandable, cause it?s their language. Cause I mean, Itouch it, I try to, I touch and listen too and then, I didn?tfind the connection. I did find that there was somethingwhere I touch the button here and then the sound wouldchange a little bit, I did like, try to touch it faster then,maybe a little bit of change over there but I didn?t connectit with the movement.Continued on next page144Appendix B. Study ResultsTable B.5 ? continued from previous pageI tried to figure out the connection between the button Itouch and the sound, but I didn?t figure it out so (. . . ) finallyI realized a little bit the connection over there but it wasreally hard to figure out what the connection actually is.Like if it?s how hard you touch it or how fast you touch it. Imean, I found it could change accordingly but I didn?t figureit out, just very interesting. It was a good experience.Burke Is there anything that you are still confusedabout? Yes, so many things, like, how could we get thesound to go from Vancouver Aquarium? (. . . ) How can weplay around with the buttons and then reflect it to differentpoints.Barbara Do you think that you were able to figure what wasgoing on? At first no, then after I read the instructionsand after I play with the buttons several times, I realize thatwe can change the ? through the button you can changethe position of the sound. Were you frustrated when youcouldn?t figure that out? Umm. . . a little bit.Beth I realized that oh, it seemed to be there were a lot of differentkinds of sounds that arose from touching the buttons so thatI thought, was quite interesting.Do you see any connection between the sound andthe Belugas in the video? Some relation.. . like somesounds were definitely Belugas playing around or frolickingaround but others they seemed more like sound effects.Byron There are the coloured balls, through the screen it is a biteasy to change the direction of the balls but if there is nosuch ball then stop the sound, and it is quite difficult tocatch where the sound is from.Continued on next page145Appendix B. Study ResultsTable B.5 ? continued from previous pageI did think that the movement of the ball, the direction ofthe movement of the ball was the same as where the soundis from.I think there was direct relationship between where the soundcame from and the direction of the balls.I think I could figure it out, I can identify the relationshipbetween the sound and the ball and the change of the cir-cles but if you ask further if I could identify further the re-lationship of further how the system works, I don?t thinkso. (Beth: Yeah that was pretty much what I got too.)Did anything frustrate you? Yeah, the sound, I mean,I sometimes use my ears to catch the sound and the soundchanges very rapidly so I try to push this button and (. . . )but I can?t catch the movement of the sound.But frustrating as I said, that I cannot catch the, it?s too fastand my reaction I think is a bit slow. I cannot match themovement of the sound. I know the sound is here, I want topress all of these buttons but I couldn?t.Carl Later I found like, when I pressed these buttons, it willchange the direction of the reflecting part, the circles.Sometimes when the sound is intense, the whale, the wholebody of that whale will show up. But I don?t know duringthe experiment, sometimes it like, resets, after a while thereis low whale and low sound.(. .. ) maybe two buttons, goes to (?) one speaker.(. .. ) just repeated things, just repeated sounds, by switchingthose buttons you can change the sequence of the sounds(. .. )Continued on next page146Appendix B. Study ResultsTable B.5 ? continued from previous pageI haven?t figured out the circles. They have different colourslike there is yellow, purple, red and I think (. . .) and alsoI don?t know how many whales were in the video interface.I don?t know if circles corresponding to each one of them.There are too many things that I don?t know (. . . ) howit works (. .. ) speakers corresponding to reflection pointand change orientation and sound will change and nothingbeyond that (. . .) maybe sounds corresponding to actions ofthe whales (. . . ) Ok, so not being able to get thoseconnections, did that frustrate you a little bit?Yeah kind of. . .Claude When you touched that, did you expect it to be atouch screen? Yeah.So you read the cards before you tried to pressany of the buttons? Yes, and I didn?t know the buttonis for me to press, I thought they were only the decora-tions. OK, so at this point are you still confused?Yeah, sort of. Were you a little bit frustrated?Umm. .. not really.Did you see the connection between the audio andthe video? I tried but it sounds like... this circle is kindof connected to the audio. Okay, then what about thewhales in the background? The whales in the back-ground? Yeah. I didn?t focus on them (. . .) Yeah, I saw thewhales and the audio, but I didn?t focus on the fish and theaudio I mean.Continued on next page147Appendix B. Study ResultsTable B.5 ? continued from previous pageWhat?s the background, the fish ? it?s actually not relatedto the audio at all? (. . . ) Cause I thought that the relationis the, position of the fish with the direction of the audio(. .. ) the audio is up and down, up and down, and that iscorresponding to the water, not the fish mostly. Do you getwhat I mean? (. . .) What the strong feeling to the visitoris the water, not the fish. So the volume of the audio is justlike, gives the impression to the visitor that they are in awater environment.Cory I heard the sounds and then I saw, like, all these things, andthen, it makes sense after I read the cards.Could you hear the sounds and match them up withthe moving the moving balls on the screen? Yes.I did count the speakers and there was six of them and eightdots and I was like okay, it?s missing two but it travels in acircle anyway.Did you see any connection between the sound andthe belugas in the video? Well there?s like four soundsand when I look at the sounds and the video it like, matchesI guess. I assume like, closer by they make these chirpynoises and when they swim farther away or just suddenlyappear or something like that. ..Why it is 6 speakers when there?s like, 8 buttons? (. .. ) I wastrying to follow the sounds and was like, okay so when I cometo this circle the sound is just like, somewhere in between.Continued on next page148Appendix B. Study ResultsTable B.5 ? continued from previous pageChris At the beginning, it took me some time to get used to the en-vironment because I couldn?t differentiate the sound sources.(. .. ) and then I can get used to the space and the soundmovement in the space, and then I can tell where the soundis moving, in which direction, and because there was a screenon the table so I can see where the sound is travelling and Ican hear it at the same time so the table is very useful andhelpful.At first, I don?t know what happens if I push any one ofthe buttons so I just keep on pushing and pushing and then Ifigure out that it will change the orientation of each reflectionpoint then after that I know how to use the interactive part.I think the speakers actually control the sound source, andwhen the sound is travelling, each speaker will change thevolume of its sound so it will create the movement of thesound traveling through the space.I think when the Beluga whales is moving, so is the sound.And I think, I don?t think it?s that much closely related butI definitely can see there is a connection.Just wondering, because there are 8 reflection points, butonly 6 speakers, how does that. . . ?Charles I see that, as the card says, the ball reflects and the direction,it is going according to your current node.(Asked about noticing connection between Belugasand sound.) It took me a while to figure anything out andthe only thing that I could find out is that the balls, thecoloured balls becomes more darker when the Belugas cometo the screen. Yeah that?s what I found.Continued on next page149Appendix B. Study ResultsTable B.5 ? continued from previous pageCassandra Were you frustrated? Frustrated? No (. . .) Actually con-fused. I don?t know what is going on (. .. )What are you seeing when you are pressing thebuttons? For the first time I didn?t notice anything I justpush the buttons and there will be some balls in the screenand maybe after 1 or 2 minutes I just realized that once Ipush those buttons then there will be a move to an angle.So I noticed that and I tried another several times and thenI see when the ball hits, hit on the angle then it will tryto change to its direction. That?s the second thing that Iperceived. That I tried to push the combinations to see butI don?t know the result. I just don?t know what the directionor meaning of the game. ..Did you hear the sound going in the same directionas the balls on the screen? No, I didn?t notice that(. .. ) Actually I can hear the sound moving and sounddifferent directions but I didn?t try to relate the sound withthe direction of the balls.Deborah I saw the little circles, and I don?t know what it is so I justpress it, then certain sound came. And I thought, ok whensound comes, I don?t know right. . . but it bounces, like thesound is also in circles, the first time, so like, ok, maybe if Ipress this it will go to another direction.Were you frustrated at all? Not really. Not really,did you feel confused? Yeah, a bit confused.What type of questions were you asking yourself?Like, what are these buttons for? Then, in the cards itmentioned about the sound or something like that, and Ididn?t hear the sound, so I was like, where?s the sound?What connection exists between the buttons and the sound?Continued on next page150Appendix B. Study ResultsTable B.5 ? continued from previous pageDid you see any, sort of connection between thesound and the Belugas? Yeah, the Beluga is on thescreen, the dots are like, colour not blurry, then when theBeluga is really far away, it?s blurry.David Yeah, yeah. I could hear that the sounds were mov-ing. How about you (to Denise) did you hear that aswell? (Denise: Yeah.)I had to look at the papers for the little ball bouncing wasthe food or was the whale itself.151Appendix B. Study ResultsTable B.6: Participant statements related to sensation andenjoyment.Anne I like the lighting (. . .) the lighting is comfortable.I liked the 3D audio part.The audio part was kind of comforting, I don?t mind stayinguntil I can.It would be nice to sit in the middle and watch a movie.Andrea (. .. ) I found that it is like a mystery.Did you like the sound? No actually (laughs) I find it?sweird. I feel scared when just this purple comes. It?s just, it?smuch more weird and scares me. And blue one and yellowone that?s fine. It?s not too heavy.What were the aspects that you liked? When I catchthe balls (laughs).It?s like, one of the site of the Star Wars you know? TheStar Wars, the movie Ok yeah... like the planet and the(?) table, really like the planet, like control something Likethe control centre? Yeah, yeah.Did you enjoy your time? Yeah, although sometimes Ifeel scared, but yeah, It?s like an adventure I think. Wasit scary because you thought that you were beingtested or scary because the music was scary? Musicand the whole room, the decorations, the lights, I mentionedit?s like a spaceship.Continued on next page152Appendix B. Study ResultsTable B.6 ? continued from previous pageAmbrose The sounds were quite soothing and nice (. . .) the intensityof the sound wasn?t all that noticeable, but it was reallynice. I found that the design of the soundboard thing hadsome aesthetics to it as well, visually.The material at the bottom kind of looked like it came fromsome sort of glacier or something.Are there other terms that you would use todescribe the space? Maybe quasi-futuristic.I like that it?s audio based ? something new and differentbecause art is usually visual in nature.(talking about sounds) They were soothing I can say I guess,with, slightly punctuated by discord and yeah, but mostly itwas unobtrusive.Arthur Oh, it?s like.. . the room itself is so cool because, I don?t knowlike, the way I see it is, it?s like lots of speakers there it?s like,oh wow this is so cool but, like, I didn?t really see into thetable at first. ..(Asked what he liked the most.) Technology (laughs)(. .. ) My first impression when I came into the room waslike, wow, it?s like, lot?s of speakers, it?s like you?re in a movieand you?re going to watch big screen tv and lot?s of speakersthere and it?s like wow. . .that?s the thing that I liked.Audrey Is there anything that you particularly likedabout being in there? Not really.. . (Amanda: I don?tknow. Well we want to just, keep the sound out.)Ben It?s a very quiet room, I feel comfortable (. . . ) and a littlebit curious cause it?s my first time there.Continued on next page153Appendix B. Study ResultsTable B.6 ? continued from previous pageThe things I liked, it?s possible you can play around it, andalso the lights and stuff ? not too bright, a little bit dim, Ifeel comfortable there.The sound. . . it?s not like music, it?s just natural sound, it?snot noise, but it?s not enjoyable. I have to say, it?s not enjoy-able (. . . ) It?s not enjoyable cause for human beings, they likemusic right? But for that sound it?sI still feel comfortable,but not like say, say I enjoy it.Burke I?m pretty sure that?s the technology you have. But I?mpretty curious how to program that stuff (. . . ) I?m one ofthe lead (?) sound system, so I?m pretty curious to how wecan set up that kind of thing (. . .) How could you put thecode in the computer and it like, works with the 8 speakers?Did the sounds in general make you feel anything?Not really. I mean like, that?s kind of useful, to sound like(makes sound effect) and for me, umm, there?s nothingspecial. What makes me really, really excited is how can we,like, what type of program. . .(Max/MSP and programming background explained tohim.) Is there any particular course that teaches this kindof stuff at UBC? (. .. ) I?ve never worked with this kind ofstuff (. .. ) pretty amazing for me. Maybe, this, this is thefirst time.(. .. ) Almost all my friends here, they are like, musical. So,for me, if I can know stuff like this, at least I can have bettercommunication with them maybe. Cause it?s pretty difficultif they talk about music and I know nothing about recordingand stuff but at least if I can learn about this I can bettercommunicate with them and spend time with them.Continued on next page154Appendix B. Study ResultsTable B.6 ? continued from previous pageBarbara I was impressed by the lights. Then I start to listen to thesound that was coming from all different directions so I feltlike I?m actually in the middle of where the sound is comingfrom.I liked the lights and the sounds, it made me, calm down andstuff. And, I liked the fact that I can change the direction ofthe sound.(Asked to describe experience to hypotheticalfriend.) The sounds were like whale sounds, it was prettycool and I felt like it calms me down and I feel relaxed. Ithink it would be fun if there was somewhere like that thatyou can go into when you are stressed out or something, youknow, to get relaxed.(. .. ) I think it would be a lot better if I can actually hearthe sounds. Like usually when I go to the aquarium I justlook at it and that?s it. But if I can actually hear it thenI think that I feel like I?m in the ocean or something (. . . )Could you like, make the whole room with whales like, withpictures and webcam videos and clips to make it feel morelike an aquarium?Beth I thought it was rather nice to just be able to experience thething that was present, and not have a set of instructions, tojust be able to do what we want when we were in that kindof setting.Sort of reminded me of, especially with the occasional Bel-uga sounds, my most recent trip to the Vancouver Aquariumwhere in the atrium they just had all kinds of, I think theyhad this theatre or something where they just played out,just broadcasted all kinds of whale sounds and when I heardthat it kind of just reminded me.Continued on next page155Appendix B. Study ResultsTable B.6 ? continued from previous pageI think it was rather interesting how most of the sounds werefrom nature and they were intermingled with sounds fromBeluga whales.Byron Then I hear some sound and that kind of sound is very, likesome noise, some kind of noise, like the sound from the forest,like you are in the forest, and feel very removed very far away.Sound is very mysterious like in some movie (. . .)I like the kind of atmosphere. It?s kind of a very mysteri-ous sound and like you are in a forest. You are surroundedby nothing but trees (. . .) You are in a forest and nobody is(. .. ) only you in the atmosphere and the sound like, it seemsthat the sound is from very removed area, very removed dis-tance. Like you were in a mountain or in a forest and youhear something and this also very interesting to catch thesound, catch the direction of sound by pushing button.(. .. ) It seems I was dreaming, feeling I was dreaming, I meansometimes when I am dreaming I have a bit similar feeling.I can conclude my feeling as that, I feel that I?m alone, butI?m not lonely. That is my feeling, I?m alone, but I?m notlonely. And I think that, that music, that sound, can makeme calm.I heard some music that seems came from very remote places,very remote place. I thought that the music sound very mys-terious and can leave me feeling calm (. . .) music is veryhelpful for you if you are very nervous for your exams, foryour finals, you just listen to that kind of music, can makeme feel relaxed, not so nervous.Continued on next page156Appendix B. Study ResultsTable B.6 ? continued from previous pageCarl It?s a natural environment, sea animals, they can only hearthese kind of sounds (. .. ) I was separated from the outsideworld and I was like, in nature and kind of those feelings,kind of. . .peace.Interesting, cause if you play with these buttons and you canactually change the sound, how the system sounds, you canlike, if I do this I can affect something. I can enjoy what Idid, if I find some particular setting I can get better soundeffect, it?s worth while to make this effort to try to figure out(. .. ) maybe there are things for curious people you want tospend more time to play with this kind of work and try tofigure out more.Claude (Asked about his first impressions.)Like.. . something. . . mystery about. . .not mys-tery.. .mystical. . .yeah mystical.(Asked to relate experience to another.) Scientificmuseum or like, ghost house (laughs).Comfortable. . . it is comfortable for me (. . . ) I like to playaround the balls on the screen.When the visitor goes into such an environment, they mayexpect the audio is, to give them an impression of the water,it?s like, you are in the water, you are under the water andthe frequency is like that also.I was happy with playing around with this. . .Cory The aquarium sounds, the beluga whale sounds, that?s verycomforting I guess.I got bored after a while (. . . ) maybe just like, one minutebefore I call you?Continued on next page157Appendix B. Study ResultsTable B.6 ? continued from previous pageI thought it was really interesting how you can put a soundwith the video and then you can feel like you are actuallyin this place by just like, standing there and listening to thesounds going around you.Chris I think the setting is very nice and the room is dark and quietso the lighting is very comfortable and it kind of gave me thefeeling that I am entering an art museum or an art galleryand not just a lab either.It?s very peaceful there.There is nothing like this that I have experienced before.Like a total surrounding environment that you are actuallyinvolved in.Charles Kinda cool, like an aquarium inside, and the sound is very(?) home stereo, the sound is very good.I just like to, I feel like, sound like goes around me, and that?svery cool (. . . ) The environment is very fantasy.The coolest thing about this experience is the room, thesounds in the room.Cassandra Sometimes when I tried to press the buttons and it wouldchange the angle, then if it moves 8, or maybe, sometimesseveral times it will be a full round, then if a ball hits intothat button, then it will just, stuck there. So I think thatwas one of the interesting things I think.I think the whole setting is similar to all the other aquariumsthat I?ve been to. I?ve never been to the Vancouver aquariumbut I?ve been to another aquarium in Baltimore, the settingsand the sound there are similar.Continued on next page158Appendix B. Study ResultsTable B.6 ? continued from previous pageDeborah I just like, playing around with the buttons, just like, I thinkit?s fun (laughs) I?m playing around.I was curious at first. And then I thought, how come I neversee this before, even at the aquarium, and then, well it?s coolI think.Cool buttons (. .. ) and it looks cool.Denise Whales actually swimming around you or something, kindof a visual part of it would be more interesting (. . . ) ButI think making it soft, like making it dim like that reallymakes it more about the sound and where they are comingfrom. Cause if you were in a bright room and there?s soundscoming from different directions. . . kind of obvious.It actually reminded me of being in the aquarium with thedarkness.I kind of liked, you know that room was kind of small. Iwas thinking it would be a lot bigger, but it actually felt,quite, kind of intimate, you know, dark and kind of creepyand these noises going around you.(Asked to describe experience to hypotheticalfriend.) I would talk about the dark room and the noisesgoing around, like you can see, directions. .. I don?t reallyknow. I enjoyed it I think, yeah, I would have liked it if thesound went on a bit longer.It made me think of like a, spacecraft or something.Continued on next page159Appendix B. Study ResultsTable B.6 ? continued from previous pageI think I expected maybe like, pictures of whales going aroundon the walls or something. And I think probably you?ll seeon the video, I think I was kind of like, looking around go-ing, is there anything else? (laughs) (David: yeah) I guessI expected pictures to come with the sounds as they movedmaybe? (David: yeah, there was just low visual part.)David It?s kind of cool but you know, I?ve been at Disney Worldand similar places where they have installations and specialeffects and stuff and you know you find well mounted things,like walls, cables. You don?t see the speakers, you don?t seethe stuff, you see only whales and (. . .) So, although my ex-pectations were high cause I have already seeing those typesof things at the giant places, it was really special because ofthe set.(Asked what Echology reminded him of.) Wheneveryou are swimming under the sea, you can here sometimessounds.(Asked to describe experience to hypotheticalfriend.) (. . . ) The blue lights made me think I was in thesea, submerged, and the console in the middle, I don?t know,like a cockpit or navigation kind of thing.160Appendix B. Study ResultsTable B.7: Participant statements related to connecting withthe Belugas and the live webcam feed.Anne The first thing I notice is the table and then and then I saw,I?m going to see some whales but they?re really, they are justbackground, very transparent. I can?t really see the whales.The thing that I noticed most was those little pacmen.It was kind of blurry. I didn?t even pay close attention to thewhales.Any thoughts you have about the Beluga whales?Any thoughts? I just know they are unique with their voice,with their heads.Andrea I?m wondering that, is that a video of underwater? Yeah,it?s... Is that a video of underwater at the aqua centre?If you could say anything to the whales, is theresomething you would say? I just wonder how theycommunicate with each other. And I have a question ? thesound is artificial or it?s real? (. .. ) Can I hear the soundswhen I visit the Aqua Centre?Did you notice the Belugas or have any thoughts onthe video? No cause when they come, I heard the sound.That?s it.Ambrose Asked about the Belugas as content. I don?t know, Iguess they could have used any other animal or anythingthat makes a sound and it could have worked out either way.If you could say anything to the whales, is theresomething you would say? Hmm. . .hi, how?s it going?Continued on next page161Appendix B. Study ResultsTable B.7 ? continued from previous pageDid you feel any sort of connection with them inthe video? Not really. Any enjoyment of watching thevideo? A little yeah, it moved a little bit too slowly. Itwould take a while for the Beluga whales to come and stuff.Arthur (Asked about video in the background.) The video inthe background? I don?t find it helpful (. .. ) Yeah, I didn?treally see it.Amanda Then we saw the whales but we, I don?t know if they werethe same ones or different whale? Is it the same reel butrepeated or like, different reel?(. .. ) We saw the big one and it was sort of like.. . waaahhh(they laugh). It was a big whale.When the sound was traveling we heard this noise (imitatesbeluga voice). I told you (imitates again, both laugh) (. . . )Like the beluga was crying or something.Audrey I don?t know, but yeah the video was sort of, you couldn?treally see the whales that well. (Amanda: no.) It was kindof a blurry feel. (Amanda: mmhmm.) Did that frustrateyou a little bit? Well yeah, sort of like.. .(Asked about seeing the Belugas at the VancouverAquarium.) (. . . ) You know how they have that thingunderneath? Like, I?ve seen that and that?s cool. (. . .) Ithought they were cool they were (. . .) they were reallyneat cause the ones, when we went, one of them was stickingits tail out and slapping the outside and when we saw themon top we could see the tail and then down at the bottomyou could see them sort of like, vertical? (. . .) You knowthe tails out on the outside and it?s kind of neat to see that(. .. ) Yeah, I like the beluga whales. They?re probably thebest things there honestly.Continued on next page162Appendix B. Study ResultsTable B.7 ? continued from previous pageWe were mostly focused on the balls, the little circles. Thebelugas were sort of just the background. It seems like wenever thought about connecting them.So like, what was the concept? (Echology conceptionwith webcam feed explained.) Oh yeah that?s why.Ben I don?t think it?s very interesting. Cause you know, the screenshowing up there, it?s not close to a real thing, it?s just afigure there. I mean, if there?s real whales somewhere, evenif it?s just. .. it?s not really thing so.The whole experience is about the whale I would say, I triedto, cause I read article about the whale that said the whalesound is very ahh, very special, cause whales can talk toeach other through their sound. (talking about tv programviewed in past) But whales definitely, they can understandeach other, and then. I think whales are very good animal,very smart animal, just like a, even smarter than dogs, myunderstanding of them (. .. ) they communicate well withhuman beings.If you could say anything to the whales, is theresomething you would say? I would like to kiss them(laughs). Whales really smooth right? I think whales knowthey are very friendly to human beings. The only thing Idon?t like is just, a lot of time it?s not like, real whales. It?sjust figures, still very interesting (. . . ) you touch somethingand then you hear something.Continued on next page163Appendix B. Study ResultsTable B.7 ? continued from previous pageWhy are the whales never going to. . .like let?s say you haveto touch it, touch the button like you touch the whales andthere, on the screen, close to you, it allows you to communi-cate with the whales, but these ones, they always stay in thesame place. Like in the video? Well, because it?sa live webcam feed so those are real whales. Oh!Oh my god, I don?t realize (chuckles) so if I was confusedbeforeI?m definitely going to pay more attention to how thewhale. .. oh, really, it?s real whales? Yeah, so we don?thave much control over where the whales go... Ohmy god, that is very interesting, I would like to do it again(. .. ) It?s really interesting, that interesting, that?s realwhales over there, no that side, oh, omigosh. I didn?t re-alize. I just think it?s like a computer game stuff.So what?s behind touching the button? Is it actually like, Ican touch the whale? (. .. ) When you touch the button overhere, and there are balls over here, the whale will know thatyou touch it?Did anybody actually know what the whale is taking about?The whale language?Burke Yeah like, since, I don?t know about biology and stuff, thatspecies, I just want to have time maybe looking around Van-couver Aquarium next summer ? I?ll go this summer (. . . )Are you advertising for the Vancouver Aquarium?That?s pretty interesting for me to see something live.Continued on next page164Appendix B. Study ResultsTable B.7 ? continued from previous pageIf you could say anything to the whales, is theresomething you would say? Maybe like, how long can awhale live like, what does he eat, or something like that.How could they keep it alive for a long time in that kindof place, like, what was the person in charge thinking thefirst time when he gets the whale and puts it in there.Ok, so you are thinking a little bit about whalecaptivity? Yeah, maybe not really, but how, not what,but how.Barbara I was thinking about the time I actually went to the Aquar-ium (. . .) Since the sound thing, it was whales. . .If you could say anything to the whales, is theresomething you would say? I think I would ask them tocome near me so that I can touch them.(Asked about video in background.) I thought that itworked with the sound, but then I was distracted by thecircles going around.Beth I think the video seemed to be a little bit static, like it wasjust images that changes every once in a while. Perhapsbecause I was too focused on trying to press the buttons toget the sound (. . . ) but when I did pay attention to thebackground video, I thought, it seems to be just images of, Ithink there was two Beluga whales?Byron At first, I have thought that that was not the animal that youmentioned, I have thoughts that that thing, I don?t know howto say that in English. Like a beauty fish but like, a beautywoman Oh Mermaid. Yeah, at first I thought that was it.Continued on next page165Appendix B. Study ResultsTable B.7 ? continued from previous pageCarl There are sounds that are like (. . . ) I think it?s kind ofsad. It?s like (. . .) maybe it feels not that good. Kind ofbut I don?t know how to like, release those kind of painsthey?re maybe feeling, like, try to cheer them up. I don?tknow (chuckles).I also don?t know the answer to that question, if this is live ornot live, I don?t know, just repeated things, repeated sounds.I think this, is uh, just a sequence of image I don?t thinkit?s, it?s not as realistic as you like, stand in front of, justseparated by a glass and you see whales there and they areswimming and coming out and it?s more realistic. For me it?sjust the sequence of images and video.Ok, it seems to me, if I do something, this will affect thewhales in the aquarium (. . .) It seems like you take care,maybe you can give something to the whales. Maybe thewhales are trying to get some food like it?s a game where youadjust those orientation of those reflection points maybe it iseasier for them to get food then you won?t, sounds won?t besad and you get cheerful sounds. I was trying to see whetherit works (. . . ) after I while I thought it just a sequence ofvideo and sounds get repeated and I change orientation ofreflection point.Claude The video in the background, I didn?t focus on the back-ground, I mean on the fish. I didn?t focus on the fishes, justfocused on the moving ball and the circles around.The purpose? The purpose just like, you press this to playaround with the fish. That is what my guess.Continued on next page166Appendix B. Study ResultsTable B.7 ? continued from previous pageCory If you could say anything to the whales, is theresomething you would say? I would ask how they feelabout living in such a place that?s not exactly theirenvironment.(Asked what he thinks a Beluga-whale relatedmessage of the artwork might be.) Maybe some-thing with beluga whales, their environment. I guess causethat?s what my thing is so I would say, their lifestyle andtheir habitat and they?re just trying to say something tohumans and then we can?t actually relate to them becausewe don?t understand them as well as we think we do. Doyou know a lot about like whale conservation andwhales in captivity and stuff? Yeah. What are yourthoughts on that? It?s not good, their not doing a verygood job of it. Did you identify with the whales atall? My thoughts on them is like, pretty crazy but. . .Iwould identify myself as a whale because, I dunno, thisenvironment maybe, like, being put in a different place, in adifferent home I guess, that sort of thing.That?s what I?m going into actually, macro biology and envi-ronmental sciences. Do you have a particular interestin whales? Yes I do.Chris What were the things that you didn?t like toomuch? I think it would be the resolution of the screen is nottoo high so sometimes the image of the whales is a little bitblurry and it?s not so clear.I think my wife would be interested in this because she lovesanimalsCharles If you could say anything to the whales, is theresomething you would say? Are you bored in there? (welaugh) Yeah, it?s so small place.Continued on next page167Appendix B. Study ResultsTable B.7 ? continued from previous pageDid you pay more attention to the sounds or thebelugas? The sounds cause I was playing with thedirections of the sounds I guess.Cassandra (Asked about video in background.) Which one? (. . . )There is a background video? ...with the beluga whalesswimming... Oh yes! So you mean like, on the screenthere is some whales, some swimming. Okay, I un-derstand. Did you sort of notice connection betweenthem and the sound? Sometimes yes.Deborah (Asked about the video in background.) Oh, yeah, Ididn?t really notice it. I more focused on the sound.At first I don?t really notice the Belugas but then like, ok,there is something swimming around so I look at it (. . . )Belugas, oh yeah, but at first I didn?t really notice it.Denise I was a bit kind of confused I think because it talked in theinstructions about the visual thing but it was actually kind ofhard to see the whales on the screen (. . .) Is there a stronglink between the whales and the sound I think, cause thescreen quite small.The pictures were small and it was actually quite hard to seethe whales so it was a bit disconnected for me, from I think,the whales. It was like, these noises going around.Is the sound the actual sounds that the whales are makingat that time?David I imagined swimming around whales.(Asked to describe experience to hypotheticalfriend.) I would say that it would be a place to familiarizeor interact with whales, to know more about whales, andthe way they communicate.168Appendix B. Study ResultsTable B.8: Participant statements related to participant ex-pectations and technique.Anne I should?ve checked what Echology was before I came here.Andrea I just have an overview of the room and then I just went tothe table and watch the screen and just play (laughs).(Asked to describe experience to hypotheticalfriend.) I would tell them at the beginning I have no ideaand at the end I still have no idea what is the right answeror the wrong answer. I just found, it?s weird, that?s it. Doyou think that there needs to be an answer? If it isnot a test I don?t think so. It maybe could have a score, youknow, if you catch it you can get a high score? That?s mysuggestion.Ambrose Are you experiencing any uncertainty or confusion?Yeah a bit of, I?m trying to figure out what those buttonsactually do.Not really sure how it has a purpose exactly except maybeto - I know it?s using the Beluga whale sounds because theexperiment says so and then you have the occasional whalefloating through the screen and, but other than that, I amnot really quite sure what it would mean. Did you come upwith anything on your own? Not really, maybe a sort ofoceanic experience of some sort.(Asked about reading card on table.) Yeah. I wentback to look at it a few times (. .. ) will this give me ad-ditional information as to how this exhibit works.(. .. ) By the time I left I had already figured out all I felt Ineeded to.Continued on next page169Appendix B. Study ResultsTable B.8 ? continued from previous pageArthur When I saw the table like, I was confused like, what should Ido now? Then I pressed a button there and the sound startsplaying.I read the card but in the middle of my reading I just presseda button.I played around a little bit like, 2, 3 minutes but then, afterI know how it works I just call you.I just wanna know how it works, like the sound or whatever,how it works, that?s all, not like the process itself.I?m an electrical engineering student (. . . ) so I probablydon?t enjoy art that much.(Asked to describe experience to hypotheticalfriend.) I went to an experiment and the experimentis like, how you move different sound to different speaker,I?ll probably say that, not like the beluga and Vancouveraquarium stuff. . . I?ll probably say just the technology is likethere?s (. .. ) lots of speaker and the sound goes throughthat speaker and you feel it, you feel the movement.(Had a friend who did the study before him.) It?s funnyactually because after I spoke with my friend like, she talksabout the art stuff instead of like the technology side and like,now I know ok, she probably don?t like the technology sidemore than the art side (. . .) She?s just telling me that there?slike, the movement of sound on this experiment but like,that?s all. Did she tell you how it works? No Did youhave expectations? No, it?s like, I know that I?m gonnado this, I don?t really wanna ask her before that.Continued on next page170Appendix B. Study ResultsTable B.8 ? continued from previous pageHave you done other studies? Yeah a lot of them actu-ally. Did you think that you were just going to sitin front of a computer and... I think I would do thatyeah, because all of my studies, they are all like that. Like,one person makes a program and I have to like, play withit or like, see which one does better or something. Lot?s ofthem have to do with graph, plots, and like, also personalitystuff.I like to know how stuff is working so that when I first cameinto the room instead of enjoying it has an art I just, I figureout what does this do and something like that.Amanda (. .. ) Then we read the instructions and we understandand, it was sort of understandable thing. (Audrey: Yeahwe were trying to figure out what the point of the thingwas. Were you trying to figure out what the pointwas as in, why we made it or, what it does? Like,how it works.We want to just keep the sound out. What do you mean outlike, not playing? Uh-huh, well we turn all the circlesaround to the closed circles (?) but then if you press it againyou will have different sounds like, it will be one.Like okay, let?s play and see what happens. And we startturning the. . .circles? You know, on the screen and makechanges in the position and (?) and then we want to justkeep the sound up (Audrey agrees) and play (. .. ) for me atleast.Audrey Yeah well, first we were trying to figure out like how pressingthe thing would do, and what the little balls were and howit related to the sound.Continued on next page171Appendix B. Study ResultsTable B.8 ? continued from previous page(Asked about previous experience with interactiveart.) Well yeah I went to one and they had like mechanicalthings that would move but nothing like, electronic oranything, or sound or anything, visuals (. . . ) No wait, theydid have a tv thing once (. .. ) there?s some weird wordsflash on the screen, but other than that I?ve never hadinteractive where I am actually touching, just watching atthe art gallery.I was frustrated for a little bit of it because I just didn?tsee the point (Amanda agrees) Did you try and think ofwhat the point might be? No not really, I just thoughtok (Amanda: let?s play) Yeah (both laugh).Are there still questions that you have... Yeahlike, I?m not so, this is supposed to be like an art displaywith sound? Is that what it is? (Talk about interactiveart for a while.) It just looked like the point of it was tobe an art piece (. .. ) Do you think it would be maybe a bitdifferent if the experience of going to an art, knowing that itwas like an art specific thing that. . .abstract. There?s reallyno just like art I can just go and look at it and try to findsomething the artist is trying to say. Like if I had come infrom that point of view then it probably would have mademore sense but because, it was just that by itself in a roomand I had never seen or heard anything from it, it was reallyconfusing. (Amanda: and you think that it?s a game) Yeah,you?re expecting some sort of activity.Ben At first I tried to find the camera (. .. ) I?m just curiousbecause you just say there are buttons here so I just touchand try to play around with it so I am just curious (. . .) thatwas my first feeling about it. You just play around and youtouch it and the whales swim around and different figuresshow up there.Continued on next page172Appendix B. Study ResultsTable B.8 ? continued from previous pageI tried each of them, actually, and I tried to figure out what?sthe difference between those balls, but I don?t think there?sa big difference there. I also pay attention to the music andactually at the very beginning I didn?t notice the music there.At the very beginning, I feel really curious, not sure whatI?m doing there, but I have to read the manual, read theinstructions and then decided what I can do.(talking about the sound) . .. at last I just ignore it (. . . )didn?t pay attention to the sound, I try to focus on the buttonstuff.I?m very curious about the whole thing and how the wholething works cause I?m a computer guy so I try to figure outhow something works.Burke I walked around and then I looked at the speaker and stuffand then I started reading the instructions on the table andit says like, I should push the button and then I pushed itone by one (. .. )Barbara (Asked to describe experience to hypotheticalfriend.) I walked in, I didn?t know what to do at firstwhen I walked and she just left. I felt like, why she goingaway (laughs). I thought, oh she should give me someinstructions but she just left so I started to walk around theroom and I saw these lights and I saw this screen and then Istart to press the buttons but I couldn?t figure out what itwas so I start to read the instructions and then I start toplay with the buttons and it worked.Do you have things that you are wondering about?What?s this for? What do you think it is for? For me,it just sounds like, for the one, when you cannot go to theaquarium and want to, or like, when you want to relax.Continued on next page173Appendix B. Study ResultsTable B.8 ? continued from previous pageBeth Rather than reading through the card carefully I justskimmed through it and got a general idea.I more so paid attention to the coloured balls on the screenthen followed the sound even though they were kind of like,like, all the sounds were like, just coming out all around me,I was somehow just paying more attention to the colouredballs.I was more thinking like, how to keep the sound going as well,like, realizing that I made the circles in the perimeter donutshaped they would stop so I would try to make the circles beable to bounce from one end to the other to have kind of twoopenings.(Asked to describe experience to hypotheticalfriend.) I did this interactive study today that happenedin this darkened room where I could just explore how thesounds that I heard made me do certain things like pressbuttons or pay attention to the screen, while also having meto try and think about what all that I said before had to dowith Beluga sounds.Byron After I came into the room I tried to read the green card toknow what we should (. .. ) I try to change the direction ofthe sound by pressing the buttons. If I hear that sound isfrom that direction I hit this button.I think by pushing buttons, sometimes I can trick, by pushingbuttons sometimes I can trick the sound to happen.Continued on next page174Appendix B. Study ResultsTable B.8 ? continued from previous pageMy experience is that I need to push several buttons to chasethe sound. I need to push several buttons and then there canbe some kind of sound. But if I just push button one timeand then, I need to push several times and then there will besound.Yeah you can see I always move my hand right. I?m tryingto catch the sound here push button and there push button.(Asked to describe experience to hypotheticalfriend.) You push buttons, you catch sound by pushingthese buttons, you can use your ears to catch the sound, themovement of the sound.I think next time when you have other people do this ex-periment, maybe you should have a, not step by step list,but perhaps detailed instruction because I think we at leastshould have read the green card here at the very beginningand not in the room. Cause I think that maybe if I read thegreen card here in this room maybe I could perform better.Carl I just wonder what?s around this and like, how it works. Justkind of just curious.I went to see whether these buttons were control buttons tothe speakers and if I press one, if one of your speakers wouldsound.No I just wondering what would the sounds maybe soundslike if I try different settings to see (. . . ) want to hear thesounds, the system response to those settings.Continued on next page175Appendix B. Study ResultsTable B.8 ? continued from previous pageBy curiosity I want to know if I do this, what will happennext at the same time when you hear this sound, sometimesit?s cheerful, sometimes it?s kind of, it?s not good feeling,when I hear those kind of sounds. But maybe by curiosity Ijust want to, how can I make those cheerful sounds appearagain and try to avoid those (?) sounds. Try to figure a wayto like, I want better feeling inside.I tried to figure out which speaker is correspondingto. .. because there are 8 of those buttons and only 6 workingspeakers and I tried to figure out which speaker is corre-sponding to which button (. . . )Were you hoping that there would be instructions?Yeah, um, a little bit. Like just, something, you press thisbutton and what will happen if you like, press that button.Like, I even tried to turn this button because I didn?t knowwhether to press or turn to change that direction (. .. ) Itried to figure out the rules, the settings.(Near end of interview) Yeah, I?m still trying to figure out. ..Claude I can play around with many things. When I got into theroom, I thought just computer, I can play around here (. . . )I might walk around before I came to the table.I try just like, move the ball, actually it?s randomly, yeah. Iarrange it just randomly. But I try to form some triangle orsomething like that.The speakers, I thought there should be something on thespeaker, not just the table because it?s like a very (?) en-vironment, a very spatial place and what I can play aroundwith is just table. That is why when I stepped in the room,I walked around to see (. .. ) Or don?t show me the speakers,that is okay.Continued on next page176Appendix B. Study ResultsTable B.8 ? continued from previous pageCory I?m looking at the screen, trying to follow the sounds.That?s when I read the cards like, the second time (. . .) Ican press these buttons and then I can change the directionsand then, make the noise go where I want it to go.Chris So when I first came in, I look around the room and the spaceand I tried to push the buttons and see how the reactionwould be (. . .) I walk around and try to stand at differentpositions around the table and I want to see whether thereis any difference (. . . )Charles Before I press any buttons I read the cards so I know I shouldpress any button to see the ball reflect around (. . . ) But I stillused maybe like, 30 seconds, to actually manipulate anything(. .. ) I was trying to figure out when the sound actuallycomes on. Cause sometimes it is very loud and sometimes itfades away and the sound is quite, like, it is not as loud asother times.I tried to make the sound go like a circle and I found that ifI like, when I try to change the direction of one node, and ifI make the angle too large it will become like a ball with thecircle inside and it will trap the sound inside. And I wouldtrap maybe 2, 3 balls inside and then I release them. I alsomake two nodes face to face and maybe 2 or 3 sound balls gobetween those two.I just think, I should like, try to find more about this to an-swer your questions better. What kind of questions didyou expect that we would... I don?t know, that?s why Iread the cards maybe 3 or 4 times, and try to find somethinglike a clue about your questions. Ok, so did you feelthat you might be quizzed or tested? Yeah, probably(. .. ) when the colour of the ball becomes darker, you mayask why.Continued on next page177Appendix B. Study ResultsTable B.8 ? continued from previous pageCassandra (. .. ) I don?t quite understand that so I try to play on thebuttons so at that time I didn?t read the instructions so Itried several times, I guess, 5 minutes or more I tried topress all the buttons and see what?s going on.(. .. ) Then I tried to figure out what is going on and I findthe instruction cards there so I tried to read it Did you readany of the cards under it? No I didn?t. (. . . ) So thewhole purpose is, I was trying to figure out what I should,the result that I can get from such a game but I still don?t,didn?t figure it out.(Asked to describe experience to hypotheticalfriend.) It would be a little difficult for me to describe thisbecause I don?t know the goal of such a setting so I wouldjust simply say there is this screen with a lot of buttonsaround and there?s a (?) around the settings so if youpress a button then the angle of each button will change itsdirection, then you can play with these buttons and changethe direction of the balls. I think that?s all, cause I?m notquite clear about the goals. I also tell them that it?s justlike a game but I?m not clear about the, about your goals.What?s the purpose of such a setting? Echology conceptsexplained. So when I press the button and change the direc-tions maybe it?s a little bit angle so the sound will change?I didn?t notice that. Sound control explained. Yeah Ialso noticed that sometimes like, the red one will be verystrong and sometimes it will be dim, the colour will change.Simplicity and playing with whales explained. Okso just like, people and whales can play with each other?Okay (. . .) I didn?t realize that when I was playing that.Yeah, probably it seems like a good piece, seems like a goodidea (. . .) I would like to know that beforehand so that I canplay with it, so that like, purposefully I can try it. But, if Ididn?t know that maybe I can never figure that out.Continued on next page178Appendix B. Study ResultsTable B.8 ? continued from previous pageDeborah From that description in your consent, I?m not really sureabout what I am going to do and when I walked into theroom I was like, ok there?s buttons so what should I do? Idon?t know. And then I read the cards, the first card, thetop one, and I thought ok, I will try to push it and I sort ofget it, what I?m doing.Denise It wasn?t really what I was expecting. I think that maybe Ifelt that there would be more of a visual thing, as well as thesound.(Asked about reading the instruction cards.) I readthe cards.I think if we had had a little bit more time with the noises,that would have been a bit better.(Asked about opinion of interactive art.) I thinkit?s more of a personal preference but I?m just not reallyinto (. . . ) and I guess, it?s kind of frustrating too, if youdon?t know, what it?s meant to do or what you can do withit. You can press buttons or do things on the screen andnothing happens or, I think it?s kind of frustrating when youtry to interact so I just watch.David I was expecting something dark and blue, because of theaquarium.(Asked about reading the instruction cards.) I waskind of reluctant to read them. How come? I don?t know itwas more interesting pushing the buttons and looking at thescreen.I was more interested in figuring out what was this about(. .. ) and associating the sounds to the ball bouncing to oneside of the circle.Continued on next page179Appendix B. Study ResultsTable B.8 ? continued from previous pageIt would be better if the screen has something more different,more interactive, more interesting than the pacmen turningaround (. . . ) Cause you have to wait for the balls to get tothe point where the sound is resurrected right? So that?swhere you come into play, you have to wait for that so, whenthere?s no sound you get frustrated, and when the sound ismoving around then you can start pushing the buttons.I totally prefer to interact, it makes it easier to understand,and more fun and maybe, I can appreciate art but, wheneverI can touch it, and I can do something with it, it makes memore focused.180Appendix B. Study ResultsTable B.9: Participant Statements related to breakdowns.Ben There?s a button here and I?m not sure why it didn?t go on(referring to video) cause I didn?t touch it at that time butit?s still gone (. .. ) cause that?s why I?m curious about it andI try to read the instruction manual to figure out why it?sgone (. . .) And then I try to touch the game and then it?sback, you see what I mean, you see what I mean? That wasconfusing? Yeah I try to find the magic or something there.Not confusing, it was just, I was trying to think of if it wasdesigned that way, even when I didn?t touch anything its gotno light there.Burke I pushed it one by one (. .. ) nothing happens and then,alright, what was I supposed to be doing and then I walkedto the laptop and then saw what was on the laptop but thennothing?s happening and then ok, like, I decided to call you.Were you frustrated? Not really frustrated, but, I waslike, just kind of weird, like, I was supposed to play butthen, ok, and then I just decided to call you because maybesomething is wrong, that?s what I was thinking.Are you at all disappointed that when you firstcame in it wasn?t working? Disappointed? Hmm. .. no,cause I know that there might be something wrong andI?m pretty sure that you can fix it cause you are like, theperson that makes the installation work, I?m not, but thenI?m not feeling like I was frustrated or something cause Iwas thinking like, maybe I did something wrong or maybe Ididn?t understand the guideline very well and that?s why Icalled you.Continued on next page181Appendix B. Study ResultsTable B.9 ? continued from previous pageWhat do you think the purpose of Echology is?Maybe it?s like to get to know how a person feels the firsttime when things are not in proper? And then, but afterthings are working very well, you might want to investigatehow a person maybe plays with the (?) and how curiousthat person is. .. I?m maybe wrong (laughs).Did it make up for the fact that, well, you sortof understand the webcam technology and it slowsdown... Oh yeah, I know that, and it depends on theInternet connection. .. if the connection slows down.Barbara Did you call me right after the music had stopped?No, I waited for a while and it wasn?t. . .I thought I didsomething wrong so I keep pressing buttons, and I thought itwasn?t working, and then I thought that maybe the sessionended, like automatically. That?s why I called you.Denise (. .. ) When all the sounds stopped, we were kind of like,why did the sounds stop and what?s it really like, is there astrong link between the whales and the sound I think, causethe screen quite small.I think if we had had a little bit more time with the noises,that would have been a bit better.I thought it was interesting until the sound stopped. Andthen we were sort of like, in a dark room with these buttonsthat weren?t doing anything (laughs) and we?re kind of like,what the hell are we supposed to be doing, what the hell isthis? It doesn?t do anything.I wondered if it only was, like if it only went for two minutesor something, and then at the end of the two minutes it wasdone and nothing else happened.Continued on next page182Appendix B. Study ResultsTable B.9 ? continued from previous pageDavid When did the whales stop, was that pretty muchright after you guys... Like, umm, five minutes itstopped. (Denise: Yeah, they went for a bit.)Yeah cause I got impatient because nothing was happening.There were not any lights. So many things around and noth-ing happened. Ok so at this point here, is thereany noise right now or are you guys kind of...Yeah there?s no noise. There?s nothing moving around sothere is no sound. And so you just kind of sat thereand... . . . and pushed everything. Did you think that ifyou pushed something that the sounds would start again?(both) Yeah Then did you guys go back to the cards?(both) Yeah.I also wondered if I touched the wrong button or if I movedthe things the wrong way.183Appendix B. Study ResultsTable B.10: Participant statements related to collaboration.Andrea I find the table is too large (laughs) and actually I can workaround the table but I find if I stand at one point in place itis more convenient to concentrate.Barbara It was hard to reach the other side so I was a little bit frus-trated about that.Beth (Mentions skimming through instructions then) I think it wasmy partner who first started touching the buttons.So were you guys working together at all to tryand? Not really (Byron: No.)I did notice that near the end of my partner trying out, thatthere seemed to be more sound coming out and he was defi-nitely pressing a lot of buttons.Denise Did you tell him what the cards were saying? Notreally, I think we talked a little bit about what, when youwere moving around, pressing the buttons and changing thedirections did to those circle things.184Appendix CBREB Certificate ofApprovalFigure C.1: Certificate of Approval received from the UBC BehaviouralResearch Ethics Board (BREB)185


Citation Scheme:


Usage Statistics

Country Views Downloads
China 10 34
United States 9 1
Germany 2 10
Macedonia 1 0
United Kingdom 1 0
France 1 0
Sweden 1 0
South Africa 1 0
India 1 0
City Views Downloads
Beijing 10 0
Ashburn 5 0
Unknown 5 11
Washington 2 0
Stockholm 1 0
Plattsburgh 1 0
Rockville 1 0
Johannesburg 1 0
Chennai 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items