Science, Faculty of
Computer Science, Department of
DSpace
UBCV
Inkpen, Kori Marie
Doctor of Philosophy - PhD
University of British Columbia
The presence of computers in schools has grown tremendously over the last ten years. In the wake of this enormous growth, sound research on how to effectively design learning environments and successfully integrate computers into the classroom is needed. The research described in this dissertation evaluates computer-based collaborative learning environments for children using three important criteria: (a) the social environment in which the technology is placed, (b) the technology that provides for explicit collaboration, and (c) the low-level interface design. An additional focus of the research, which crosses all three themes, is gender. The research comprised three experimental studies that were conducted in the three research themes. All these studies employed a creative problem-solving game as the research vehicle. The social theme of the research focuses on the interactions between children mediated by computers. We examined whether the ways children were assigned to work on computers affected their achievement and their motivation. Our results show that how children are asked to use computers does in fact affect their achievement. Grouping children around a single computer can have a positive effect on both achievement and motivation compared to having children play on their own computers. The technology theme of the research focuses on extending computer technology from single-user computers to technology more suited to supporting collaboration in a multi-user environment. We modified the computer environment (both the hardware and the software) to allow the addition of a second mouse to see how this change would affect the children's achievement, learning, and behaviour while playing a puzzle-solving game collaboratively. The results show that the addition of a second mouse to the computer can positively affect children's achievement and learning in the game as well as the temporal patterns of who controls the mouse. The interface design theme of the research focuses on the usability of the graphical user interfaces found in children's software. Even if we understood how to structure the computer environment in the classroom, and we knew how to modify the computer to support children's collaboration, our learning environments might still be ineffective if we are not careful with the design of the low-level details of the user interface. We examined children's use of two common mouse-interaction techniques, drag-and-drop and point-and-click, to see whether the choice of mouse interaction style affects children's ability to move objects around on the screen. The results show that children are able to perform a point-and-click movement faster and with fewer errors than with a drag-and-drop movement and that more children prefer the point-and-click interaction style over the drag-and-drop interaction style. When these two mouse-interaction styles are used in a commercial puzzle-solving environment our studies reveal that the choice of interaction style can affect both achievement and motivation. While many children adapt to the user interfaces with which they are presented, our results show how even a widely accepted interaction style such as drag-and-drop can be difficult for some children and can affect motivation and achievement in a learning environment. Gender differences were observed in all stages of the research, which strengthens the conventional wisdom that girls and boys often interact differently with technology. We emphasize the need to be sensitive to these differences and we provide specific recommendations in this regard.
25404546 bytes
application/pdf
Adapting the Human-Computer Interface to Support Collaborative Learning Environments for Children by Kori Marie Inkpen B.Sc, Dalhousie University, 1992 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Doc to r of Phi losophy in THE FACULTY OF GRADUATE STUDIES (Department of Computer Science) We accept this thesis as conforming to the required standard/ The University of British Columbia August 1997 © Kori Marie Inkpen, 1997 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of Campf/feT S c / f /XV The University of British Columbia Vancouver, Canada Date v 5 £ y E r , isj?l DE-6 (2/88) Abstract The presence of computers in schools has grown tremendously over the last ten years. In the wake of this enormous growth, sound research on how to effectively de-sign learning environments and successfully integrate computers into the classroom is needed. The research described in this dissertation evaluates computer-based col-laborative learning environments for children using three important criteria: (a) the social environment in which the technology is placed, (b) the technology that pro-vides for explicit collaboration, and (c) the low-level interface design. An additional focus of the research, which crosses all three themes, is gender. The research com-prised three experimental studies that were conducted in the three research themes. A l l these studies employed a creative problem-solving game as the research vehicle. The social theme of the research focuses on the interactions between children mediated by computers. We examined whether the ways children were assigned to work on computers affected their achievement and their motivation. Our results show that how children are asked to use computers does in fact affect their achieve-ment. Grouping children around a single computer can have a positive effect on both achievement and motivation compared to having children play on their own computers. The technology theme of the research focuses on extending computer tech-nology from single-user computers to technology more suited to supporting collab-oration in a multi-user environment. We modified the computer environment (both the hardware and the software) to allow the addition of a second mouse to see how this change would affect the children's achievement, learning, and behaviour while playing a puzzle-solving game collaboratively. The results show that the addition ii of a second mouse to the computer can positively affect children's achievement and learning in the game as well as the temporal patterns of who controls the mouse. The interface design theme of the research focuses on the usability of the graphical user interfaces found in children's software. Even if we understood how to structure the computer environment in the classroom, and we knew how to modify the computer to support children's collaboration, our learning environments might still be ineffective if we are not careful with the design of the low-level details of the user interface. We examined children's use of two common mouse-interaction techniques, drag-and-drop and point-and-click, to see whether the choice of mouse interaction style affects children's ability to move objects around on the screen. The results show that children are able to perform a point-and-click movement faster and with fewer errors than with a drag-and-drop movement and that more children prefer the point-and-click interaction style over the drag-and-drop interaction style. When these two mouse-interaction styles are used in a commercial puzzle-solving environment our studies reveal that the choice of interaction style can affect both achievement and motivation. While many children adapt to the user interfaces with which they are presented, our results show how even a widely accepted inter-action style such as drag-and-drop can be difficult for some children and can affect motivation and achievement in a learning environment. Gender differences were observed in all stages of the research, which strength-ens the conventional wisdom that girls and boys often interact differently with tech-nology. We emphasize the need to be sensitive to these differences and we provide specific recommendations in this regard. iii Contents Abstract ii Contents iv List of Tables viii List of Figures x Acknowledgements xiii Dedication xvi 1 Introduction 1 2 Background Research 9 2.1 Cooperative Learning 11 2.2 Cooperative Uses of Computers in Education 15 2.3 Computer Supported Collaborative Learning 17 2.4 Computer Supported Cooperative Work 19 2.5 Human-Computer Interaction Issues for Children in Education . . . 21 2.6 Gender Issues for Computers and Technology 23 2.7 Electronic Games for Education 25 2.8 The E-GEMS Research Program 27 2.9 Early E-GEMS Research: Science World, 1993 29 2.9.1 Method 31 2.9.2 Results and Discussion 33 iv 3 The Incredible Machine 38 3.1 Playing the Game 40 3.2 Sample Puzzle Solutions 46 3.2.1 Solution to Puzzle #1 46 3.2.2 Solution to Puzzle #2 52 3.2.3 Solution to Puzzle #3 53 3.3 IBM-Compatible Version vs. Macintosh Version 58 3.3.1 Moving Regular Objects 59 3.3.2 Moving Connector Objects 61 3.4 A Testbed Collaborative Learning Environment 66 4 Exploring Collaborative Play 68 4.1 Phase 1: Kerrisdale School, 1994 71 4.1.1 Method 71 4.1.2 Results and Discussion 78 4.2 Phase 2: Science World, 1994 92 4.2.1 Method 92 4.2.2 Results and Discussion 95 4.3 Conclusion 99 5 Multiple Input Devices 103 5.1 Phase 1: Science World, 1994 '. 105 5.1.1 Method 105 5.1.2 Results and Discussion 110 5.2 Phase 2: Vancouver Elementary Schools, 1996 114 5.2.1 Method 115 5.2.2 Results and Discussion 122 5.3 Conclusion 131 v 6 Drag-and-Drop vs. Point-and-Click 136 6.1 Phase 1: Mouse Interaction Style in The Incredible Machine 140 6.1.1 Method 140 6.1.2 Results 143 6.1.3 Discussion 144 6.2 Phase 2: Mouse Interaction Style in a Simplified Setting 146 6.2.1 Method 147 6.2.2 Results 157 6.2.3 Discussion 163 6.2.4 Issues Related to Experimental Design 168 6.3 Conclusion 175 7 A Research Agenda 180 7.1 How Many Computers? 181 7.1.1 Research Goal #1: Situated Observations 181 7.1.2 Research Goal #2: Shared Virtual Spaces 182 7.1.3 Research Goal #3: Collaboration at a Distance 182 7.2 How Many Input Devices? 182 7.2.1 Research Goal #4: Concurrent Collaboration 183 7.2.2 Research Goal #5: Longitudinal Studies 183 7.2.3 Research Goal #6: Two-Mouse Protocols at a Distance . . . 183 7.3 How Do We Measure Learning? 184 7.3.1 Research Goal #7: Alternative Measures of Achievement and Learning 185 7.3.2 Research Goal #8: Extended Collaborative Sessions 185 7.3.3 Research Goal #9: Structured Collaboration 185 7.4 How Do We Facilitate Children's Interactions With Computers? . . 186 7.4.1 Research Goal #10: Alternative Input Devices 186 7.4.2 Research Goal #11: Naturalistic Observations 187 vi 7.4.3 Research Goal #12: Ubiquitous Computing 187 7.5 How Do We Deal With Gender Differences? 187 7.5.1 Research Goal #13: Observations in Varied Environments . . 188 Appendix A 199 A. l Sample Questionnaire - Kerrisdale 1994 199 Appendix B 201 B. l Sample Video Transcript - Kerrisdale 1994 201 vii List of Tables 4.1 Collaborative Play I: children per experimental condition 76 4.2 Collaborative Play I: observations per experimental condition . . . . 76 4.3 Collaborative Play I: mean number of puzzles solved 78 4.4 Collaborative Play I: children's playing preference 80 4.5 Collaborative Play I: number of exchanges, requests and refusals . . 83 4.6 Collaborative Play II: observations per experimental condition . . . 95 4.7 Collaborative Play II: mean number of puzzles solved 96 5.1 Multiple Mice I: children per experimental condition 109 5.2 Multiple Mice I: mean number of puzzles solved 110 5.3 Multiple Mice I: mean number of exchanges 112 5.4 Multiple Mice II: dyads per experimental condition 121 5.5 Multiple Mice II: mean number of puzzles solved in collaborative sessionl23 5.6 Multiple Mice II: mean number of puzzles solved in the followup solo session 123 5.7 Multiple Mice II: mouse control time and number of exchanges . . . 125 5.8 Multiple Mice II: mean number of puzzles solved by children who had control of the mouse less and children who had control of the mouse more 126 5.9 Multiple Mice II: children's ranking of playing preference by condition 129 5.10 Multiple Mice II: percentage of children who preferred to play either alone or with a friend 131 6.1 Interaction Style I: mean number of puzzles solved 143 viii 6.2 Interaction Style II: average movement times 158 6.3 Interaction Style II: average number of errors 158 6.4 Interaction Style II: Fitts's Law analysis 160 6.5 Interaction Style II: children's preference of interaction style 163 6.6 Interaction Style II: percentage of children who preferred each inter-action style 163 ix List of Figures 2.1 Velcro covered box with manipulatives used as part of the design station 31 2.2 A child's game design drawing 32 3.1 The Incredible Machine Control screen 41 3.2 Parts of the game screen 42 3.3 Run machine icon used to start the machine 43 3.4 Objects used in The Incredible Machine 44 3.5 Flipping an object 45 3.6 Re-sizing an object 45 3.7 Recycling and object 46 3.8 Attaching a connector object such as an elastic 47 3.9 Puzzle #1: Initial configuration 48 3.10 Puzzle #1: Step 1 49 3.11 Puzzle #1: Step 2 50 3.12 Puzzle #1: Step 3 51 3.13 Puzzle #1: Running the solution 51 3.14 Dialog box that appears after completing a puzzle 52 3.15 Puzzle #2: Initial configuration 53 3.16 Puzzle #2: Step 1 54 3.17 Puzzle #2: Step 2 54 3.18 Puzzle #2: Running the solution 55 3.19 Puzzle #3: Initial configuration 56 3.20 Puzzle #3: Step 1 57 x 3.21 Puzzle #3: Step 2 57 3.22 Puzzle #3: Running the solution 58 3.23 State transition diagrams for placing regular objects in The Incredible Machine 62 3.24 State transition diagrams for placing connector objects in The In-credible Machine 65 4.1 Parallel and Integrated Play conditions 72 4.2 Video equipment configuration 74 4.3 The snack/questionnaire area of the experiment room 74 4.4 Mean number of puzzles solved in the Solo, Parallel and Integrated conditions in phase I 79 4.5 Mean number of puzzles solved in the Integrated and Parallel condi-tions in phase I 81 4.6 The number of exchanges, requests and refusals for each pair . . . . 84 4.7 Requests for mouse control 86 4.8 Requests for mouse control in mixed pairs 87 4.9 How often girls and boys reported they used computers 88 4.10 How often girls and boys reported they played video and computer games 88 4.11 Mean number of puzzles solved for the Solo, Parallel and Integrated conditions in phase II 97 4.12 Mean number of puzzles solved for the Parallel and Integrated con-ditions in phase II 98 4.13 Number of children who left early 99 5.1 Computer setup for the two-mouse conditions 107 5.2 Children playing with the two-mouse configuration 108 xi 5.3 Mean number of puzzles solved by children in the Shared, Give and Take conditions in Phase I I l l 5.4 Videotape of children's hands . . . 116 5.5 Interface training task 119 5.6 Pinwheels used to rate children's preferences for playing alone or with a friend 120 5.7 Mean number of puzzles solved by the children who had control of the mouse less than their partner and the children who had control of the mouse more than their partner 127 5.8 Children's preferences for playing alone or with a partner 130 6.1 Time intervals when girls left the session 145 6.2 How often girls and boys reported that they played computer or video games 148 6.3 Initial screen configuration for the drag-and-drop and point-and-click software 149 6.4 Visual feedback from the drag-and-drop and point-and-click software 151 6.5 State transition diagrams for the drag-and-drop and the point-and-click interaction styles 154 6.6 Pinwheels used to rate children's preference of interaction style . . . 156 6.7 Children's ranking of preferred interaction style after using the drag-and-drop interaction style first 162 6.8 Children's ranking of preferred interaction style after using the point-and-click interaction style first 162 6.9 Children's improvement over experiment blocks 170 6.10 State transition diagrams for the two implementations of point-and-click 174 xii Acknowledgements It is hard to believe that this dissertation is actually finished. It is even harder to believe that I have a Ph.D. I had no intention of even going to university when I finished high school, then, I had no intention of spending more than three years at university, then, I decided a Masters was ok but I couldn't do a Ph.D. (which Maria never believed and never took "no" for an answer!). There are many people who have helped me reach this milestone, too many to mention them all here. The two people that contributed most to this chapter of my life are my two supervisors, Dr. Kelly Booth and Dr. Maria Klawe. I couldn't have asked for a better team of supervisors. They were always supportive, pushing me to be the best that I could be. Maria deserves the credit for getting me to do a Ph.D. and Kelly deserves the credit for helping me write coherently. They have taught me so much, not only about research but about all aspects of my career. I need to thank them both for being there everytime I needed them, normally on very short notice. Somewhere along this path they went from being my supervisors to being my colleagues and most importantly, my friends. Special thanks also goes to the rest of my supervisory committee, Dr. Shelley Hymel, Dr. Richard Rosenberg and Dr. Rena Upitis, for support in many ways including reading drafts, bringing me up to speed on research in other disciplines, helping me with my experimental design, and helping me with my writing. Thanks also goes to Dr. Izak Benbasat and Dr. David Poole who served as the University Examiners, Dr. Steven Tanimoto who served as the External Examiner and Jim Boritz who was my student reader. Support for this research was given in part by the Natural Sciences and En-xiii gineering Research Council of Canada, through postgraduate scholarships, various research grants, and a Cooperative R&D project grant. Funding was also provided by the TeleLearning NCE, the Advanced Systems Institute, Electronic Arts Ltd., and Apple Canada. Special thanks also goes to Science World BC for providing a venue for several pieces of this research, to the Vancouver School Board, and to the many schools, teachers, and children who took part in my research. I also owe thanks to my many friends and colleagues at UBC who helped me with my research, my social life, and never minded being asked to buy a raf-fle ticket. To members of the E-GEMS team who gave "sweat labour" carrying equipment between UBC, Science World, and schools. To members of the Imager group and other students in the department who listened to my many dry-runs. To the faculty members who were there when I needed advice, and to the departmen-tal support staff who were always willing to help. In particular, special thanks to Joanna McGrenere for-collaborating on the design for the two-mouse study and to Steve Gribble for the initial implementation for the two-mouse study. I owe so much to my family, especially my parents Tom and Kay. For being there for me, supporting me in everything I did, for understanding that I had to move to the other side of the country to do my Ph.D. and most of all for believing in me and instilling in me the belief that I could accomplish anything I put my mind to. To my sister Wendi, for all those nights I sat on her bed talking, because I couldn't study for the next day's exam (which meant she couldn't study either). And to my brother Randy, who always looked out for me and could help solve any problem I ran into. Finally, most of all, I owe everything to my husband Jeff. He left Nova Scotia to share a life with me, putting up with me through all of my late nights, proofreading my writing, bringing me snacks when I didn't have time to leave school, but most of all, for being there when I needed him, supporting me all the way. Jeff you are my past, my present, but most of all my future. xiv This dissertation is dedicated to my parents Tom and Kay Inkpen xv C h a p t e r 1 Introduction Apple eMate 300 Mobile Computer In our schools, every classroom in America must be connected to the information superhighway, with computers and good software, and well-trained teachers. - U.S. President William Clinton State of the Union Address, January 23, 1996 1 The presence of computers in schools has grown tremendously over the last ten years. Estimates on the number of computers in American schools have risen from 2.4 million in 1989 to over 5.5 million computers in 1995 [Software Publish-ers Association, 1995]. In the 1993-1994 school year, $565 million was spent in the United States on instructional software and almost $1 billion was spent on all technology-based products [Software Publishers Association, 1995]. The use of tech-nology in education has become a top priority for many countries, Canada included. In the wake of this enormous growth, sound research on how to effectively design learning environments and successfully integrate computers into the classroom is needed. Ready or not here we come! Whether or not we believe technology-rich classrooms are a good thing, North Amer-ica is headed in that direction. It is up to researchers in Computer Science, Educa-tion, Psychology, and related disciplines to investigate issues surrounding the use of computers in the classroom in order to understand how to maximize the potential of the technology in terms of social and academic priorities. The research described in this dissertation evaluates computer-based collab-orative learning environments for children using three important criteria: (a) the social environment in which the technology is placed, (b) the technology that pro-vides for explicit collaboration, and (c) the low-level interface design. An additional focus of the research, which crosses all three themes, is gender. Previous research (reviewed in Chapter 2) has shown that girls and boys often interact differently with technology. Any research concerning the use of technology in education must be sensitive to potential gender differences. The social theme of our research focuses on the interactions between children mediated by computers. We are interested in the role that computers play within the social environment of a classroom and how structuring children's use of computers 2 impacts the effectiveness of the learning environment within that social setting. The technology theme of our research focuses on extending computer technology from single-user computers to technology more suited to supporting collaboration in a multi-user environment. We are interested in whether providing multiple input devices impacts children's sharing patterns and the effectiveness of computer-based collaborative environments. The interface design theme of our research focuses on the usability of the graphical user interfaces found in children's software. We are interested in how low-level interface design choices affect children's performance and outcomes in computer-based learning environments. The next three sections summarize the research methods and the results obtained in each of these themes. The final section of this chapter provides an overview of the remainder of the dissertation. Adapting the social environment In 1995 the student-to-computer ratio in North American classrooms was estimated to be nine to one [Software Publishers Association, 1995]. If the computers are dis-tributed across classrooms, often it means that there are only one or two computers per classroom [Becker, 1994]. If the computers are placed in a central location in the school, such as a computer lab, then a larger number of computers are available, but this still does not guarantee a computer for every child. As a result, teachers must frequently decide how to coordinate the children's use of the computers in either the classroom or the lab. Possible choices are to have only a subset of the class use the computers at any given time or to instead ask small groups of children to work together on a single computer. Our initial research examined whether the ways children were assigned to work on computers affected their achievement and their motivation. We observed children playing an educational puzzle-solving game either alone or with a partner. The children that played with a partner either played together on one computer 3 or side-by-side on two separate computers. We were interested in determining if the children's playing configuration affected their achievement in the puzzle-solving game or their motivation to play the game. Our results show that how children are asked to use computers does in fact affect their achievement in the puzzle-solving environment as well as their motiva-tion to play the game. Girls were able to solve significantly more puzzles in the game when playing with a partner on a single computer as opposed to when they played alone. Girls also tended to solve more puzzles when playing together on one com-puter as opposed to when they played side-by-side on two separate computers. In fact, girls playing side-by-side on two separate computers solved even fewer puzzles on average than did girls who played by themselves. Boys playing the game with a partner on the same computer solved more puzzles on average than did either boys playing alone or boys playing side-by-side with a partner on separate computers. The choice of playing configuration also significantly affected girls' motiva-tion to play the game. Ninety-five percent of girls playing with a partner on the same computer chose to play for the full thirty-minute session that was offered, compared to only eighty-three percent of girls who played side-by-side on separate computers and seventy-six percent of girls who played by themselves. For boys, a higher percentage chose to stay for the full thirty-minute session when playing with a partner as opposed to when playing alone, although this difference was not statistically significant. These results suggest that grouping children around a single computer can have a positive effect on both achievement and motivation compared to having children play on their own computers. Chapter 4 presents a detailed account of this research. 4 Adapting the computer environment If we want to group children together to work on a single computer as recommended in the previous section, we need to be aware of difficulties that can occur when chil-dren share the input devices for the computer. The computers found in most schools have been designed for individual users. As a consequence of this design goal, these computers often do not support multiple children's interactions. This limitation could impact the effectiveness of having children use computers collaboratively un-less explicit support for collaboration and sharing is provided. We modified the computer environment (both the hardware and the soft-ware) to allow the addition of a second mouse to see how this change would affect the children's achievement, learning, and behaviour while playing a puzzle-solving game collaboratively. Children played the game with a partner using a computer that was equipped with either one or two mice. When two mice were used, the chil-dren were still required to take turns interacting with the game. In the two-mouse configurations, only one mouse was active at a time, but the children could switch control between the two mice by pressing one of the mouse buttons. Two protocols for switching control between the mice were examined: "give" and "take". Using the give protocol, the child in control of the game initiated the transfer of control, while in the take protocol the child not in control of the game initiated the transfer of control. Our results show that the addition of a second mouse to the computer can positively affect children's achievement and learning in the game as well as the temporal patterns of who controls the mouse. For girls, the addition of a second mouse using a give protocol to transfer control between the two mice resulted in a slightly more equal distribution of mouse time and the girls were able to solve significantly more puzzles than were girls in the other two conditions. For boys, the addition of a second mouse using a take protocol to transfer control between the two mice had a significant effect on the distribution of mouse time: boys playing 5 with the two-mouse take setup had a significantly more equal distribution of mouse time than did boys in either of the other two conditions. This may have impacted boys' learning because a correlation was found between the amount of time a boy had control of the mouse in his collaborative session and how many puzzles he was able to solve in a later session in which he played the game on his own. The results of this research demonstrate that redesigning the computer for the purpose of supporting children's collaboration can bring about positive changes in achievement and perhaps learning. Chapter 5 presents a detailed account of this research. Adapting interface design Even if we understood how to structure the computer environment in the classroom, and we knew how to modify the computer to support children's collaboration, our learning environments might still be ineffective if we are not careful with the design of the low-level details of the user interface. With approximately $565 million dollars spent on instructional software in 1995 in the U.S., the lack of solid research on effective interface design strategies for children presents a troublesome picture. We examined children's use of two common mouse-interaction techniques, drag-and-drop and point-and-click, to see whether the choice of mouse interaction style affects children's ability to move objects around on the screen. In addition, we investigated the potential impact of using these interaction styles in a learning environment. Our results show that children are able to perform a point-and-click move-ment faster and with fewer errors than with a drag-and-drop movement and that more children prefer the point-and-click interaction style over the drag-and-drop interaction style. For girls, the differences in movement times and number of errors between the two interaction styles were statistically significant and more girls pre-ferred the point-and-click interaction style. Similar trends were observed for boys, 6 although the results were not statistically significant. When these two mouse-interaction styles are used in a commercial puzzle-solving environment our studies reveal that the choice of interaction style can affect both achievement and motivation. Girls who played an IBM-compatible version of the game that utilizes a point-and-click interaction style were able to solve signifi-cantly more puzzles and were more motivated to play than were girls who played a Macintosh version of the same game that utilizes a drag-and-drop interaction style. Due to time constraints, only girls were studied in this phase of the research. While many children adapt to the user interfaces with which they are pre-sented, our results show how even a widely accepted interaction style such as drag-and-drop can be difficult for some children and can affect motivation and achieve-ment in a learning environment. This is an important issue when designing educa-tional software for use in the classroom because the software must be accessible to all children. Chapter 6 presents a detailed account of this research. Overview of the dissertation As the previous sections suggest, integrating technology into the classroom requires more than simply coming up with the money to purchase the equipment, successful learning environments involve more than just presenting educational material on a computer, and supporting children's collaborative use of computers is more than assigning a group of children to work on the same computer. Developing effective ways of using computer technology in the classroom encompasses many levels and must be approached from a variety of perspectives. The research presented in this dissertation provides recommendations for: (a) structuring children's use of computers in the classroom to take into account chil-dren's natural social interactions in a technology-rich environment, (b) enhancing children's collaborative use of a computer through the addition of explicit support for collaboration by augmenting both the hardware and the software, and (c) deter-7 mining the type of mouse-interaction style to use when developing a graphical user interface for children to use in learning environments. Because gender differences were observed in all stages of the research, which strengthens the conventional wis-dom that girls and boys often interact differently with technology, we emphasize the need to be sensitive to these differences and we provide specific recommendations in this regard. Our research is a first step in these areas. In drawing our conclusions, we stress the necessity for continued research in this area, focusing on all aspects of children's interactions with the computer: at the social level, at the technology level, and at the interface design level. The remaining chapters in this dissertation describe, in detail, the research undertaken with respect to the three themes introduced here. Chapter 2 summa-rizes background research from a variety of disciplines that impacts on the research presented in this dissertation. Chapter 3 describes The Incredible Machine software, the creative problem-solving game used throughout our research. Chapters 4, 5, and 6 describe the three experimental studies that were conducted in the three research themes. Chapter 7 concludes the dissertation with an agenda for future research in Computer Science as well as in the related disciplines of Education and Psychology. 8 Chapter 2 Background Research Disciplines are like cultures: for disciplines to work well together they must learn to appreciate one another's language, transitions, and values. . . . human-computer interface design is a particularly interdisciplinary field. For interface design to thrive, many disciplines must cooperate. - Scott Kim Interdisciplinary Cooperation, 1990, page 32 9 Computer support for collaborative learning environments is a new area of re-search, yet research has been conducted in related areas for decades. This stems from the interdisciplinary nature of the area. The study of cooperative learning began in the disciplines of Education and Psychology over sixty years ago. The use of com-puters in the classroom began in the late 1970's and early 1980's. Study of the use of computers for cooperative learning arose from this previous research in Psychology and Education. Meanwhile, computer scientists, psychologists and human factors specialists began studying issues related to how people interact with computers. A sub-field of Human-Computer Interaction called Computer Supported Cooperative Work evolved and focused on ways to support group collaborations with computer technology. A further specialization of this area, Computer Supported Cooperative Learning, focuses on issues related to collaborative learning using computers. With all of this research, why is the design of computer support for collab-orative learning environments a new area of research? One answer is that it is the combination of all of its component research areas that makes this area unique. It is this realization that led to the formation of the Electronic Games for Education in Math and Science (E-GEMS) research program that began in 1992. E-GEMS, led by Dr. Maria Klawe at the University of British Columbia, is a multidisciplinary research initiative that examines issues related to the use of electronic games as a way of motivating children to explore and learn math and science concepts (for a more complete description of the E-GEMS program see Section 2.8). In this chapter the following research areas will be examined with regard to previous literature that relates to computer supported collaborative learning environments: • Cooperative Learning • Cooperative Uses of Computers in Education • Computer Supported Collaborative Learning • Computer Supported Cooperative Work 10 • Human-Computer Interaction Issues for Children in Education • Gender Issues for Computers and Technology • Electronic Games for Education Following this, a description of the Electronic Games for Education in Math and Science (E-GEMS) research program is provided along with a summary of re-sults from initial E-GEMS studies that provide a framework for the subsequent research presented in this dissertation. 2.1 Cooperative Learning When considering ways to support children's collaboration in a computer environ-ment it is important to consider previous research on cooperative learning. By understanding this body of work and interpreting it in the context of a computer environment, we gain new insight into ways of using technology to support children's collaboration in computer-based learning environments. Achievement Benefits Since the 1920's, there have been numerous studies on the effects of cooperative learning on achievement. Slavin [1990] reviewed 66 previous studies on cooperative learning. Of these, 72% showed cooperative learning to have a positive effect on achievement while only 8% favored a control group with no collaboration. Johnson, Maruyana, Johnson, Nelson, and Skon [1981] performed a meta-analysis of 122 studies in an attempt to unify the vast body of research. The main conclusion of this meta-analysis was that cooperation is more effective at increasing achievement and productivity than are competition or individual effort. Both of these reviews provide strong support for the claim that cooperative learning can improve achievement. What is less clear in the literature is how to 11 structure cooperative learning to achieve the greatest impact. A high degree of vari-ability is observed across studies with respect to the effect cooperative learning had on achievement, depending primarily on the structure of the cooperative learning situation. Hymel, Zinc, and Ditner [1993], in a review of the cooperative learning litera-ture, attempted to consolidate the previous literature. They presented five elements that are necessary to achieve a successful cooperative learning environment. These were: Positive Interdependence: each child can only succeed if all the members of the group succeed in learning Face-to-Face Interaction: children must engage in direct, verbal interactions in order to truly cooperate Individual Accountability: each child in the group must contribute an equal share of work to the project, and each child must be responsible for learning the assigned material Social Skills Training: children must be taught the necessary collaborative skills Group Evaluation Opportunities: children must be given opportunities to evaluate and discuss among themselves just how the group is functioning and ways in which it could be improved Hymel et al. also noted that although the above elements are necessary for effec-tive cooperative learning, many of the strategies developed vary in the ways these elements are accomplished. In particular, many studies varied in their methods of promoting positive interdependence which "can be achieved through the establish-ment of interdependent goals, tasks, resources, and/or roles" [p. 116]. 12 Cohen [1994] further delineates the factors which enhance cooperative learn-ing by examining variations as a function of the tasks themselves and the nature of the interaction which takes place. Specifically, Cohen distinguished between tasks that have fairly clear procedures with "right-answers" and tasks that are "i l l -structured problems", often having multiple solutions. With respect to these two categories of tasks, the role of interaction is more critical for achievement gains in an ill-structured task than in a task that is more clear-cut and could be carried out by individuals. Therefore, factors that affect the amount and richness of interaction will affect productivity for ill-structured tasks. Although many researchers do not yet agree on the key ingredients for suc-cessful cooperative learning, it is important to note that "in general, for academic achievement, cooperative learning techniques are no worse than traditional tech-niques and in most cases they are significantly better" [Slavin, 1980, p.337]. Social Benefits As well as the potential for benefits in academic achievement, cooperative learn-ing has also demonstrated social benefits for students. The research overwhelmingly supports the existence of positive effects for children in terms of several social factors such as self-esteem, attitude towards peers, attitudes towards school work, motiva-tion, helping and sharing, and acceptance of and mutual concern for people who are different from themselves [Hymel et al., 1993; Johnson and Ahlgren, 1976; John-son and Johnson, 1986; Male, 1986; Sharan, 1980; Simpson, 1986; Slavin, 1980; Slavin, 1984]. "It is important to note that these non-cognitive outcomes do not appear to depend to the same extent on particular incentive or task structures, and for many practical applications these outcomes might justify use of cooperative learning methods as long as they do not reduce student achievement" [Slavin, 1984, p.62]. These outcomes are very important for the development of children and they 13 alone are a good argument for cooperative learning even if cognitive improvements are not observed. It has been suggested that an improvement in students' moods and attitudes will in turn increase motivation for academic achievement [Johnson and Ahlgren, 1976; Slavin, 1984]. Curr icu lum Development There is a current movement towards more project-based work in schools, so that children are active-problem solvers and theorists [Koschmann, Newman, Woodruff, Pea, and Rowley, 1993]. Collaborative group-based work is one way of implement-ing this type of learning. Curriculum development standards strongly recommend increasing the number of cooperative group learning situations. For example, the curriculum and assessment standards for school mathematics, prepared by the Na-tional Council of Teachers of Mathematics (NCTM), emphasizes the importance of group work for children in Grade Five through Grade Eight. Working in small groups provides students with opportunities to talk about ideas and listen to their peers, enables teachers to interact more closely with students, takes positive advantage of social characteristics of the middle school student, and provides opportunities for students to exchange ideas and hence develops their ability to communicate and reason. [National Council of Teachers of Mathematics, 1989, p.87] In addition, the standards also encourage movement from instructional teaching to project-based learning where students are active problem-solvers and theorists. The N C T M standards were developed in the United States but have been widely adopted in Canada and other countries. 14 2.2 Cooperative Uses of Computers in Education The use of computers in education has grown from computer-aided instruction sys-tems, predominant in the 1960's, to intelligent tutoring systems in the 1970's and then to microworlds in the 1980's. Today's terminology for computer applications in the classroom is "Interactive Learning Environments" (ILE). Hakansson defines an interactive learning environment as an "engaging, entertaining, interactive computer game in which a learning concept is embedded" [Rheingold, 1983, p.39]. Some researchers believe that the use of computers can have a positive effect on students' achievement [Kulik, Bangert, and Williams, 1983], while others see the computer as a vehicle for content delivery that does not in itself affect achievement [Clark, 1983]. Other researchers argue that because computers have been shown to raise children's motivation and engagement, this in itself could lead to greater academic achievement [Krendl and Lieberman, 1988]. Computers may also provide a way for children to explore concepts that would otherwise be impossible for them to explore [Papert, 1980]. Extending cooperative learning to include children collaborating on comput-ers is a natural next step because of the benefits that cooperative learning provides and the social role that computers often play in the classroom. Research has shown that students exhibited more peer interaction when they worked on computer ac-tivities than when they worked on non-computer activities [Hawkins, Sheingold, Gear hart, and Berger, 1982]. Other researchers observed that children "sponta-neously gather in social groups of two or more to use a computer" [Strommen, 1993, p.46] and that they appeared to enjoy working together in this type of environment [Hawkins et al., 1982; James, 1989; Nastasi and Clements, 1993; Simpson, 1986; Watson, 1991; Webb, 1984]. Researchers and teachers have also noticed that even when students are assigned to individual computers, they often end up working in pairs and trios [Watson, 1991]. Other researchers have had a more skeptical view of children working together 15 on computers. In describing research by Emihovich and Miller [1988], Krendl and Lieberman [1988] remarked that sometimes collaboration consists of children monitoring each other's behavior in a rather superficial and uninvolved manner, such as detecting and correct-ing only minor errors. Usually they tend to take turns on the computer, instead of working together in a truly collaborative way. [p.373] Research on cooperative learning with computers began in the late 1970's and early 1980's. At this time Johnson and Johnson noticed that "[h]ow students interact with each other is a neglected aspect of computer-assisted instruction" [Johnson and Johnson, 1986, p.12]. Many of the researchers studying cooperative learning were interested in whether the benefits of cooperative learning in the classroom would transfer to learning in a computer environment. Because cooperative learning tended to promote higher achievement [Johnson et al., 1981], it is possible that computer-assisted cooperative instruction would also have a positive effect on achievement [Johnson, Johnson, and Stanne, 1986]. Since these early reports, many researchers have noted the advantages of cooperative, computer-aided instruction on achievement [Dalton, Hannafin, and Hooper, 1989; Johnson, Johnson, and Stanne, 1985; Krendl and Lieberman, 1988; Kulik, Bangert, and Williams, 1983; Mevarech, Stern, and Levita, 1987; Trow-bridge, 1987], efficiency [Okey and Majer, 1976], and the enhancement of social skills [Fink, 1990]. As educational software has progressed from applications that merely present educational material, to more interactive exploratory environments, the benefits of cooperation are still reported [Repman, 1993; Strommen, 1993], but some researchers argue that the effects on achievement are minimal [Becker, 1992; Carrier and Sales, 1987]. Reasons for these differences may include the type of soft-ware used [Anderson, Tolmie, McAteer, and Demissie, 1993], the implementation of the cooperative structure, or individual differences among the students using the system [Carrier and Sales, 1987]. 16 2.3 Computer Supported Collaborative Learning The research surveyed in the previous section involved utilizing cooperative learning principles using computers as the medium for the work. The computers were merely a vehicle for the instruction. In contrast to this, the research area of Computer Supported Collaborative Learning (CSCL) focuses on using the computer as an integral part of children's collaboration in a learning environment. Technology can be used in the classroom in two major ways: (a) as an in-structional delivery tool where the technology carries or delivers the knowledge to be learned, or (b) as a tool for knowledge construction [O'Malley and Koschmann, 1993]. Roschelle [1992] defines collaborative technology as a "tool that enables individuals to jointly engage in active production of knowledge" [p.40]. CSCL appli-cations tend to emphasize access to learning materials as opposed to the traditional use of computers in the classroom as a tool for the delivery of instruction. This approach uses learner-centered techniques to support individuals' construction of understanding [Koschmann et al., 1993]. Research in CSCL can be separated on two dimensions: (a) locus of use: within classrooms, across classrooms, or in virtual classrooms; and (b) focus of research: technology or pedagogy. Our research focused on both technological and pedagogical issues within classroom environments, predominantly elementary and secondary, concentrating on grades 4-8. CSCL research within the classroom has examined issues such as the support of children's communication and knowl-edge building [Guzdial, Turns, Rappin, and Carlson, 1995; Scardamalia, Bereiter, McLean, Swallow, and Woodruff, 1989], collaboration on shared documents [Gold-man, Moschkovich, and through Applications Project Team, 1995; Mitchell, Posner, and Baecker, 1995], and the design of technology for classrooms [Koschmann, Fel-tovich, Myers, and Barrows, 1992; Norman, 1997]. Collaboration can vary over many dimensions including face-to-face versus at a distance, and synchronous versus asynchronous. Other researchers have noted 17 the positive interactions that occur when multiple children are working together on a single computer [Bricker, Tanimoto, Rothenberg, Hutama, and Wong, 1995; Newman, 1988; Roschelle, 1992; Steiner and Moher, 1994; Stewart, 1997]. Our focus dealt with children collaborating face-to-face in close proximity to each other. An early design of a collaborative system uncovered key issues concerning children's collaborative interactions on computers in an educational setting: The misconceptions displayed in the interviews suggest that the sys-tem we developed for supporting collaborative work in these sixth grade classrooms requires some redesign. The misconceptions did not arise from any deep misunderstanding of the nature of collaborative work. On the contrary, it appears that the system and its interface did not always correctly represent the organization of the collaborative activ-ities which, in sixth grade classrooms, have properties different from those found in research settings on which the system was modelled. Co-present work groups are one such feature. School children do not have their own workstation let alone their own offices. Collaborations in the sixth grade usually happen while students are gathered around a single computer and other project materials. This is not just for lack of space and computers. [Newman, 1988, p.302] One problem with supporting children's collaboration on a single computer is the fact that the computer has been designed for an individual user, normally with a small screen, a single keyboard and a single mouse. Having multiple children interact concurrently with this type of system is not supported on most machines. Bricker et al., [1995] have developed the Multiln module which supports multiple input pointing devices with their tracking cursors displayed on a single screen [Bier and Freeman, 1991; Bricker et al., 1995]. Another technology that has recently been suggested for use with children to allow for simultaneous collaboration is Pad++, a graphical sketchpad that provides facilities such as "portals" and "lenses" to support 18 collaboration [Bederson et al., 1986; Druin et al., 1997; Stewart, 1997]. An important consideration for research in CSCL is how computer supported collaborative learning environments will fit into the classrooms of today and how such classrooms will evolve over time. In terms of the future, the promise of having Internet technology within school systems as well as the continual decrease in the cost of hardware should bring about a much larger presence of computers in the classroom, and more opportunities for collaborative environments. The notion of the classroom as a computer lab with terminals on the desks of every student may be both unrealistic and undesirable. Researchers need to continue to investigate issues of design, implementation, evaluation, and integration for computer supported collaborative learning environments so we can make effective use of current and future technologies in the classroom. 2.4 Computer Supported Cooperative Work A parallel research area to Computer Supported Collaborative Learning is Com-puter Supported Cooperative Work (CSCW). This research area examines how peo-ple work together in groups and how computer technology can support those groups [Grudin, 1991]. Groups are different from individual users. It is important to under-stand the needs and requirements for group interactions. The software that supports CSCW systems is commonly referred to as groupware [Baecker, 1993]. In terms of user-interface issues for groupware, all of the existing problems of designing single-user systems apply in addition to new challenges arising from group interactions [Grudin, 1994]. Research areas in CSCW that relate to our interest in supporting children's face-to-face collaboration in a learning environment include: (a) the support of face-to-face interactions in a work environment, (b) issues related to shared workspaces, and (c) the design of new hardware to support collaboration in the workplace. The group dynamics for face-to-face interactions have been explored in the 19 CSCW literature by numerous researchers within the context of computer supported meeting environments. Such issues include room layout, placement of the people in the room and the sequencing of how users participate [Mantei, 1989]. Results of such research could provide insight on how to set up collaborative environments in classrooms and suggest ways of supporting multiple children's interactions with a single workstation. Shared workspaces play a significant role in having multiple people inter-act with a single system. The sharing and synchronization of available resources is a difficult problem to solve. Examining previous research on shared-workspaces for collaboration on networked computers can provide ideas on how to approach, or how not to approach, shared-screen issues. SASSE (Synchronous Asynchronous Structured Editor) [Baecker, Glass, Mitchell, and Posner, 1994] is one such tool that provides a collaborative writing environment using color and telepointers to distinguish between multiple users. SASSE also has a WYSIWIS (What-You-See-Is-What-I-See) option where both users have screens with synchronized views. An-other shared-workspace application that combines a face-to-face component is Team-WorkStation (TWS) [Ishii and Ohkubo, 1990]. TWS was designed to have a shared workspace which every member of the group could "see, point to and draw on si-multaneously" [p.131]. Several new devices have been developed to provide more natural ways of collaborating in both face-to-face and distributed meetings. One such device is the Liveboard [Elrod et al., 1992]. A Liveboard is a large interactive display used for a variety of interactions including group meetings and presentations. The designers of Liveboard recognized the importance of being able to interact with a central group display. Liveboard provides an environment where users can interact directly with the display through the use of a cordless pen. Adapting this type of device for children's use would allow collaboration on a "life-size" screen instead of having to crowd around a nine-inch monitor. Even beyond the ergonomic benefits of having a 20 display that groups of children can gather around easily, the device may also change the nature of the children's interactions. We can draw a great deal of knowledge from research within CSCW, but we must still be careful not to assume that what has been successful with adults in a "work" domain will automatically be successful for children in a "learning" environment. 2.5 Human-Computer Interaction Issues for Children in Education While it is important to investigate ways of supporting children's collaboration with computers, any collaborative environment would be ineffective if the children have difficulty interacting with the low-level interface details of the system. Human-Computer Interaction (HCI) is the study of how people communicate and interact with computer systems and how to design, build, and evaluate technologies to facil-itate those interactions. HCI research for the most part has focused on the use of technology in the setting where computers have been the most prominent, the workplace. As the presence of computers in society increases, so does the range of people that interact with the technology and the variety of contexts in which computers are used. It is important to investigate HCI issues for these new groups of users and new environ-ments. One area in which computer usage is becoming prominent is the classroom. It is therefore necessary to expand the HCI research agenda to include the use of computers in educational environments. It is also important to recognize children as a unique group of users and to understand how their interactions differ from those of adults. Childrens' perceptions and interactions with computers will also change as computers develop and as children's expertise with these systems increases. There-fore, studies on children's perceptions and interactions with computers from the 21 mid-1980's, such as Huff and Cooper's research in 1987, may not accurately charac-terize children of today and almost certainly not those of tomorrow. Over the last few years there has been an increased emphasis on the study of HCI in educational domains. Other than a few researchers such as Seymour Papert, researchers at the MIT Media Lab, and Alan Kay, previous research on the use of computers in the classroom was primarily interested in the content of educational software. Today, the scope has grown to include children's interactions with all aspects of the computer and its environment. With the increasing access children have to computer systems, it is important that research in this area continue in order for children to be able to use these systems effectively. While there has been little research on interface issues or interaction tech-niques specifically for children, the studies that have been performed indicate that children do have difficulty with some interaction techniques that are present in soft-ware designed for children. Strommen [1994] explored children's use of mouse-driven navigational techniques in a virtual environment. The results showed that children had difficulty maintaining pressure on the mouse button using the "hold and go" technique. The hold and go technique employed in the environment required the children to press and hold the mouse button down to begin movement, which would then be ceased when the mouse button was released. Because the children had dif-ficulty maintaining pressure on the mouse button they could not sustain movement. In some cases the children used two hands to hold the button down or they phys-ically picked the mouse up and held it, squeezing tightly on the button. Several of the children expressed fatigue either verbally or nonverbally. This problem has also been observed with adults [Gillan, Holden, Adam, Rudisill, and Magee, 1990], but was more prominent in this study possibly due to the physical limitations of children. Another study that investigated interaction techniques for children explored children's interactions with a graphical user interface for a software-based math 22 application [Berkovitz, 1994]. Children found it difficult to perform a marquee-type selection for a group of objects. A marquee selection involves drawing an imaginary box around a group of objects where any object completely enclosed by the box will be selected. In several cases, the children missed the targets while using this type of selection. Traditional marquee selection requires great care in choosing the ini-tial corner of the selection rectangle. When subjects chose this corner badly, no placement of the other corner sufficed to enclose the desired group of objects. Also the idea of choosing one point of an initially imaginary rectangle and then dragging to the other did not seem to be an easy gesture for young children to visualize, [p.248] The solution to this problem required a simple extension to the marquee selection procedure. The extension involved allowing the mouse to push out the edges of the marquee rectangle until all of the objects were located within the selection box. An alternative method of selection grew from this adaptation called "encirclement". Using encirclement, the mouse was dragged in a circular motion around the group of objects, resulting in a rectangle that enclosed all of the objects. The same study also suggested that other common interaction techniques such as double mouse clicks, mouse button distinctions and shift modifiers may also be difficult for children to perform. Both the Strommen [1994] and Berkovitz [1994] studies emphasize the need for empirical evidence on the usability of children's interfaces. This is true, not only for general interaction techniques, but also for application-specific interactions. 2.6 Gender Issues for Computers and Technology The issue of gender is extremely important for the design of educational computer environments. Numerous researchers have noted that girls and boys interact differ-23 ently with computers, think about computers differently, have different motivations for using computers, approach computers differently, and have different preferences, [Hall and Cooper, 1991; Inkpen et al., 1994; Lawry et al., 1995; Lockheed, 1985; Upitis and Koch, 1996; Wilder et al., 1985]. If our research is not sensitive to gen-der issues we run the risk of making inappropriate generalizations and developing learning environments that exclude a specific group of children. Early research on gender differences compared females and males on such issues as who used computers more, who liked computers better, and whether com-puters were perceived to be more appropriate for females or males [Lockheed, 1985; Wilder, Mackie, and Cooper, 1985]. A more recent study confirmed that gender differences still exist with respect to issues such as attitudes about and usage of computers [Durndell, Glissov, and Siann, 1995]. Other researchers have attempted to understand if some of the gender dif-ferences are a result of the biases and stereotypes in the design of computer soft-ware. Huff and Cooper [1987] explored whether or not software designers introduced gender-biases into the applications they developed. The results showed that pro-grams designed for girls could be classified as learning tools while those designated for boys were more like games. One disturbing observation from this research was that when software designers were asked to develop applications for students, the products often resembled those created for boys, raising the question of why girls aren't considered students too. The depiction of females in electronic environments is also a problem. Re-searchers have pointed out the issues of gender stereotyping in video games where women are portrayed as prizes or victims. In a study of 47 top-rated video games, the cover illustration portrayed a total of 115 male and 9 female characters. In addition, 13 of the 47 games had scenarios in which women were either kidnapped or had to be rescued as part of the game. This same trend can also be seen in computer software. Another study of twelve computer programs for mathematics 24 revealed more than fifteen male characters: heavy equipment operators, factory workers, shop keepers, mountain climbers, hang gliders and a genie. In contrast, only two female characters were found: a mother and a princess [Hodes, 1995]. A third approach to studying gender issues for computers and technology has been to gain an understanding of how females and males interact with the technology. Some researchers have characterized how girls and boys interact in an electronic games environment [Inkpen et al., 1994; Lawry et al., 1995], while others observered girls' and boys' classroom use of computers to uncover ways to ensure that girls had equitable access to computer technology [Upitis and Koch, 1996]. Some suggestions presented by Upitis and Koch for encouraging girls to use computers included designating some computers to be named and used by girls, having assigned computer times instead of "free time", and giving girls the option of working with a friend. Gender differences are overwhelmingly evident in the literature on girls' and boys' interactions with computers and technology. Understanding why boys and girls are different may not be the critical question. Instead, a better approach may be to simply gain an understanding of how girls and boys interact with technol-ogy and use that knowledge to effectively design and integrate educational learning environments into the classrooms. 2.7 Electronic Games for Education In the late 1970's and early 1980's the video game industry experienced tremen-dous growth. This phenomenon stimulated several research studies concerned with the advantages and disadvantages of video games as well as attempts to under-stand the powerful effect these games appeared to have on children. As early as 1982, researchers began suggesting that video games could be used for educational purposes [Blank, 1982; Bowman, 1982; David and Ball, 1986; Silvern, 1986] and speculated that "perhaps the magic of Pac-Man [could] be bottled and unleashed 25 in the classroom to enhance student involvement, enjoyment, and commitment" [Bowman, 1982, p. 14]. Suggested areas where video games could be used in an educational envi-ronment included problem solving, motor skills and spatial visualization. Several researchers demonstrated the potential that video games held for increasing spatial visualization [Dorval and Pepin, 1986; Gagnon, 1985; Greenfield, 1984; Lowery and Knirk, 1982]. There was also concern for the negative effects that video games could have on children, such as promoting addictive behavior and aggression. While many studies were conducted to show that video games encouraged addictive behavior, no strong results were found. However, several researchers did find a strong presence of violence in many video games [Braun and Giroux, 1989; Greenfield, 1984; Provenzo, 1991]. After this surge of research in the mid-1980's, very little research was per-formed on video games until the early 1990's. At this time it was important to re-investigate issues related to the use of electronic games for education because of the dramatic way that video and computer games had evolved over the preceding decade. Many researchers have commented on the potential benefits of electronic games for educational purposes. Klawe [1992] suggested that "[v]ideo games are an excellent vehicle to use to increase the exploration of mathematical concepts by children" [p. 14]. Papert [1993] stated that electronic games provide a gateway for children to enter the world of computers. The main focus for these and other researchers is the motivation that children have to play electronic games. Lepper and Malone [1987] explored the motivational factors provided by electronic games and created a taxonomy of intrinsic motivations for learning that include individual motivations (challenge, curiosity, control and fantasy) and in-terpersonal motivations (cooperation, competition, and recognition). Al l of these 26 factors can be found in electronic games. This view is supported by Long and Long [1984], who stated that video games are based on principles that motivate learning: challenge, fantasy, and curiosity. Continued research on electronic games both for educational and entertain-ment purposes is important because of the recent surge of interest in educational games. Educational games represent the fastest-growing type of software [Consumer Reports, 1995] and can be found in many schools and classrooms. Without solid research many of the new games produced may be "not-very-educational games ... and not-very-entertaining learning activities"[Consumer Reports, 1995, p.764]. 2.8 The E-GEMS Research Program The research reported in this dissertation is part of a large-scale research program on Electronic Games for Education in Math and Science (E-GEMS). E-GEMS is a collaborative effort among scientists, mathematicians, educators, professional elec-tronic game and educational software developers, classroom teachers, and children. The goal of E-GEMS is to increase, through the use of electronic games, the pro-portion of children who enjoy exploring and learning math and science concepts. The research activities of E-GEMS focus on human-computer interaction issues associated with learning in electronic game environments. Topics include studying which game formats are attractive to children and, at the same time, are suitable for learning particular concepts. Within E-GEMS we are also studying how to integrate electronic game learning with more traditional classroom learning environments, the effect of collaborative play, and the role of mediation by teachers. Ongoing research by members of the E-GEMS team includes: • Interactions with schools, teachers and students on a weekly basis to explore effective ways of integrating computers into the classroom and the curriculum [Klawe and Phillips, 1995; Upitis and Koch, 1996]. 27 • Development of the Phoenix Quest game, designed to be attractive to both girls and boys and to facilitate the learning of mathematical concepts by con-textualizing mathematical problems and activities within a full-length story with fully developed characters and plot. Within the game children are given the opportunity to play math and language puzzles, read an adventure novel, and communicate with characters in the game by sending and receiving post-cards [Klawe, Westrom, Davidson, and Super, 1996; Phoenix Quest Web Site, 1996]. • Development of two multi-user games, Island and Avalanche, to explore is-sues related to the design of multi-user games for education and to observe children's interactions in a multi-user game environment [McGrenere, 1996]. • Development of Super Tangrams, a tangram puzzle game designed to help chil-dren learn concepts of two-dimensional transformation geometry [Sedighian, 1996]. • Exploration of ways to support children's collaboration in an electronic puzzle-solving environment [Inkpen, Booth, Gribble, and Klawe, 1995; Inkpen, Booth, and Klawe, 1995; Inkpen, McGrenere, Booth, and Klawe, 1997]. For further details of E-GEMS projects and its research activity, consult the E-GEMS web site at http://www.cs.ubc.ca/nest/egems. My role in the research presented in Chapters Four, Five, and Six has been as the lead researcher, responsible for the design, set-up, implementation and analysis of the studies. Assistance was provided by members of the E-GEMS team who helped with transporting and setting up equipment in several of the studies, as well as helping to run some experimental sessions. Joanna McGrenere, as part of a directed studies course for her M.Sc. degree, collaborated on the experimental design used for the second phase of the multiple input device study (reported in Chapter 5, which was a refinement of the design used in the first phase. The field study 28 described in the final section of this chapter was a combined effort of the E-GEMS team. As one of the researchers on this team, I was responsible for overseeing the day-to-day activities of the field study, supervising other students working on the project, gathering data, and helping write the two journal articles that resulted from this field study. 2.9 Early E-GEMS Research: Science World, 1993 This section summarizes a preliminary field study undertaken by the E-GEMS group which provides background and motivation for the subsequent research discussed in this dissertation. Some of the material in this section has been adapted from the jointly-authored journal article '"We have never forgetful flowers in our garden:' Girls' responses to electronic games" [K. Inkpen, R. Upitis, M . Klawe, J. Anderson, A. Ndunda, K. Sedighian, S. Leroux, and D. Hsu, 1994]. Readers interested in a more complete description of the field study or other E-GEMS research should refer to the journal article or to the other E-GEMS publications cited above. When the E-GEMS group was first formed in late 1992, we recognized that before we could focus on how to develop electronic games to motivate children to ex-plore and learn math and science concepts, it was important to gain an understand-ing of how children interacted with the electronic games that were commercially available at the time the research took place. A field study was designed to explore children's interaction while they played commercial video and computer games in an informal setting. The research took place at Science World B.C. (Science World), at an exhibit called the Electronic Games Research Lab. Science World is an inter-active science museum, located in Vancouver, British Columbia, where children and adults explore various science concepts through hands-on activities and experimen-tation. Visitors to Science World are free to explore the exhibits at Science World as they wish. While many visitors move fairly quickly from one exhibit to another, others may choose to stay at one exhibit for extended periods of time. 29 Members of the research team, along with educational consultants and de-signers from Science World, and game designers from the commercial game devel-oper Electronic Arts Canada, developed the Electronic Games Research Lab. The research area was designed to create an informal learning environment for children of all ages. The exhibit included two types of electronic game platforms: video games and computer games. Within these categories, there were two video game units, a Sega Genesis and a Super Nintendo Entertainment System, and two computers, a Macintosh and an IBM-compatible PC. Games for each of the four machines were run for one- or two-week periods. Games included those from the popular culture, as well as ones that had been newly released. Some games, such as The Incredible Ma-chine, Operation Neptune, and New Math Blaster Plus, were designed as educational games. Others were created primarily for entertainment, such as Sonic the Hedge-hog, Lemmings, and Super Mario World. Some games designed for entertainment often had embedded educational features. A design station was also constructed as part of the Electronic Games Re-search Lab. This area included a Velcro covered box with several manipulatives (such as balls, tubes, ribbons, blocks, elastics, gears, string, fabric, and trolls), many of which could be attached directly to the walls of the box (see Figure 2.1). Clipboards, paper, pencils, and coloured markers were also available for writing and drawing. A kiosk was located near the design station where children could display their work. A second kiosk posted research news, gathered as the summer progressed. Another table had a Macintosh-based survey on electronic games, which could be filled out by children or adults if they so chose. The exhibit was supported by eleven researchers and several volunteers who assisted the children while they were playing games, provided materials at the de-sign station, conducted interviews, answered questions about the research aims and methods, arranged turn-taking systems during busy periods, and played electronic games during low-traffic periods to allow themselves to become familiar with the 30 Figure 2.1: Velcro covered box with manipulatives used as part of the design station games. In other words, the researchers and Science World volunteers were active and involved participants on many levels. 2.9.1 Method The research was conducted over the summer months of July and August, 1993. Several types of qualitative and quantitative data were collected over this time period. The qualitative data were gathered using researcher observations, interviews, artifacts from children, and intervention studies. The observations and interviews examined such issues as the attractiveness of particular games and platforms, gender issues, social interactions, watching and playing styles, attitudes and perceptions of adults towards electronic games, and children's learning in general. Data were recorded using field notes, audiotapes and videotapes. Artifacts produced by the 31 children were also collected. These artifacts included drawings made by the children of game characters and design features, as well as photographs of constructions made on the Velcro box (Figure 2.2 shows a child's drawing). Figure 2.2: A child's game design drawing Quantitative data were recorded using two methods: the electronic survey and timed samplings. The survey invited participants to describe themselves in terms of age, gender, life-style habits, likes and dislikes in terms of school subjects, as well as asking specific questions related to electronic games. We included the survey to support a number of our quantitative findings, but it is important to treat these findings with caution because not all participants at the exhibit completed the survey. The timed samplings were used to record the number of people at each area of the exhibit. Every fifteen minutes, one of the researchers would note the number of children and adults at each of the game stations, the design station, and the survey station. These observations were further classified by gender and by type of 32 participation - playing or watching. During the first ten days of research, two re-searchers completed the timed samplings until at least 95% interrater reliability was achieved. For three days, five-minute intervals were used. For the same three-day period, the total number of children who came into the exhibit area was tabulated by gender. 2.9.2 Results and Discussion The main observations from this research can be categorized into two groups: gender differences and collaborative behaviour. A summary of these results is given in the following subsections. More complete descriptions of the gender results are published elsewhere by Inkpen et al. [1994] and by Lawry et al. [1995]. Gender Differences One of the overwhelming themes that emerged from this initial field study was that girls and boys behave quite differently in an electronic game environment. Field Observations The following is a summary of some general observations made by the re-searchers, characterizing girls' and boys' interactions in the Electronic Games Re-search Lab at Science World. These observations describe a typical scenario of children playing at the Electronic Games Research Lab. It is 2:00 in the afternoon, and there is a large group of children crowded around two television sets. There are about 10 boys gathered • around each television, all waiting for a turn. There is a girl with her little sister and mother playing Scooter's Magic Castle at the nearby computer, and at another computer two boys are playing Lemmings. The Velcro wall and large table, resembling an art centre in an elementary school, are also occupied. Several parents are pointing out things to 33 their toddlers, and encouraging their children to try attaching blocks and cylinders to the Velcro wall. Four other girls are sitting around the table, drawing pictures of flowers, butterflies, and princesses. Meanwhile, another boy has just entered the exhibit. He glances over at Sonic, one of the video games, then quickly darts over to Sonic and jumps down on the floor. After watching for a while he asks the boy who is currently playing if he may have a turn. Soon after, the game control is passed over, and he begins his turn. A girl has entered at approximately the same time. She glances at all of the stations and walks towards the computer with the Lemmings game. She stands nearby to watch the boys who are currently playing. She continues to watch, remaining very quiet, while they solve several puzzles. [Inkpen et al., 1994, p.389] Interest in Electronic Games When asked if they like to play electronic games, most girls and boys re-sponded "Yes". What appeared to be different was what the girls and boys talked about with respect to the games they played. Many girls who played electronic games at home could not name the games they played, how many games they owned, or even the names of their favourite games. However, they were quick to verbalize many social aspects of the games: they could name a myriad of characters, describe storylines, and describe the relationships between characters in the games. When we talked to boys about the games they liked to play, most would list the names of all the games they played and discuss the specific details of the games such as the things you can do in the game, the secrets in the game, or all of the obstacles that needed to be overcome. Social Behaviour While Playing Electronic Games Observations of children's social behaviour revealed that both girls and boys 34 are very social when they play electronic games, but the role that games play in the children's social environment appears to be different for girls and boys. For girls, playing an electronic game appeared to be only one aspect of their social environment. Many girls engaged in other social activities while playing, such as talking with friends or with other people in the exhibit area. One group of three girls played the Sega Genesis game Sonic the Hedgehog for over an hour and a half, the longest period of time any girls were observed to play. Although these girls played the electronic game for a long period of time, the game was not their sole focus. The three girls sat on the floor with a female researcher, took turns playing the electronic game, talked about friends, school, sports etc., and drew pictures. Without the additional social aspects of the environment, the girls may not have been as motivated to play the game for that length of time. When boys played the electronic games they did exhibit social behaviour, such as talking to friends or researchers, but their activity was very focused on the game. Boys enjoyed talking with the researchers in between their turns, but they were only interested in discussing things related to the game; they were not interested in talking about other topics such as school or friends. Video Games vs. Computer Games Although both girls and boys stated that they liked to play video and com-puter games, we noticed clear differences in their preferences for electronic game platforms. As in the typical scene described earlier, girls visiting the exhibit were more interested in computer games than in video games. The timed samplings as well as the interviews and observations indicated that proportionally more girls were attracted to the computers than to the video game stations. Many boys who came to the exhibit loved to play video games! It was not uncommon to get groups of forty or more boys crowded around the floor area next to the video game units waiting for a turn to play. These boys were extremely 35 motivated to play the video games and seemed willing to wait hours to do so if necessary. The boys also enjoyed playing computer games and would crowd around the computer screen, waiting for a turn, but their enthusiasm for computer games did not appear to be as strong as it was for video games. Collaborative Behaviour Most of the children who visited the Electronic Games Research Lab, both girls and boys, enjoyed playing collaboratively and often appeared to be more successful as a result of this collaboration. The researchers did not force the children to work to-gether, so any collaboration the children engaged in evolved naturally. Observations revealed that often the children did not know their playing partners before arriving at Science World, but in the process of playing together, seemed to become instant playmates. The type of collaboration exhibited often depended on how busy the exhibit was and on the games the children were playing. Most of the time the children were left on their own to negotiate turn-taking systems. Often, when the exhibit was busy, the children would decide that each person would get "one life". If the exhibit was quiet they determined that each person would be able to play "two or three lives". Collaboration on video games was observed on many different levels such as: children playing a multi-user game, each with their own characters, discussing strategies in the game; one child explaining to another how to get past the current challenge in the game; or even a player temporarily giving up the controller to another child to get help through a difficult section and the other child returning the controller at the end of that section. Collaboration on the computers normally involved two or more children sitting around the computer screen, discussing how to achieve a goal in the game. Sometimes the children alternated control of the input device while at other times each child took on a distinct role. For example, one child 36 might give suggestions while the other carried out the suggestions in the game. Whatever the degree of collaboration, both girls and boys definitely enjoyed playing electronic games with peers. Many children said that they preferred to play at the same time with a friend. While some video games allow concurrent play, many require children to take turns. In contrast, almost all computer games are designed for only one person. The research that is described in the following chapters investigates ways of supporting children's collaboration in an educational puzzle-solving environment. First, we examine how structuring the children's social environment, such as the assignment of children to computers, can impact their interactions and achievement in a computer-based collaborative learning environment. Next, we examine how explicit support for collaboration, such as the addition of a second mouse, can impact children's collaboration, achievement, and learning. Finally, we examine the impact that low-level interface design details, such as mouse interaction style, can have on children's interactions with computers. The educational puzzle-solving environment used in all of our research studies was the computer game The Incredible Machine, produced by Sierra On-line Inc. The next chapter describes The Incredible Machine in detail after which the three research studies are presented. 37 C h a p t e r 3 The Incredible Machine Machine for Washing Dishes The professor turns on his think-faocet and dopes out a machine tor washing dishes while you are 3t the movies. When spoiled tomcat (A) discovers he is alone, he lets out a ye!i which scales mouse (8) inio jisrnping into basket (C), czi-mg lever end (D) to rise and puii string tE) which snaps automatic cigar-fighter ;F). Flame ( 6 j starts tire sprinkier (H). Water runs or; dishes (!) and drips into sink (J). Turtie (K). thinking he hears babbitng broofc babbling, and having no sense of direction, starts wrong way and putis string ;L;< which turns on switch (Mi that starts electric gtow heater (N). Heat ray (O) dries the dishes. if the cat and the turtle get on to your scheme and refuse to cooperate, simpiy put the dishes on the front porch and pray for rain. A Rube Goldberg invention which appeared in one of his cartoons [Goldberg, 1979] Video games, being the first example of computer technology applied 38 to toy making, have nonetheless been the entryway for children into the world of computers. These toys, by empowering children to test out ideas about working within prefixed rules and structures in a way few other toys are capable of doing, have proved capable of teaching students about the possibilities and drawbacks of a newly presented system in ways many adults should envy. Video games teach children what computers are beginning to teach adults - that some forms of learning are fast-paced, immensely compelling and rewarding. - Seymour Papert The Children's Machine, 1993, pages 4-5 The Incredible Machine and The Even More Incredible Machine1 are com-mercial puzzle-solving computer games produced by Sierra On-Line, Inc. [1993] that invite players to construct "Rube Goldberg"-style machines [Marzio, 1973] by arranging collections of objects to achieve particular goals comprising the puzzles. The two games are identical except that The Even More Incredible Machine has more puzzles than The Incredible Machine. The following sections describe, in depth, features of The Incredible Machine and how players solve puzzles within the game. Complete descriptions for solving the first three puzzles in the game are provided. The final section discusses why this software was chosen for our research and the benefits it provides as an educational learning environment. 'This chapter contains several images from the The Incredible Machine and The Even More Incredible Machine ©1992,93 Sierra On-Line, Inc. The Incredible Machine and The Even More Incredible Machine are trademarks of Sierra On-line 39 3.1 Playing the Game When the game starts, players are presented with a control screen, similar to the one illustrated in Figure 3.1. The left-hand side of the screen contains the control panel, where players can modify settings in the game such as the volume for sound, or the amount of gravity or air pressure that is present in the environment. The control panel also has a play puzzle button, a reset puzzle button and a quit button. Children can choose between two modes of play: freeform mode or puzzle mode. In freeform mode players may use any of the objects provided in the game to construct their own machines. In puzzle mode the players are presented with a goal and a number of objects with which to build a machine to satisfy the goal. Typical goals for puzzles include breaking all the balloons on the screen, putting a basketball through a hoop after a series of bounces from a set of strategically placed trampolines, or saving Mort the Mouse from alligators. The right-hand side of the control screen shows a reduced-sized picture of the current puzzle and specifies the goal for that puzzle using a textual description displayed below the puzzle. Starting a Puzzle To begin playing a puzzle a player first reads the goal for the puzzle, then clicks on the picture of the puzzle on the control screen or clicks on the green start arrow on the control panel. A new screen appears, divided into three sections: the playing screen; the parts bin; and the run machine icon (see Figure 3.2). The playing screen is the area where the machines will be constructed. It contains objects that have already been placed. The initial objects can be used to solve the puzzle but cannot be moved. The parts bin, on the right-hand side of the screen, contains other objects available for the puzzle. Each object is represented by an icon and a number below the icon that shows how many instances of that object are available. Objects can be moved from the parts bin, onto the playing screen, and configured with the objects already on the screen to try to achieve the goal of 40 Figure 3.1: The Incredible Machine Control screen. the puzzle. The run machine icon, located in the top right-hand corner, causes the machine to start when it is clicked on. The run machine icon is a picture of a person who is about to begin running a race. When the cursor is placed over the icon the words "start machine" appear, reminding the player that this icon is used to start the machine that they have configured (see Figure 3.3). Playing the game (solving a puzzle) involves moving objects from the parts bin onto the playing screen and configuring the objects among objects already on the playing screen so as to achieve the goal required for that puzzle. When a player has a configuration of objects to test as a solution to the puzzle, the machine can be run to see if the configured objects perform the right actions to achieve the required goal. If the configuration successfully achieves the goal, a small dialogue box appears in the middle of the playing screen, congratulating the player for solving the puzzle and inviting the player to advance to the next puzzle (see Figure 3.14). If the configuration does not successfully achieve the goals, no dialogue box appears 41 Run Machine Icon Playing Screen 3 3 # A _ Parts Bin Figure 3.2: Parts of the game screen. and the player is required to press a mouse button to stop the machine. Stopping the machine results in all of the configured objects being returned to the positions they occupied before the machine was run and permits the player to re-configure objects or add new objects from the parts bin before running the machine again. Objects Used in the Game The objects used to construct machines include many household items such as balls, gears, balloons, elastics, and scissors along with a host of entertaining characters such as mice, cats, alligators and people with lemming-like behaviour (Figure 3.4 shows all of the available objects). Many of the objects have behaviors, which vary from simple mass properties (such as falling under the influence of gravity when the machine is run) to more complex interactions that result from objects contacting each other or through forces applied to objects by indirect interactions with other objects. In this sense, The Incredible Machine resembles the Alternate Reality Kit 42 Figure 3.3: Run machine icon used to start the machine. [Smith, 1987] and similar systems that permit objects to be configured and which then simulate real or exaggerated laws of physics applied to the objects [Borning, 1986; Cockburn and Greenberg, 1995; Roschelle, 1992]. Manipulating Objects Once on the playing screen, objects can be moved to new positions (translation), flipped to face the opposite direction (rotation), re-sized (scaling), or placed back in the parts bin. To perform any of these actions, the object must first be placed on the playing screen. When the cursor is placed over an object, several icons may appear around the object which then can be used to perform various actions. Objects that can be flipped display two red circular arrows in their bottom, left-hand corners. When the cursor is placed over the red arrows, a second larger pair of red circular arrows appears attached to the cursor, providing feedback to the player that the cursor is in the correct position to perform a rotation. Clicking on these arrows causes the object to rotate 180°. Figure 3.5 shows a pair of scissors being rotated with the two pairs of red circular arrows visible. Objects that can be re-sized display two blue, double-headed arrows appear-ing on both sides of the object. Clicking on either of the double-headed arrows puts the object into re-size mode and moving the mouse to the left or right causes the object to change size. Only the object's width is affected by re-sizing, and each 43 Figure 3.4: A l l of the objects found in The Incredible Machine. object has a minimum and a maximum width. Once the object is the desired size, clicking the mouse button freezes the object at its currently selected size. Figure 3.6 shows a ramp being re-sized on the left, and three possible sizes for the ramp on the right. Al l objects can be placed back in the parts bin by clicking on the trash-can icon on the top left-hand corner of an object. Figure 3.7 shows a basketball being put back into the parts bin. 44 Figure 3.5: A pair of scissors being flipped to face the opposite direction. Figure 3.6: The figure on the left shows a ramp being re-sized and the figure on the right shows three possible resulting sizes for the ramp. Connector Objects Elastics and ropes are connector objects that are used to connect two or more objects together. A connector object is placed by first picking it up from the parts bin, then attaching it to a connection point on an object. Once the first end-point has been placed, a red line appears from where the connector object was attached, to the current cursor location, providing visual feedback that one end has been attached. When the cursor has been moved to a correct place to attach the second end of the connector object, the red line turns to green, signifying that the cursor is at a valid second end-point. Figure 3.8 shows the four steps involved in attaching an elastic 45 Figure 3.7: A basketball being recycled back to the parts bin by clicking on its trash can icon. between a mouse motor and a conveyor belt. 3.2 Sample Puzzle Solutions The main purpose of The Incredible Machine, at least as we use it in an interactive learning environment, is to provide children with an opportunity to enhance their creative problem solving skills. Reasoning by analogy and drawing on common sense knowledge about everyday objects, players are expected to solve the rest of a puzzle by figuring out how to configure some or all of the objects from the parts bin to completely achieve the required goal. The following sections describe, in detail, solutions to the first three puzzles that children execute in The Incredible Machine. Often there are a number of different configurations that will solve a given puzzle and these may or may not use all of the objects available from the parts bin. 3.2.1 Solution to Puzzle #1 The puzzle shown in Figure 3.9 is the initial configuration for the first puzzle, where the goal is to get the basketball through the hoop. Initially the basketball is sitting on a conveyor belt that is not moving, near the top of the screen. The parts bin contains additional objects (three elastics, three mouse motors and three ramps) 46 Figure 3.8: The steps involved in attaching an elastic between a mouse motor and a conveyor belt: (a) One end of the elastic is attached to the wheel on the mouse motor, (b) The cursor is moved towards the second end-point. A line appears from the first end-point to the cursor, providing feedback that one end of the elastic is attached. At this stage the line is red, signifying that the cursor is not in a valid place to connect the second end of the elastic, (c) The cursor is placed over the small wheel on the conveyor belt which will be the second end-point for the elastic. The red line between the first end-point and the cursor turns to green signifying that the cursor is in a valid place to attach the second end-point, (d) The second end-point is attached and a visual representation of an elastic is shown between the mouse motor and the conveyor belt. 47 that can be moved onto the playing screen to assist in achieving the goal of getting the basketball through the hoop. Clicking on the "run machine icon" is often a good strategy for obtaining hints about how the goal is to be achieved. This allows a player to see how the machine operates before any new pieces are added. In the case of Puzzle #1, the only thing that happens is that the bowling ball on the far left-hand side of the screen falls straight down and off the screen due to the force of gravity. None of the other balls or the conveyor belts move. Figure 3.9: Initial configuration for Puzzle #1. The goal for this puzzle is to get the basketball to fall through the hoop on the right-hand side of the puzzle. A possible next step is to get the bottom conveyor belt to move. This can be achieved by using a mouse motor and an elastic. A mouse motor is a caged mouse that runs and spins his treadmill whenever his cage is disturbed. Figure 3.10 shows how a mouse motor can be placed underneath the bowling ball on the far left-hand side of the screen (the bowling ball that falls when the machine is run) and how an elastic can be attached from the mouse motor to the bottom conveyor belt. Now 48 when the machine is run, the far left bowling ball falls onto the mouse cage which in turn causes Mort the Mouse to run in his treadmill, making the bottom conveyor belt spin and the bowling ball on that conveyor belt rolls off to the right. Figure 3.10: A mouse motor and an elastic are positioned on the left-hand side of the screen to cause the bottom conveyor belt to turn when the machine is run. Because the bowling ball on the bottom conveyor belt now moves, it can be used to start another mouse motor to turn the middle conveyor belt. A second mouse motor can be placed in a position where the moving bowling ball will hit it and an elastic placed between this mouse motor and the middle conveyor belt (see Figure 3.11). When the machine is run now, the bowling ball on the middle conveyor belt also rolls to the right. A similar configuration of mouse motor and elastic can be positioned where the middle bowling ball will bump the mouse motor and therefore cause the top conveyor belt to turn, so the basketball rolls to the right (see Figure 3.12). Although it appears that ramps are needed between the top conveyor belt and the edge of the 49 * ........ .J^ Figure 3.11: A second mouse motor and elastic are positioned to cause the middle conveyor belt to turn when the machine is run. hoop to fill in the gap (a notion that is reinforced by having three ramps available in the parts bin), running the machine reveals that this is not the case, although it can be done if desired. When the basketball rolls off the conveyor belt it falls down through the hoop, achieving the goal for Puzzle #1. Figure 3.13 shows the solution to Puzzle #1 when the machine is run. Figure 3.14 shows the dialogue box that appears after successfully solving the puzzle. 50 T-Figure 3.12: A third mouse motor and elastic are positioned to cause the top con-veyor belt to turn and the basketball to roll to the right when the machine is run. Figure 3.13: When the machine is run, the basketball falls through the hoop, solving Puzzle #1. 51 D k Figure 3.14: A dialog box appears whenever a puzzle has been solved. 3.2.2 Solution to Puzzle #2 The puzzle shown in Figure 3.15 is the initial configuration for the second puzzle in The Incredible Machine. The goal is to get each bowling ball into a pipe basket. On the left-hand side of the screen a mouse motor is connected to a conveyor belt with an elastic, and a basketball is positioned above the mouse motor. When the machine is run, the basketball falls onto the mouse motor, causing Mort the Mouse to run in his treadmill, causing the conveyor belt to turn and the bowling bowl that is on the conveyor belt to roll to the right, falling into the pipe basket on the left-hand side of the screen. At first glance, it would appear that a similar setup of mouse motor, elastic, and basketball on the right-hand side would cause the right-hand side conveyor belt to turn and the bowling ball to fall into the pipe basket on the right-hand side of the screen. Closer inspection reveals that the mouse in the mouse motor is facing towards the right, so this configuration would cause the bowling ball to roll to the 52 * 1 1 * * Figure 3.15: Initial configuration of Puzzle #2. The goal for this puzzle is to get both bowling balls into the pipe baskets. right, eventually falling off the screen. The mouse motor needs to be flipped to face the opposite direction so the bowling ball moves to the left instead (see Figure 3.16). Once the mouse motor has been flipped, the mouse motor, elastic and bas-ketball can be configured similarly to the left-hand side of the puzzle (see Figure 3.17). When the machine is run now, both of the basketballs fall onto mouse motors, causing both Morts to run in their treadmills, causing both conveyor belts to turn, resulting in both bowling balls falling into their respective pipe baskets. Figure 3.18 shows the solution to Puzzle #2 when the machine is running. 3.2.3 Solution to Puzzle #3 The puzzle shown in Figure 3.19 is the initial configuration for the third puzzle in The Incredible Machine. The goal is to pop all of the balloons on the screen. Initially, there are three balloons (each trapped under inert structures that keep the 53 Figure 3.16: The mouse motor must be flipped to face the opposite direction or else the bowling ball will roll to the right and fall off the screen. • m m, • Figure 3.17: The flipped mouse motor, the elastic and the basketball are configured to match the left-hand side of the puzzle. 54 1 ; V r t'm - n Figure 3.18: When the machine is run, both bowling balls fall into the pipe baskets, solving Puzzle #2. balloons from rising upward when the machine is turned on), one pair of scissors, one bellows, one baseball, one cannon ball, and one mouse motor connected via an elastic to a gear. When the machine is run, the balloon on the left-hand side of the screen pops because the baseball falls down by gravity onto the bellows, causing the bellows to contract and thereby blow the balloon into the sharp points of the scissors. This pops the first balloon. The other two balloons remain un-popped, although a player may discover, while running the machine, that dropping the cannon ball onto the mouse cage excites Mort the Mouse into running in place on his treadmill which, through the elastic pulley, makes the gear turn. A similar configuration of tennis ball, scissors, and bellows can be used to pop the balloon at the bottom of the screen. Moving these three objects from the parts bin to appropriate positions on the playing screen and then testing the configuration by running the machine demonstrates that this is a good idea, although a player 55 w Figure 3.19: Initial configuration for Puzzle #3. The goal of this puzzle is to pop all of the balloons on the screen. quickly discovers that blowing the second balloon into the dull end of the scissors does not succeed in popping the balloon. It is necessary to reverse the orientation of the second pair of scissors before the balloon will pop (see Figure 3.20). Breaking the third balloon is a little more difficult because only two pairs of scissors are available and both are in use already for popping the other two balloons. The cannon ball falling down on the mouse cage that starts the attached gear spinning is an additional resource. Because the gear has jagged edges, it might pop the balloon. The third set of bellows is placed on the right-hand side of the top balloon, reversed so it blows in the correct direction, and the basketball is placed above the bellows (see Figure 3.21). When the machine is run, the basketball falls onto the bellows, blowing the third balloon into the spinning gear, which breaks it. Figure 3.22 show the solution to Puzzle #3 when the machine is running. 56 Figure 3.20: A tennis ball, bellows, and scissors are placed near the bottom balloon, in a configuration similar to that on the left-hand side of the screen, in order to pop the second balloon. Figure 3.21: The basketball and bellows are set up to blow the top balloon into the spinning gear. 57 Figure 3.22: When the machine is run, two balloons get broken by scissors and one breaks on the spinning gear, solving Puzzle #3. 3.3 IBM-Compatible Version vs. Macintosh Version The IBM-compatible version of The Incredible Machine uses a point-and-click in-teraction style to move objects, whereas the Macintosh version of The Incredible Machine uses a drag-and-drop interaction style to move objects. Drag-and-drop is the dominant mouse interaction style found in software developed for Macintosh computers. A point-and-click interaction style is commonly found in software de-veloped for IBM-compatible computers although it is not used exclusively and some software is developed using a drag-and-drop mouse interaction style. Playing The Incredible Machine requires moving objects from the parts bin onto the playing screen. For most objects this involves selecting the object from the parts bin and placing it on the playing screen. However, an object cannot be placed in the same position on the screen as another object, and certain objects are constrained to have particular relationships to other objects, such as the elastics shown in the previous 58 section, which connected the wheel on a mouse motor to either a gear or a conveyor belt. The key distinction, for our purposes, between the IBM-compatible point-and-click mouse interaction style and the Macintosh drag-and-drop mouse interac-tion style is that drag-and-drop requires a player to maintain pressure on the mouse button during a placement movement (a muscular task that is reported in the lit-erature to be difficult for young children [Strommen, 1994]), and it also requires additional time for error recovery because of the need to return to the parts bin and then repeat movements after placement errors have occurred. We examined chil-dren's use of these two mouse interaction styles as part of our research. The study and its results are discussed in Chapter 6. The following two subsections describe the differences between the two interaction styles for The Incredible Machine. 3.3.1 Moving Regular Objects In the IBM-compatible version of The Incredible Machine, which uses a point-and-click interaction style, most objects are moved by clicking once on an object in the parts bin to pick the object up (a click refers to pressing the mouse button down and then releasing it), moving the cursor to its desired position on the playing screen, and finally clicking the mouse button again to place the object at that position on the playing screen. Once an object has been picked up, the object disappears from the parts bin and an iconified picture of the object is attached to the cursor to provide visual feedback confirming that the object is picked up. In the Macintosh version of The Incredible Machine, objects are moved by pressing the mouse button down on an object in the parts bin to pick the object up (the mouse button is not released at this stage), dragging the object to the desired position on the playing screen, and releasing the mouse button to place the object. The visual feedback provided is identical to that provided in the point-and-click interaction style, but additional kinesthetic (proprioceptive) feedback is provided 59 by the muscular tension required to keep the mouse button down while dragging. Using either of the two techniques just described, if an object is dropped somewhere other than where it was intended, the object will be placed at that position so long as the position on the screen is not occupied by another object. If the position on the screen is occupied, then an error occurs. Using the point-and-click interaction style in the IBM-compatible version, the object will not be placed when an error occurs, but the object will remain picked up, attached to the cursor and ready for a second attempt at placing the object. However, using the drag-and-drop interaction style in the Macintosh version, once the mouse button has been released it cannot be re-released to re-do the placement part of the action. Thus, if there is an error and the object cannot be dropped at the selected location, something else must happen to it. In this version of the game the object is returned to its original position in the parts bin when a drop error is detected. To attempt the placement again, the player must return to the parts bin, pick up the object again, and move it back to the position where it is to be placed. This is obviously a more time consuming recovery procedure than in the point-and-click version, thus the time penalty for making a drop error is significantly greater for drag-and-drop than for point-and-click. State transition diagrams for placing regular objects using the IBM-compatible and the Macintosh versions of The Incredible Machine are shown in Figure 3.23. The diagrams show the necessary steps for completing the placement by moving from top to bottom in a diagram. The states are represented by the long rectangles. A state comprises a position on the screen and whether or not the object is picked up. The position on the screen could either be inside the parts bin or inside the playing screen. Inside the playing screen has two variants: on top of another object or on top of an empty space. Transitions between states are represented in the diagram by arrows and correspond to mouse movements, mouse clicks (in the point-and-click version), and mouse button-down or mouse button-up events (in the drag-and-drop 60 version). Downwards arrows represent forward progress towards completion of the placement while upwards arrows represent errors or backwards movement through the steps. Errors are represented by double-lined arrows. The long arrows passing through the fourth state (represented by the dotted lines through the state) indicate that this movement skips a state (i.e., during the placement of an object, the cursor may never need to pass over another object on the screen). Analyzing the state transition diagrams we notice that two of the three errors are the same for both interaction styles: (1) an error in the first state causes the player to remain in the first state and (2) an error at the third state causes the player to back up one state. In contrast, the third error (the one occurring in the fourth state) has quite different consequences for the two interaction styles. In the point-and-click interaction style the player remains in the same state, ready for another try, but in the drag-and-drop interaction style the player is required to move back three states, all the way to the initial state. This long backwards error for the Macintosh drag-and-drop style suggests a difficulty with this interaction style that could be frustrating to players if this type of error occurs. 3.3.2 Moving Connector Objects In addition to regular objects, The Incredible Machine has special objects called connector objects, such as elastics and ropes, which are used to connect two or more objects together. The main difference between connector objects and regular objects is that a connector object needs to be attached in two or more different places to other objects. Each attachment requires a placement operation to determine the attachment point, which is constrained to specific parts of certain types of objects, such as the wheel on the mouse motor. Figure 3.8 illustrates the connection of an elastic between a mouse motor and a conveyor belt. Connector objects are moved using the point-and-click interaction style in the IBM-compatible version of The Incredible Machine in a manner similar to that 61 Placing Regular Objects (a) Macintosh IBM-Compatible Drag-and-Drop Point-and-Click Inside Playing Screen Object not picked up if ^ W EE Inside Parts Bin Object not picked up Inside Parts Bin Object picked up Inside Playing Screen, On Another Object Object picked up 5SD ^ M f W Inside Playing Screen, On Empty Space Object picked up Inside Playing Screen Object Dropped ^ ^ M o v e Click ^ j j ^ Button Down tyfa Button Up ^ Error Figure 3.23: State transition diagrams for placing regular objects in The Incredible Machine for (a) the Macintosh version on the left and (b) the IBM-compatible version on the right. 62 used for regular objects. For a connector object that needs to be attached to two connection points, such as an elastic, the mouse button is clicked once on the con-nector object in the parts bin to pick it up, the connector object is then moved to the first attachment point, the mouse button is clicked to attach the first end of the connector object, the connector object is then moved to the second attachment point, and finally the mouse button is clicked again to connect the second end of the connector object. In contrast, the drag-and-drop interaction style in the Macintosh version of The Incredible Machine moves connector objects by first clicking on a connector object in the parts bin to pick it up. Because the pick-up action for the connector object is a click, the mouse button is released immediately. If a drag is attempted at this stage (if the mouse is moved without releasing the button), an error occurs and the connector object returns to its original position in the parts bin. Once the connector object has been successfully picked up, the cursor can be moved to the first attachment point. From the first attachment point to the second, a drag-and-drop motion is performed to attach the connector object. The mouse button is pressed down on the first attachment point to attach the connector object, then the cursor is moved to the second attachment point where the mouse button is released to make the second attachment. Connector objects can only be attached to specific objects. For example, elastics can be attached to objects such as motors and conveyor belts whereas ropes can be attached to objects such as balloons, buckets, and teeter-totters. Both ends of a connector object must be connected to valid objects. Using either of the two techniques described above, if an attachment point for a connector object is an incorrect position (not on a valid object or violating a constraint), an error occurs. Using the point-and-click interaction style in the IBM-compatible version, if an error occurs the end of the connector object will not be placed, but the connector object will remain picked up, attached to the cursor and ready for a second attempt at 63 making the attachment. Using the drag-and-drop interaction style in the Macintosh version, if either attachment point is in an incorrect position, an error results and the connector object is placed back in its original position in the parts bin. Thus, in the Macintosh version, to attempt to re-connect the objects the player must return to the parts bin, pick up the connector object again and move to each of the attachment points again. Not only is the penalty for an error greater for drag-and-drop than for point-and-click, it is even more severe if it happens after the first attachment is successful because both the incorrect second attachment and the correct first attachment will have to be performed again with the drag-and-drop interaction style. The state transition diagrams for placing connector objects that attach in two places using the IBM-compatible and the Macintosh versions of The Incredible Machine are shown in Figure 3.24. The diagrams show the necessary steps for completing the placement by moving from top to bottom in a diagram. The states are represented by the long rectangles. A state comprises a position on the screen and a status of the placement. The position on the screen could either be inside the parts bin or inside the playing screen. Inside the playing screen has three variants: on an invalid endpoint, on the first endpoint, or on the second endpoint. The status of the movement includes whether or not the connector object is picked up and whether or not the first endpoint has been attached. Transitions between states are represented in the diagram by arrows and correspond to mouse movements, mouse clicks (in the point-and-click version), and mouse button-down or mouse button-up events (in the drag-and-drop version). Downwards arrows represent forward progress towards completion of the placement while upwards arrows represent errors or backwards movement through the steps. Errors are represented by double-lined arrows. Analyzing the state transition diagrams we discover that the impact of errors is significantly different for the two interaction styles. The drag-and-drop interaction 64 (a) Macintosh Drag-and-Drop Placing Connector Objects ( b ) I B M . C o m p a t i b l e Point-and-Click tart Inside Playing Screen Object not picked up U U u M l M r J M L Inside Parts Bin Object not picked up I Inside Parts Bin, Object picked up I Inside Parts Bin Object picked up Inside Playing Screen, Not at a Valid Endpoint Object picked up 55 5 Inside Playing Screen, 1st Endpoint Object picked up 5 Inside Playing Screen, 1st Endpoint First Endpoint Attached lid Endpoint Inside Playing Screen, Not at a Va mFirst Endpoint Attached Inside Playing Screen, 2nd Endpoint First Endpoint Attached 5 Inside Playing Screen, 2nd Endpoint Second Endpoint Attached ^ j ^ M o v e Click fyfy Button Down tyfy Button Up Error Figure 3.24: State transition diagrams for placing connector objects in The Incred-ible Machine for (a) the Macintosh version on the left and (b) the IBM-compatible version on the right. style has four long errors, causing the player to move backwards anywhere from two to six states. In contrast, the point-and-click interaction style only has two backwards errors, one of which only moves the player back one state. Al l errors in the drag-and-drop interaction style cause the player to return to the initial state to start all over again. Having to continually repeat steps as a result of errors could be frustrating for players. Another obvious difference between the two interaction styles is the addition of a new state between the second and third states for the Macintosh drag-and-drop interaction style. This means that players must be aware of this additional state, which doesn't exist when moving regular objects and therefore presents an inconsistency in the interaction style for the two types of objects in the Macintosh version of the game. 3.4 A Testbed Collaborative Learning Environment We chose to use The Incredible Machine as our educational learning environment in many of our studies on supporting children's collaboration. After watching hundreds of children play the game we noticed many unique qualities that this game provides. Our most overwhelming observation of children playing The Incredible Machine was children's interest in playing the game with others. It was not uncommon to see many children crowded around a small computer screen. These children were not merely waiting for their turn on the computer, they were active, engaged participants in the problem solving activities of the game. This "mutual exchange process in which ideas, hypotheses, strategies, and speculations are shared" [Cohen, 1994, p.4] is the type of interaction that Cohen states is effective for conceptual learning. The Incredible Machine provided many features that coincided with the task dimensions Cohen [1994] suggests can affect the amount and type of interaction that occurs between children. The puzzle-solving environment of The Incredible Machine can be viewed as an ill-structured problem: each puzzle has multiple solutions, and the way in which certain objects are used to construct machines may vary 66 from puzzle to puzzle. The Incredible Machine presents an abstract representation of objects employed together. As a consequence, when groups of children play together, they must work with each other to gain an understanding of the puzzle in order to communicate more effectively. Children playing The Incredible Machine often manage the process of solving problems through explicit verbalization of their problem-solving activity. They specify goals, plan procedures, generate and select alternatives, and review and modify their plans. This can also be an important part of the learning process, as noted by Cohen. The excellent environment for creative problem-solving provided by The In-credible Machine, and our earlier observations of children's willingness to engage in collaborative behaviour when they encountered the game, led us to select The In-credible Machine as the computer-based collaborative learning environment for the research that we performed. Three studies, each using The Incredible Machine, are reported in the following chapters. 67 Chapter 4 Exploring Collaborative Play Boys playing The Incredible Machine at Science World, June 1993 Joey: Put it down and a little bit over.... Put it down. Colin: I see.... Move your hand. Joey: That's good there. Mark: Just wait! Joey: Oh! Another one. Shawn: But you won't have enough room! 68 Joey: Yeah, but ... Mark: Put a ramp there... Put a ramp here and it will lead right to the... . Just put this right there and it will lead to the trampoline. Shawn: But it has to... Joey: He's right! He's right! He's right! This discussion took place between a group of eight teen-aged boys playing the puzzle-solving computer game The Incredible Machine at Science World during the summer of 1993. The boys played for approximately two hours and successfully solved many puzzles. As the boys played they continually discussed, debated and suggested ideas. Control of the mouse was passed between different members of the group and the boys' fingers were constantly on the screen pointing out new ideas. Would the boys have been as successful if they had played the game individually on separate computers? Would they have played as long? Children often gather in groups around computers and video games, espe-cially to play games. This is obvious to casual observers of children in arcades, living rooms, or classrooms, but it was also overwhelmingly evident to us during our field study of children playing electronic games at Science World in 1993 [Inkpen et al., 1994]. In that study we observed children interacting in an electronic games environment consisting of two video game units with television sets (a Super Nin-tendo Entertainment System and a Sega Genesis), two computers (a Macintosh and an IBM-compatible PC), an on-line questionnaire, and a design station (for a more complete description of this study see Section 2.9). Two major research foci which emerged from this study were gender issues and children's collaboration, discussed in Section 2.9. The collaborative observations revealed that many of the children preferred playing electronic games in groups and often they appeared to be more successful in the games as a result of their collaborations. As discussed in sections 2.1, 2.2, and 2.3, numerous researchers have inves-69 tigated cooperative learning for both computer and non-computer tasks and have found significant benefits for both achievement and social outcomes [Hymel et al., 1993; Johnson et al., 1981; Johnson and Johnson, 1986; Slavin, 1984; Slavin, 1990]. Despite these obvious benefits, many classrooms continue to use the computer pri-marily as a tool for individual use. We wanted to examine whether children would be more successful in a computer-based interactive learning environment playing alone or playing with a partner. When children did play with a partner, we also wanted to investigate the children's performance and behaviour when the children played side-by-side on separate computers compared to when two children played together on the same computer. A formal study was designed to explore children's interactions while play-ing an educational electronic game in both a school environment and an informal learning environment. The issues investigated were: solo play vs. group play: children's achievement and learning while playing alone as opposed to playing with a partner a single shared computer vs. one computer per child: children's achieve-ment and learning while playing with a partner either together on one com-puter or side-by-side on two computers collaborative behaviour and attitudes: children's mouse sharing and interac-tions with their partners while playing in various collaborative configurations motivation to play: how children's playing configuration affects their motivation to play gender issues: examining all of the above issues for girls and boys separately to better understand girls and boys styles of interaction The game chosen for the study was The Incredible Machine created by Sierra On-Line, Inc., a puzzle-solving game featuring a wide variety of simulated tools used 70 to construct machines to solve particular challenges. It is described fully in Chapter 3 and our justifications for its use are given in Section 3.4. 4.1 Phase 1: Kerrisdale School, 1994 The first phase of the study examined children's interactions while playing The Incredible Machine, in a school environment, during school hours. The focus of this phase was to explore possible trends in children's performance and motivation as they played in various collaborative setups. We hoped to gain a better understanding of the children's interactions in this environment. 4.1.1 Method Subjects and Setting This phase of the study was conducted at Kerrisdale Elementary School, a public elementary school in an upper-middle class neighbourhood of Vancouver, British Columbia, Canada. The participants were 104 children (52 girls and 52 boys) be-tween the ages of 9 and 12 who had not previously played The Incredible Machine. The children were assigned to one of the three experimental conditions: Solo Play: children playing alone on one computer Parallel Play: children playing with a partner on two computers, side-by-side Integrated Play: children playing with a partner on one computer A pseudo-random assignment was used where children from each class contributed to each of the experimental conditions. For the two-player conditions, the children were assigned a partner from their class, corresponding to a particular gender-dyad. The gender groupings for the dyads were: girls: a pair of girls 71 boys: a pair of boys mixed: one girl and one boy A l l children who participated were volunteers who had received permission from their parents. Of the classes involved in the study, a 77% participation rate from the students was achieved. The setting was the Kerrisdale School annex, located across the playground from the main building. In previous years, the annex had housed a pre-school program which was not active at the time of the study. Either one or two IBM-compatible computers, depending on the experimental condition, were placed on a large table at one end of the room. When two computers were used, they were placed side-by-side on the table. A chair was provided for each child, but all chairs were initially placed away from the table, requiring the children to place them where they desired. Figure 4.1 shows the experimental setup for the Parallel Play condition (left) when two computers were used and for the Integrated Play condition (right) when one computer was used. Figure 4.1: Parallel Play (left) and Integrated Play (right) conditions 72 Both the children and their interactions within the game were recorded using video. Figure 4.2 shows the configuration of the video equipment. A video camera with a tripod was placed behind the computer table, off to the right, against a wall. The camera was positioned to capture the children's faces and body movements so as not to be distracting to the children. A flat PZM microphone was placed on the table between the children. This microphone was chosen because its presence is fairly unobtrusive to children and therefore might not intimidate them as much as a microphone that the children would be required to wear. The output from the video camera was sent to a VCR (VCR #1) to create a backup videotape. This allowed the original videotape to be archived and the backup to be used for video analysis. The output from V C R #1 (indirectly from the video camera) was sent to a television, only visible to the researcher, in another area of the room. This allowed the researcher to monitor the images the camera was recording. The children's interactions in the game were also recorded on a second V C R (VCR #2) using a VGA-to-NTSC converter box attached to the computer. As the children played the game, an archived copy of the computer screen was recorded on V C R #2. The other half of the room contained a sofa and a small table where the children could comfortably enjoy a snack, fill out a questionnaire and take part in casual discussion with the researcher at the end of the session. Figure 4.3 shows the snack/questionnaire area of the room. Procedure The total length of each experimental session was forty minutes, equivalent to one class period. The children were excused from class to come to the annex for the session. Upon arrival the children were welcomed and given a brief introduction to the project and the game, The Incredible Machine. The children were asked to try to complete the first three puzzles in the game and were told that they would be allowed to play for as long as they desired, up to a maximum of thirty minutes. Children 73 VCR #2 Figure 4.3: The snack/questionnaire area of the experiment room 74 assigned to the Solo Play condition were asked to play the game by themselves. Children assigned to the Parallel Play condition were told that two computers were available and that they had the option to play either together on one computer or side-by-side with each child having their own computer. Children assigned to the Integrated Play condition were given only one computer and they were told that it was up to them to sort out how they would play. In both pair conditions, the children were permitted to talk freely with their partner and collaborate as they wished. The children were given no directions on how to play to game; but, the game manual was placed on the table beside the computer(s). The children were told that the game manual contained information about the game and that they could look at it if they wished. The children were encouraged to work out any problems they might have amongst themselves. If a puzzle was solved, the children recorded the password provided by the game and the time at which the puzzle was solved. The password for each puzzle could only be obtained by achieving the puzzle's goal. The children were required to record the passwords to provide a confirmation that a particular puzzle had been solved. In addition, having the children write on a piece of paper provided a visual cue for the researchers, during direct observation and video analysis, which indicated that children had achieved the goal of the puzzle. After the children finished playing, they went into the snack/questionnaire section of the room for a snack. The snack consisted of healthy foods (raisins, cheese and crackers, granola bars, etc.) and a drink of milk or juice. Once in the snack area, the children completed a questionnaire and engaged in casual discussion until their forty minutes of research time were completed. Following this, the children returned to their classes. Experimental Variables The independent variables in this phase of the study were cooperative condition and gender grouping. The cooperative conditions included three physical setups using 75 either one or two computers and varying in the amount of interaction implicitly encouraged between the two children. The first condition, Solo Play, consisted of one child playing the game on her or his own. The second condition, Parallel Play, consisted of two children each playing with their own computer. The final condition, Integrated Play, involved two children playing together on one computer. Within the Solo Play condition, both individual girls and individual boys were studied. The gender groupings for both of the pair conditions were girls, boys, and mixed. Table 4.1 shows the number of children assigned to each experimental condi-tion and gender grouping. Table 4.2 shows the number of observations recorded for each cell. Two pairs of girls, two pairs of boys, and one mixed pair were assigned to the Parallel Play condition but chose to play in the Integrated Play setup with both children sharing a single computer. These pairs were included as Integrated Play observations in the analyses. Solo Play Parallel" Play Integrated Play Girls 8 16 16 Boys 8 16 16 Mixed n/a 12 12 "represents the number of children who were given the option of playing in parallel. Five pairs (two girl pairs, two boy pairs, and one mixed pair) chose to play on the same computer. Table 4.1: Number of children assigned to each experimental condition Solo Play Parallel Play Integrated Play Girls 8 6 10 Boys 8 6 10 Mixed n/a 5 7 Table 4.2: Number of observations per experimental condition and gender grouping The dependent measures in this phase of the study were grouped into four 76 categories: achievement in the game, motivation to play, attitude towards collab-orative play on electronic games, and sharing of the mouse during the Integrated Play condition. Qualitative observations were also recorded for twenty-two of the collaborative pairs, selected randomly. Achievement in the game was measured by the number of puzzles each child or pair of children was able to solve in the thirty minutes of play time. For the Parallel Play condition, the scores of the two partners were averaged to produce one score for the pair. For the Integrated Play condition the pair of children received one score for the total number of puzzles they solved. Motivation to play was mea-sured by whether or not the children choose to play for the full thirty minutes1. Children's attitudes towards collaborative play were assessed through a question-naire the children filled out at the end of the session. This questionnaire was made up of twelve Likert-style questions dealing with collaborative and solo play of elec-tronic games, nine multiple-choice questions exploring children's playing experience and preferences, and two short-answer questions asking what the children liked and didn't like about the game. A sample questionnaire can be found in Appendix A . l . The Likert-style questions were summed to produce a score representing the chil-dren's attitude towards collaborative play. The multiple choice questions on how often the children reported playing video or computer games or using computers were scored on a five-point scale, ranging from "never" to "every single day". The remaining multiple choice questions and the two short answer questions were used as supplementary background data. Data on the children's sharing behaviour and qualitative observations were gathered using the videotapes of the sessions. The videotapes were partially transcribed, focusing on the children's sharing behaviour (i.e., how and when control of the mouse was switched between the two children). A sample transcript of a videotaped session can be found in Appendix B . l . 1Time on task is a standard measure of motivation in the literature. We have chosen to use this measure in a discretized form 77 4.1.2 Results and Discussion The data for girls and boys were analyzed separately because we chose to focus on how the collaborative configurations affected each gender individually and not make comparisons between girls' and boys' performance. Nonparametric tests were used to analyze the achievement data from this study because the assumptions of normal distribution and homogeneity of variance were violated. As described in the subsections which follow, results of these analyses revealed no significant effects, possibly due to the low number of children per condition. Table 4.3 shows the mean number of puzzles completed for each of the experimental conditions, by gender groupings. The following sections present trends suggested in the data that were investigated in the second phase of this study. Condition n Mean SD Girls Solo 8 children 1.50 1.69 Parallel 6 dyads 0.42 0.80 Integrated 10 dyads 2.20 1.93 Boys Solo 8 children 1.75 2.19 Parallel 6 dyads 2.50 1.67 Integrated 10 dyads 2.40 2.12 Mixed Parallel 5 dyads 1.40 2.10 Integrated 7 dyads 2.14 2.40 Table 4.3: Mean number of puzzles solved by children in each of the experimental conditions and gender groupings. Solo Play vs. Pair Play Figure 4.4 shows the results for the Solo Play condition compared to the pair con-ditions, Parallel Play and Integrated Play. Only same-gender pairings were used in this analysis because there was no mixed-gender representation for the Solo Play condition. The Mann-Whitney U tests revealed no significant differences in the re-sults for the Solo Play condition and the two pair conditions, but some interesting 78 trends emerged. Both girls and boys solved more puzzles, on average, playing to-gether on the same computer in the Integrated Play condition than playing alone in the Solo Play condition. Boys playing side-by-side on two computers in the Parallel Play condition also solved more puzzles than did boys in the Solo Play condition. Girls playing side-by-side on two computers in the Parallel Play condition solved very few puzzles, even fewer than did girls playing alone in the Solo Play condition. In fact, 75% of girls playing in the Parallel Play condition couldn't solve any puzzles at all, compared to only 38% of girls playing in the Solo Play condition. G i r l s Boys 0Solo Play • P a r a l l e l Play Q Integrated Play Figure 4.4: Mean number of puzzles solved by children in the Solo Play condition and the two pair conditions, Parallel Play and Integrated Play. Only same-gender pairs were analyzed. Not only did the children tend to be more successful playing in some of the collaborative sessions, they also expressed a preference to play video and computer games with a friend. The questionnaire results on children's playing preference are shown in Table 4.4. A chi-square analysis revealed that significantly more girls and boys stated that they preferred to play computer and video games with a friend as 79 opposed to alone: girls, x2 = 180, p < .001; boys, x2 = 18.8, p < .001. Alone With a Friend Doesn't Matter n % n % n % Girls 4 8% 28 54% 20 38% Boys 5 9.5% 31 59.5% 16 31% Table 4.4: Children's playing preference for video and computer games. Collaborative Play on One vs. Two Computers Figure 4.5 shows the results for the Parallel Play condition compared to the Inte-grated Play condition for all three gender groupings. The Mann-Whitney U tests revealed no significant effects, but some interesting trends emerged. Girls play-ing side-by-side in the Parallel Play condition solved very few puzzles on average compared to girls playing together on the same computer in the Integrated Play condition. Boys solved roughly the same number of puzzles on average in the Par-allel Play condition as in the Integrated Play condition. The mixed gender pairs solved more puzzles on average in the Integrated Play condition than in the Parallel Play condition. Motivation The children's motivation to play The Incredible Machine was high, independent of gender or experimental condition, with 99 of the 104 children choosing to play for the full thirty minutes. Given this ceiling effect, analyses of variations in time played were not conducted. Of the five children who left the session early (all girls), three children left because they had solved more than three puzzles (the initial goal set for the children) and there were less than five minutes remaining in the session when they left. The other pair of girls left because one of the girls wanted to have her snack and play in the other end of the room. Even at the beginning of the session, it 80 G i r l s Boys Mixed • P a r a l l e l Play sIntegrated Play Figure 4.5: Mean number of puzzles solved by children playing in pairs either to-gether on one computer, Integrated Play, or on two computers side-by-side, Parallel Play. was clear that this particular girl was not interested in playing the computer game. Mouse Sharing Children playing in the Integrated Play condition solved more puzzles than did children in the Solo Play condition, casual observations indicated that sharing the mouse was a problem for both girls and boys in the Integrated Play condition where two children played together on one computer. The following are two excerpts from the video transcripts which illustrate how children fought over the mouse. Nick i and M a r y 9:21 Nicki tries and fails to pull the mouse from Mary. Mary has managed to position her body and left arm so that they separate 81 Nicki from the mouse. 9:22 Nicki reaches with both arms for the mouse and is pushed back by Mary. 9:23 Saying Let me try, Nicki successfully grabs the mouse from Mary. Mary puts her hand on Nicki's and soon has control of the mouse again. 9:24 Nicki says Let me use that thing and grabs the mouse. Craig and Conner 10:56 Craig says Here can I try? and grabs at the mouse but fails to get it. 10:57 Having said Can I try? three times and being ignored every time, Craig starts to slouch, getting bored. 10:58 Craig says Can I fry? and grabs at the mouse but fails to get it. 10:58 Craig says: Can I try, Conner? Conner gives him the mouse. 10:59 Conner pulls Craig's hand off the mouse and takes it. 10:59 Craig says: Here, c'mon Conner, your hogging the whole thing. Craig reaches for the mouse and Conner gives him the mouse. Using video analysis, the mouse sharing behaviour of twenty-two pairs of children playing in the Integrated Play condition was analyzed. Table 4.5 shows the average number of exchanges, requests and refusals for children in each gender grouping. An exchange occurred when control of the mouse was transferred from one partner to the other partner. A request occurred when a partner asked for control of the mouse. A request could be a verbal statement, such as asking if they could try, or a physical motion attempting to take control of the mouse such as reaching for the mouse. A refusal occurred when one child made a verbal statement or physical action, declining to give control of the mouse to their partner, such as saying no 82 or physically pulling the mouse away. An exchange could occur without a partner initiating a request, or a series of requests could result in only one exchange or refusal. Therefore, there was no direct mapping between the number of exchanges, requests, and refusals. Interrater reliability was not evaluated because the video data was coded by only one observer. Min Max Mean SD Exchanges 7 28 18 6 Girls (n = 8) Requests 1 28 16 8 Refusals 0 14 5 5 Exchanges 13 28 21 6 Boys (n 8) Requests 9 57 27 17 Refusals 0 37 12 14 Exchanges 3 49 21 17 Mixed (n = 6) Requests 2 67 24 24 Refusals 0 25 6 10 Table 4.5: Average number of exchanges, requests and refusals for sharing of the mouse in the Integrated Play condition. Figure 4.6 shows a summary of mouse exchanges, requests, and refusals for all pairs analyzed. The pairs are ordered by number of requests. Sharing patterns in terms of number of exchanges, requests and refusals, are discussed for all three gender groupings. Al l of the eight girl pairs exhibited similar mouse sharing behaviour in terms of the number of mouse requests and exchanges. The number of refusals for each pair varied more, with three pairs having either one refusal or no refusals but two other pairs had higher occurrences of refusals (9 and 14 refusals). A l l of the eight boy pairs exhibited similar mouse sharing behaviour in terms of the number of mouse exchanges between the two partners. The number of requests was also similar across five pairs (average 19 requests), but two pairs had a higher numbers of requests (49 and 57 requests). The number of refusals varied between pairs in the boy grouping. Three pairs of boys had no refusals while two other pairs 83 70 60 S 5 0 r?40 •g 30 W 20 10 0 70 60 to 50 S 4 0 §• 30 0 4 20 10 0 70 60 • 5 4 0 ' 4 - 1 30 <D • - > u 20 10 • 0 • P a i r s o f C h i l d r e n • G i r l s • B o y s Hi Pairs of Children • G i r l s "Boys + 1 1 ji l nil 1 E M i x e d [•'•'''•'•tv 1 1 1 1 1 B M i x e d + I I i l l P a i r s o f C h i l d r e n El G i r l s " B o y s E M i x e d Figure 4.6: Summary of the number of exchanges, requests and refusals for each pair of children by gender grouping. Pairs are ordered by number of requests. 84 had over 25 refusals during their session. The six mixed pairs demonstrated three different sharing patterns. Three of the pairs had a small number of exchanges, requests, and no refusals, therefore, little interaction between the partners. Two of the other pairs had a large number of exchanges, requests and refusals, while the final pair had a very high number of exchanges, requests and refusals, suggesting that both children were active but that contention occurred over sharing the mouse. The methods the children used to request the mouse can be categorized into four types: (a) no-contact; (b) verbal; (c) touch; and (d) pull. A no-contact request occurred when one partner made a motion towards acquiring the mouse (i.e., reaching for the mouse) but didn't actually make physical contact with the partner. A verbal request occurred when a child asked the partner if they could have the mouse. A touch request occurred when a child touched the partner's hand to signify that they wanted the mouse. Finally, a pull request was one in which the child physically tried to take the mouse away from the partner. The distribution of occurrences in these mouse request categories is shown in Figure 4.7 for each gender grouping. For girl pairs, a Friedman test demonstrated significant differences in the number of occurrences of each category of requests, x 2 = 9.86, p < .05. Touch was the dominant method for requesting control of the mouse and a Wilcoxon post-hoc analysis revealed that girls used the touch method of requesting control of the mouse significantly more than they used either the non-contact or pull methods, p < .05. For boy pairs, a Friedman test demonstrated significant differences in the number of occurrences of each category of requests, x 2 = 19.54, p < .001. Touch was the dominant method for requesting control of the mouse and a Wilcoxon post-hoc analysis revealed that boys used the touch method of requesting control of the mouse significantly more than they used any of the other methods, p < .05. The non-contact method of requesting control of the mouse was the least used method 85 25.On G i r l s Boys Mixed B N o - c o n t a c t " V e r b a l E T o u c h • P u l l Figure 4.7: Distribution graph showing the number of occurrences and types of requests for control of the mouse. by boys compared to all other methods, p < .05. For mixed pairs, a Friedman test demonstrated no significant differences in the number of occurrences of each category of requests, X 2 = 5.53, ns. Although the average use of the methods is similar to the other two gender groupings, as mentioned in Section 4.1.2, children's behaviour in the mixed grouping varied a great deal across pairs. If the six mixed pairs were grouped into two categories, a low-interaction group and high-interaction group, each with three pairs, the distribution of mouse requests for these two groups is quite different, as shown in Figure 4.8. Background Demographics The children in this study were very familiar with computer and video games: 84% had computers at home and 91% had a video game player at home. Only two of the 104 children had neither a computer nor a video game player at home. Children's self-reports on how often they used computers and played video and computer games 86 Figure 4.8: Number of mouse requests by mixed pairs, contrasting pairs that exhib-ited a low level of interaction with those that exhibited a high level of interaction. are shown in Figures 4.9 and 4.10, respectively. No significant correlations were found between how often children used computers or played video and computer games and how many puzzles they were able to solve in this phase of the study. Using the Mann-Whitney U test, no significant difference was found between how often girls and boys used computers, but boys reported playing video and computer games significantly more often than did girls, U = 950, p < .01. Qualitative Observations from the Video Data Qualitative observations retrieved from the video data revealed that children inter-acted differently in the various collaborative conditions, and gender differences were clearly evident. We previously highlighted a key problem with the Integrated Play condition that was revealed by the video data collected : sharing the mouse. This can be seen in the excepts from the videotapes presented earlier where Nicki and Craig both had difficulty getting use of the mouse from their partners, Mary and 87 25.0 G i r l s Boys 0 Never use computers • Use computers a few times a year Q Use computers a few times a month • Use computers a few times a week Q Use computers every day Figure 4.9: How often girls and boys reported they used computers. a u o u 25.0' 1IBIBB BBBBBB nnaaun niannn B B B B B B 1 B B B B B 1 B B B B B I B B B B B I B B B B B I B B B B B Figure 4 games. G i r l s Boys • Never use computers • Use computers a few times a year 0 Use computers a few times a month • Use computers a few times a week n Use computers every day 10: How often girls and boys reported they played video and computer 88 Conner. For children playing in the Parallel Play condition, a different problem arose - because each child was working on a separate game, there was no incentive to help the partner. Some children frequently asked for help from the partner but were ignored. When two children work together on the same game, each has a vested interest in solving the puzzle and therefore helping the other. For example, the following description taken from the videotape of a session that shows one boy constantly asking for help, but his partner is only concerned with his own game and ignores his partners requests. In this segment, the two boys, Henry and Liam, are playing on side-by-side computers in parallel play mode. Henry is quite talkative, but Liam is fairly quiet. Often when Liam talks, he mumbles to himself. Henry leans back and forth between his own computer and Liam's computer frequently while Liam rarely looks over at Henry and focuses primarily on his own screen. Liam: Oh, I did something. Oh Cool, look what I did. Henry: Henry leans over and looks at Liam's screen How'd you do that? Liam: giggles and mumbles Oh Cool. Henry: How'd you do that? Henry: How'd you do that? Both boys giggle. Oh wait, I think I know. Oh I know! Oh yeah, I got it, I got it, I got it, I got it, I got it. How did you do that? I don't get it! Liam: mumbles I'm smart. Oh this is cool. Yes, Oh, look at it. Henry: How did you do it? This is working out finally. Liam: 89 Henry: But how did you get the black thing? How did you get the black thing? No really, how did you get the black thing? Liam: I think I can do this. Henry: No really, how did you get the black thing? How did you get the black thing? Liam: Yes, oh yes, I did it. Liam has solved the first puzzle. Henry: How did you get the black thing? Liam: Liam points to his screen as he explains to Henry what the black thing is. It's an elastic band. Connect it to here and then connect it to one of these Henry: I KNOW but ... I don't get it. Liam: mumbles to himself then reaches for his sheet to record the password for the puzzle he has solved. Henry: But I don't get it. What do you do with the elastic band? How did you do that. How did you get the black elastic band. Henry taps Liam on the arm and says Wait. How did you get the black elastic band. This excerpt is a good example of the type of data that can be captured through the use of videotapes. Videotape data can provide rich dialog, non-verbal cues and contextual information to help interpret the results of the study. The language that children used in each of the conditions also suggests that children playing on separate machines lacked positive interdependence. In the In-tegrated Play condition, children tended to talk using group language such as "we did it". Children in the Parallel Play condition instead tended to use individual language such as "I did it". Previous research has revealed that children are more 90 likely to refer to the dyad ("we/us" pronouns) and less likely to refer to themselves ("I/me" pronouns) and their partners ("you" pronoun) while playing cooperative games compared to playing competitive games [Hymel, Beaumont, McDougall, and Zarbatany, 1997]. This suggests that children playing in the Integrated Play condi-tion may have viewed the task as a cooperative task more than did children playing in the Parallel Play condition. Both field observations and analysis of the videotapes revealed that children playing with a partner often engaged in detailed discussions about how to solve the puzzles in the game and sharing ideas. In this next segment, two girls, Laura and Cindy, are working together, solving puzzles and passing the mouse back and forth, each giving suggestions. Cindy is operating the mouse and Laura has come up with an idea. Laura: Oh! I see. We get that blower thing ... and "push" Cindy: We need the ball to drop on to it. Laura: No, that's the wrong way around. Cindy: [inaudible] Laura: No, because that's the blower part of it. Cindy: I see, I see, I see. Laura: OK, we need a ball to drop on that. In addition, the observations also suggested that children playing in the In-tegrated Play condition had more verbal interaction with their partners than did children playing in the Parallel Play condition. This is an important observation because the discussion of thoughts and ideas may be an integral part of the learning process [Johnson et al., 1981; Ross and Raphael, 1990; Yager et al., 1985}. The Par-allel Play condition in this phase did not seem to encourage this type of discussion. 91 4.2 Phase 2: Science World, 1994 The second phase of this research was designed to further investigate the results of the first phase, using a larger sample size to explore statistical differences in achievement and motivation. Again children interacted with an educational com-puter game, but this time the study took place in an informal learning environment rather than a school environment, and the children may or may not have known their partners in this second phase. 4.2.1 Method Subjects and Setting This phase of the study was conducted at Science World, during the summer of 1994. The participants were 331 children (247 girls and 84 boys) between the ages of 9 and 13. The disproportionate number of girls was deliberate because the playing condition appeared to have an effect on the number of puzzles girls were able to solve and therefore it was important to ensure that the power of the study for girls was sufficient to detect significant differences between experimental conditions. The children who participated were visitors to Science World who volunteered to take part in the study and whose parents signed consent forms. As in the first phase, none of the children had previously played the computer game The Incredible Machine. The children were chosen to play either alone or with a partner in one of three experimental conditions: Solo Play: children playing alone on one computer Parallel Play: children playing with a partner side-by-side on two separate computers Integrated Play: children playing with a partner on one shared computer 92 For the two-player conditions, the children were assigned a partner of the same gender, who was not more than one year older or younger. The children may or may not have known their partners previously. Because of the added complexity of mixed gender interactions only same gender pairs were used in this phase of the study. The setting was a research lab set up on the second floor of Science World. The research lab consisted of four Macintosh LCIII computers placed on a large table. The use of multiple computers facilitated running multiple sessions simul-taneously. A subset of children playing in either of the two pair conditions were videotaped. As in the first phase, a video camera captured the children's faces and body movements and a VGA-to-NTSC converter and VCR captured the children's progress in the game. The children wore lavaliere microphones to capture their discussions throughout the session. Lavaliere microphones were necessary because of the high noise level at Science World. The entire research lab was enclosed by four-foot risers, helping to block out distractions for the children and to prevent people passing by from interfering with the session. Procedure The length of each experimental session was approximately thirty minutes. The procedure for this second phase was identical to the first phase except that no snacks or questionnaires were provided. When the children finished playing, they left the research lab and continued to explore Science World. We wanted to run multiple sessions simultaneously in order to decrease the time required for the study. Four computers were needed to enable this. At the time, we had access to more Macintosh computers than IBM-compatible comput-ers. Because The Incredible Machine runs on both platforms, we chose to use the Macintosh computers. After initially setting up the environment and observing a few children, it 93 became clear that children were having difficulty with the interface in the game. As discussed in Chapter 3, while both the IBM-compatible and the Macintosh versions of The Incredible Machine looked identical on the surface, each utilized a different interaction style. The Macintosh version utilized a drag-and-drop interaction style while the IBM-compatible version of the game utilized a point-and-click interaction style. We found that the children experienced difficulty using the drag-and-drop version of the game and this problem could affect how well the children performed in any of the experimental conditions. In order to compensate for this problem, a discussion of the interface was given to the children before the sessions began. The children were shown how to move pieces around in the game and the inconsistencies of the interface were pointed out to the children. Experimental Variables The independent variables in this phase of the study were the same as in the first phase: cooperative condition and gender grouping. Again, the cooperative condi-tions included the three physical setups of Solo Play, Parallel Play, and Integrated Play. The gender groupings included girls and boys in the Solo Play condition, and girl pairs and boy pairs in the Parallel and Integrated Play conditions. No mixed gender pairs were used in this phase of the study because previous research has indicated that there may be an interaction of gender with other variables when us-ing mixed gender pairs, complicating the results [Lockheed and Hall, 1976]. Table 4.6 shows the number of observations in each experimental condition and gender grouping. In this phase, children assigned to the Parallel Play condition were re-quired to play individually on two side-by-side computers; they could not choose to collaborate on just a single computer. The Solo Play condition had significantly more observations because of the nature of recruiting children as subjects at Science World. The day was divided 94 Solo Play Parallel Play Integrated Play Girls 155° 23 23 Boys 46 9 10 "three outliers were excluded from the analyses because their values were more than three standard deviations away from the group mean. Table 4.6: Number of observations per experimental condition and gender grouping up into several thirty minute blocks and children signed up for a particular time block. During some times, only one girl or boy signed up. As a result they would be required to play in the Solo Play condition because a partner was not available. Only two dependent measures were investigated in this phase of the study: achievement in the game and motivation to play. Achievement in the game was measured by the number of puzzles each child or pair of children could solve in thirty minutes of play time. For the Parallel Play condition, the scores of the two partners were averaged to give one score for the pair. For the Integrated Play condition, the pair of children received one score for the total number of puzzles they were able to solve. Motivation was measured by whether or not the children chose to play for the full thirty minutes. 4.2.2 Results and Discussion As in the first phase, the data for girls and boys were analyzed separately. Non-parametric tests were used to analyze the achievement data from this study because the assumptions of normal distribution and homogeneity of variance were violated. Three outliers were removed from the girls Solo Play condition because their values were more than three standard deviations away from the group mean. Table 4.7 shows the mean number of puzzles completed for each of the ex-perimental conditions, by gender groupings. The Kruskal-Wallis test revealed a significant overall effect of experimental condition for girls, X 2 — 6.73, p < .05. For 95 boys, the Kruskal-Wallis test found no significant effect of experimental condition, X2 = 1.52, ns. Condition n Mean SD Girls* Solo 152 1.14 1.71 Parallel 23 0.98 1.36 Integrated 23 1.83 1.67 Boys Solo 46 2.35 2.37 Parallel 9 2.78 1.91 Integrated 10 3.30 2.50 *an overall effect of condition was found for girls, p < .05 Table 4.7: Mean number of puzzles solved by children in each of the experimental conditions and gender groupings. Solo Play vs. Pair Play Figure 4.11 shows the results for the Solo Play condition compared to the two pair conditions, Parallel Play and Integrated Play. Mann-Whitney U tests were used to analyze the solo and pair data. Girls solved significantly more puzzles playing together on the same computer in the Integrated Play condition than did girls playing alone in the Solo Play condition, U — 1221, p < .05. No significant difference was found between girls playing side-by-side on two computers and girls playing alone in the Solo Play condition, U = 1686, ns. Boys in both pair conditions solved more puzzles on average than did boys playing alone, but these differences were not statistically significant, U = 177, ns. Collaborative Play on One vs. Two Computers Figure 4.12 shows the results for the Parallel Play condition as compared to the Integrated Play condition for both gender groupings. The Mann-Whitney U test for girls revealed a marginally significant result, U = 186, p = .072, with girls tending to solve more puzzles playing together on one computer than playing side-by-side 96 Figure 4.11: Mean number of puzzles solved for the solo and pair conditions on two computers. Boys solved slightly more puzzles on average playing together on one computer than playing side-by-side on two computers, but this difference was not statistically significant, U = 39.5, ns. Motivation The percentages of children who left the session early is shown in Figure 4.13. A chi-square analysis revealed a marginally significant overall effect of condition on the proportion of girls who were motivated to stay and play the game, X 2 = 5.726, p = .057. Girls playing alone in the Solo Play condition had the highest percentage leave the session early, while playing with a partner on the same computer in the Integrated Play condition had the lowest percentage of girls leave the session early. Boys playing alone in the Solo Play condition had the highest percentage leave the session early although a chi-square analysis found no significant effect of condition on boys' motivation, X 2 = 0.36, ns. 97 Boys • P a r a l l e l Play • Integrated Play Figure 4.12: Mean number of puzzles solved for children playing side-by-side on two computers or playing together on one computer Other Observations The results from this phase of the study validate the achievement trends observed in the first phase for girls. Girls solved significantly more puzzles playing together on one computer in the Integrated Play condition than playing by themselves in the Solo Play condition. Girls also tended to solve more puzzles playing with a partner if they played together on one computer than playing side-by-side on two computers. This phase also repeated the trends observed for boys in the first phase. Boys on,average solved more puzzles playing with a partner in one of the pair conditions than playing alone in the Solo Play condition, although these differences were not statistically significant. Boys also completed more puzzles playing with a partner if they played together on one computer as opposed to playing side-by-side on two computers, although this difference was not statistically significant. 98 H 25 D -i <u G i r l s Boys B S o l o P l a y • P a r a l l e l P l a y B I n t e g r a t e d P l a y Figure 4.13: Number of girls and boys who left the session early. 4 . 3 Conclusion Achievement The results of this study indicated that children's success when playing with a partner may depend on their playing condition. Pairs of children playing together on one computer solved more puzzles on average than children playing alone or pairs of children playing side-by-side on two computers. This was especially true for girls where the differences were statistically significant. This result supports previous research that demonstrates the advantages of small groups sharing a single computer [Dalton, Hannafin, and Hooper, 1989; Hall and Cooper, 1991; Steiner and Moher, 1994]. Because the children had never played The Incredible Machine before, this finding is also consistent with research that shows co-discovery learning is more effective than self-discovery when using a new computer system [Lim, Ward, and Benbasat, 1997]. The benefit of children working together on one computer could be attributed 99 to the increased verbal interaction that results. During our study we observed that children playing in the Integrated Play condition had a greater frequency of verbal discussion than did children playing in the Parallel Play condition. This notion of increased verbal interaction during collaborative work on one computer is also supported by other research: children playing together on one computer were observed to have more verbal interactions than children playing side-by-side on two computers [Steiner and Moher, 1994]. Elaboration, the discussion of and expanding on ideas, is recognized by many researchers as one of the underlying cognitive explanations for the benefits of cooperative learning [Slavin, 1990; Yager, Johnson, and Johnson, 1985]. The results of this study impact the notion that one computer for every child is ideal, especially for girls, given the trend that that in both phases of our study, girls playing side-by-side on two computers solved fewer puzzles on average than did girls playing by themselves. This stresses the importance of understanding children's interactions with computers so as to not unintentionally employ computers in a manner that is ineffective for some children. Motivation Not only were the children more successful playing together on one computer, they were more motivated to play in this configuration than when playing side-by-side on two computers or when playing alone. This result was dependent on the environment in which the study took place. Ninety-five percent of the children who took part in the first phase of the study, in the school environment, stayed for the full thirty minutes of the session. Many of these children expressed that they enjoyed taking part in the study and that it was a desirable break from their regular school day. Given this, it is not surprising that the vast majority of students remained for the full thirty-minute session, regardless of experimental condition. In the second phase of the study, at Science World, the children who took part gave up time that could 100 have been spent at other highly attractive exhibits. Under these conditions, more children left early when playing alone, and fewest children left early when playing with a partner in the Integrated Play condition, although these effects were only significant for girls. Overall, these results suggest that cooperative computer play may be more engaging for children, especially girls. A second factor that could have affected the children's motivation to play the game was the type of platform and interaction style used. The first phase of the study used the IBM-compatible version of The Incredible Machine, which utilized a point-and-click interaction style. The second phase of the study used the Macintosh version of The Incredible Machine, which utilized a drag-and-drop interaction style. Children using the Macintosh version of the software had difficulty using the drag-and-drop interaction style. The use of a more difficult interaction style may have affected the children's motivation to play the game. A subsequent study of girls using both types of interfaces showed a significant difference in the average number of puzzles solved, and in motivation to play the game between the two interaction styles (that study is described in Chapter 6). Children's motivation to play in the second phase of this study appeared to be linked to two factors: success in the game and whether or not the children played with a partner. Success in the game seemed critical because all but four of the children who solved at least one puzzle stayed and played for the full thirty-minute session. Forty-nine of the 53 who left did so without solving any puzzles at all. The remaining four children left after solving only one puzzle. Whether or not the children played with a partner also appeared to have contributed to children's motivation. The Solo Play and the Parallel Play conditions had a similar percentage of girls who couldn't solve any puzzles, 60% and 56% respectively, but a higher percentage of girls in the Solo Play condition chose to leave early. 101 Gender Although the achievement and motivation patterns found in this study were similar for girls and boys, the results for girls revealed significant differences across collab-orative condition while the results for boys did not reveal significant differences. This may be partially attributed to the smaller sample size gathered for boys but it also suggests that girls and boys may interact differently in the various playing configurations or that playing configuration may have a stronger impact for girls than for boys. We did not set out to explore differences between girls and boys, but our observations revealed that in both phases of our study, boys solved more puzzles on average than did girls. This suggests a possible gender bias in our study resulting from our experimental design or from our choice of software. Both girls and boys enjoyed playing The Incredible Machine, but the nature of this game, or the task of solving puzzles may have been of more interest to boys than to girls. The results of this study support findings of other researchers, which indicate that girls and boys interact differently with computers [Hall and Cooper, 1991; Wilder et al., 1985]. These results also emphasize the need to be aware of gender differences when researching issues related to children's use of computers or when developing hardware and software to be used by children. Mouse Sharing Although the children in this study enjoyed playing with friends, sharing control of the mouse was difficult for some pairs. One solution is to add a second mouse to the computer and explore how this change affects children's interactions in the environment. Finding a system that would allow children to work together as well as maintain the ability for individual exploration could be an important advance in collaborative learning with computers. This possibility is investigated further in the next chapter. 102 Chapter 5 Multiple Input Devices • Sometimes sharing the mouse gets to be a little physical. From Questionnaire in 1994 Kerrisdale School Study: Question #21: What did you like least about The Incredible Machine Answer: "My friend had more time than me." 103 Research reported in the previous chapter showed that children enjoy play-ing computer games with friends and sometimes are more successful as a result of peer collaboration. One of the main difficulties in having small groups of children work together on the same computer is that contention can arise over sharing input devices, such as the mouse that is used to control many computer games [Bricker et al., 1995}. One possibility to overcome the difficulty of sharing the input device would be to provide each child with an individual input device. Video games often have multiple controllers, but computer games seldom have multiple mice. It is unclear whether having control of the input device affects children's achievement or learning in a collaborative session. As observed in an early E-GEMS study (see Section 2.8), children playing single-player video games sometimes want to hold the inactive controller while waiting for a turn, even if the inactive controller is never used in the game. Why is it important to hold on to a device that has no impact in the game? Perhaps there is a heightened sense of engagement when children have possession of an input device. Whether or not control of the mouse is a good indicator of how well a group is working together is unclear. A previous study qualitatively explored mixed-gender groups of four children working together on one computer [Cole, 1995]. Cole con-cluded that although control of the mouse played a significant role in the group dynamics, it was not a reliable indicator of group collaboration. While qualitative observations such as Cole's are important in understand-ing how groups interact with each other, possessing control of the mouse may af-fect children in ways an observer cannot see. Our study performed a quantita-tive analysis, examining same-gender pairs of children working in the collaborative problem-solving environment of a mouse-driven computer puzzle game. We wanted to determine if access to and control of the mouse affected children's achievement, learning, and sharing patterns in a collaborative environment. Achievement refers to the children's progress in the collaborative environment. Learning refers to the 104 children's ability to transfer skills and knowledge from the collaborative session to a subsequent solo session. Sharing patterns are the ways in which children manage resources while playing in the collaborative environment. Three turn-taking proto-cols for sharing control of the mouse-driven cursor were examined: (a) two children sharing a single mouse; (b) two children, each with their own mouse, using a "give" protocol to transfer control between the two mice; and (c) two children, each with their own mouse, using a "take" protocol to transfer control between the two mice. 5.1 Phase 1: Science World, 1994 The first phase of the study investigated whether having multiple input devices for a single-user computer game affected the children's interactions or their achievement in the game. Different protocols for passing control between the two mice were explored. 5.1.1 Method Subjects and Setting This phase of the study was conducted at Science World. The participants were 138 children (72 girls and 66 boys) between the ages of 9 and 13 who had not previously played The Incredible Machine. The children who participated were visitors to Science World who volunteered to take part in the study and whose parents signed consent forms. The children were chosen to play with a partner on a single computer in one of three experimental conditions: One-Mouse Shared: two children sharing a single mouse Two-Mouse Give: two children, each with their own mouse, using a "give" protocol to transfer control between the two mice 105 Two-Mouse Take: two children, each with their own mouse, using a "take" protocol to transfer control between the two mice The children were assigned a partner of the same gender, who was not more than one year older or younger. The children may or may not have known their partners previously. The setting was a research lab on the second floor of Science World. The research lab consisted of an IBM-compatible computer, running The Incredible Ma-chine, and a Silicon Graphics Personal IRIS workstation (SGI), used to support the two-mouse conditions. The entire research lab was enclosed by four-foot risers to prevent people passing by from interfering with the session. In order to allow for the use of two mice in the game, the SGI received input simultaneously from two serial mice and determined which of the two mouse inputs would be sent to the IBM-compatible computer running the game software (see Figure 5.1). The game software was not modified in any way for the two-mouse conditions. There still was only one cursor visible on the screen and only one mouse was active at a time. The serial mice used in the experiment had two buttons. The game software only used the left mouse button, so the right button was reserved for exclusive use by the turn-taking protocol. Instead of physically passing a single shared mouse between two partners, control of the cursor was transferred between the two mice by button clicks on the mice. To provide feedback on turn-taking, the SGI workstation screen displayed a large arrow to indicate to the children which mouse was active. Two protocols for passing control of the game cursor between two mice were examined: give and take. In the two-mouse give condition, when the partner in control of the game cursor pressed the right mouse button, control of the game cursor would be passed over to the other partner's mouse (nothing happened if the partner not in control of the game cursor pressed the right button). In the two-mouse take condition, either partner could take control of the game cursor by 106 Figure 5.1: Computer setup for the two-mouse conditions pressing the right mouse button at any time (for the partner who had control, this would be a "no-op", but the software permitted this action). The SGI recorded when control of the mouse was passed between the two children. Figure 5.2 shows a pair of children playing in one of the two-mouse conditions. Both the children and their interactions within the game were recorded using video. A video camera with a tripod was placed behind the computer table, off to the right, against a wall. The camera was positioned to capture the children's faces and body movements, but so as to not be distracting to the children. The children wore lavaliere microphones to capture their discussions throughout the session. Lavaliere microphones were necessary because of the high noise level at Science World. 107 Figure 5.2: Children playing with the two-mouse configuration. Procedure The total length of each experimental session was forty minutes. At the beginning of a session the children were welcomed and given a brief introduction to the study and to the game, The Incredible Machine. The children were given a short verbal discussion on how to manipulate objects in the game but no directions on how to solve puzzles in the game were given. The game manual was placed on the table beside the computer and the children were told that it contained information about the game and that they could look at it if they wished. Children playing in either of the two-mouse conditions were also shown how to switch control between the two mice. The children were asked to try to complete as many puzzles as they could and they were told that they would be allowed to play for as long as they desired up to a maximum of thirty minutes. If a puzzle was solved, the children recorded the password provided by the game and the time at which the puzzle was solved. The password for each puzzle could only be obtained by achieving the puzzle's goal. 108 As with similar experiments, the children were required to record the passwords to provide a confirmation that a particular puzzle had been solved. In addition, having the children write on a piece of paper provided a visual cue for the researchers, during direct observation and video analysis, which indicated that children had achieved the goal of the puzzle. Experimental Variables The independent variables in this phase of the study were cooperative condition and gender grouping. The cooperative conditions included three configurations of a computer equipped with either one or two mice and two protocols for switching control when two mice were used. The one-mouse shared condition consisted of two children playing together on one computer, sharing a single mouse. The two-mouse give condition consisted of two children playing together on one computer, each with a separate mouse, using a give protocol to transfer control between the two mice. The two-mouse take condition consisted of two children playing together on one computer, each with a separate mouse, using a take protocol to transfer control between the two mice. The two gender groupings were pairs of girls and pairs of boys. Each gender was analyzed separately and no mixed-gender pairs were studied. Table 5.1 shows the number of children assigned to each experimental condi-tion and gender grouping. The number of observations for each cell was half of the number of children in each cell because the children worked in pairs. One-Mouse Shared Two-Mouse Give Two-Mouse Take Girls 24 24 24 Boys 20 24 22 Table 5.1: Number of children assigned to each experimental condition The dependent variables in this phase of the study were achievement and collaborative behaviour. Achievement in the game was measured by the number of 109 puzzles each pair of children was able to solve in the thirty minutes of play time. Collaborative behaviour was examined through qualitative analysis and, for the two-mouse conditions, the mouse sharing data logged by the SGI. 5.1.2 Results and Discussion As in the previous studies, the data for girls and boys were analyzed separately because we chose to focus on how the collaborative configurations affected each gender individually and not make comparisons between girls' and boys' performance. Nonparametric tests were used to analyze the achievement data from this study because the assumptions of normal distribution and homogeneity of variance were violated. Achievement Table 5.2 shows the mean number of puzzles completed for each of the experimental conditions, by gender grouping. The median test on achievement revealed a signifi-cant overall effect of experimental condition for girls, X 2 — 8.67, p < .05. The median test found no significant effect of experimental condition for boys, X 2 = 1-52, ns. The following two subsections explore the post-hoc analyses for the one- versus two-mouse conditions and the give versus take conditions. Condition Number of Dyads Mean SD Girls One-Mouse Shared 12 1.42 1.56 Two-Mouse Give 12 2.83 1.40 Two-Mouse Take 12 1.75 2.05 Boys One-Mouse Shared 10 3.30 2.50 Two-Mouse Give 12 2.50 2.07 Two-Mouse Take 11 4.36 3.04 Table 5.2: Mean number of puzzles solved by children in each of the experimental conditions and gender groupings. 110 One Mouse vs. Two Mice Figure 5.3 shows the results for the one-mouse shared condition compared to the two-mouse conditions (two-mouse give and two-mouse take). The post-hoc analyses revealed a significant difference between the one-mouse shared condition and the two-mouse give condition for girls, Mann-Whitney U — 38, p < .05. Girls were able to solve significantly more puzzles using two mice with a give protocol than they did sharing a single mouse. No other significant differences were revealed by the post-hoc tests for girls. Boys solved the most puzzles on average in the two-mouse take condition although no significant effect of condition was found. Figure 5.3: Mean number of puzzles solved by children sharing a single mouse in the one-mouse shared condition, and children using two mice in the two-mouse give and two-mouse take conditions. I l l Give vs. Take The post-hoc analyses for the two-mouse give and the two-mouse take conditions revealed no significant differences for either girls or boys (girls: U = 55, ns and boys: U = 49, ns). Examining the data for trends suggests that children's interactions do differ as a result of the protocol used to transfer control between the two mice. Girls solved more puzzles on average in the two-mouse give condition than in the two-mouse take condition. The results for boys were the opposite, boys solved more puzzles on average in the two-mouse take condition than in the two-mouse give condition. Mouse Exchanges Table 5.3 shows the mean number of times control of the cursor was passed between the children in the two-mouse conditions. An independent t test found no effect of condition for girls in terms of the number of exchanges, £(22) = 0.30, ns, while condition did have a significant effect on the number of exchanges for boys, t(20) = 2.31, p < .05. Boys in the two-mouse take condition exchanged control significantly more times than did boys in the two-mouse give condition. Condition Number of Dyads Mean SD Girls Give 12 28.67 12.41 Take 12 30.75 20.44 Boys Give 12 28.82 6.23 Take 11 46.36 24.37 Table 5.3: Mean number of exchanges in the two-mouse conditions. Mouse Sharing Observations The qualitative observations of children playing in the three collaborative configu-rations in this study revealed some interesting patterns for children's interactions. 112 Girls playing in all three experimental conditions were reluctant to take the mouse from their partners. Even in the case of the two-mouse take condition, many girls would first ask the partner if it were OK before taking control of the cursor. Girls playing in the two-mouse give condition were observed passing control to the partner without the partner requesting an exchange. Occasionally, after trying out several ideas, a girl would press her give button and say to her partner, "Here, you try." This action prompted the girl's partner to share her ideas, which she might have been reluctant to otherwise bring forward. Prompting for ideas was not as evident in either of the other two conditions. This could explain why girls were more suc-cessful while playing in the two-mouse give condition than in the one-mouse shared or in the two-mouse take condition. Observations of boys sharing a single mouse in this study and in our previous research revealed that struggles can occur over who controls the mouse. Some boys were observed physically preventing their partner from taking the mouse by shielding them from the mouse with their bodies. Because of this behaviour, the researchers were concerned that boys using the two-mouse take condition would spend more time fighting over the mouse because of the ease of obtaining control. This was not the case. In fact, boys seemed to fight over control of the mouse less than while sharing a single mouse. This could be attributed to the fact that when sharing a single mouse, if one boy allows the partner to take control of the mouse, the partner could physically prevent the boy from regaining control of the mouse. As a result, some boys may have been reluctant to give the mouse to the partner for fear of not getting the mouse back. In the two-mouse take condition, both boys have easy access to control of the mouse. The boys appeared more tolerant of their partners taking control perhaps because they knew that getting control back was easy. The children in this phase of the study appeared to adapt easily to the ad-dition of the second mouse and many expressed a preference for this type of config-uration. One difficulty which did occur for many children playing in the two-mouse 113 give condition was the problem of accidentally pressing the right-mouse button and inadvertently giving control of the game to the partner. Children either pressed the wrong button while trying to interact with the game or bumped the right mouse-button while pressing the left mouse-button. This problem did not occur for the two-mouse take condition because only the inactive partner could cause control to be passed. If the partner currently in control pressed the right mouse-button, noth-ing would happen because that partner already had control. If the partner not in control pressed their right mouse-button, they obviously wanted control of the game because there would be no other reason to press the mouse button because they were not controlling the game. Individual Learning In this phase of the study it was possible that one partner did most of the puzzle solving while the other partner did not engage in the puzzle solving activity. As a result, one partner may have learned how to solve puzzles in the game while the other partner gained very little new knowledge. Because a pair of children is given one score for the composite effort there is no way to distinguish the amount of knowledge that any individual gained from the session. The next phase of the research attempted to address this issue by measuring individuals' learning. 5.2 Phase 2: Vancouver Elementary Schools, 1996 The second phase of the study was designed to extend the results of the first phase by exploring the transference of children's skills and knowledge from the collaborative session to a subsequent solo session. We will label this transfer as learning in our discussion. We also wanted to examine the impact children's sharing patterns had on their achievement and learning while playing collaboratively. Children again played an educational computer game in one of three collaborative setups, but this time the children also played in a second session by themselves in an attempt to 114 measure the knowledge the children gained while playing in the first collaborative session. This phase of the study took place in school computer labs rather than the informal learning environment of Science World. A more complete description of the experimental design for this study is given by McGrenere and Inkpen [1996]. 5.2.1 Method Subjects and Setting This phase of the study was conducted at four elementary schools on the East Side of Vancouver, British Columbia, during May and June of 1996. The time spent at each school ranged from two to four days, depending on the number of classes participating in the study. The participants were 250 children (126 girls and 124 boys) between the ages of 9 and 12, from twenty Grade Five and Grade Six classrooms. The children volunteered to take part in the study and permission was given by their parents. Of the classes involved in the study, a 70% participation rate from the students was achieved. As in the first phase, none of the children had previously played the computer game The Incredible Machine. The children were required to play with a partner on a single computer in one of three experimental conditions: One-Mouse Shared: two children sharing a single mouse Two-Mouse Give: two children, each with their own mouse, using a "give" protocol to transfer control between the two mice Two-Mouse Take: two children, each with their own mouse, using a "take" protocol to transfer control between the two mice A control group of children playing by themselves was also examined. The children taking part in the collaborative sessions were assigned a partner of the same gender from the same classroom. Children in the control group played without a partner. 115 The sessions were run in the computer laboratories of the schools, all of which had IBM-compatible computers. As in the first phase of the study, a Silicon Graphics Personal IRIS workstation (SGI) was brought into the laboratory to support the two experimental conditions that required multiple mice (see Section 5.1.1). Regular use of the computer laboratory was restricted during the majority of the time the study was being run. Children's hands in the one-mouse shared condition were videotaped to record when the children exchanged control of the mouse. The video camera was placed on the computer table, off to the side of the computer, focused downwards on the children's hands. Figure 5.4: Children's hands were videotaped to record when the children exchanged control of the mouse 116 Procedure This phase of the study was conducted in sets of two sessions. The first session involved having children play The Incredible Machine using one of three turn-taking protocols (the collaborative session). The second session involved the students play-ing alone (the solo session). Each session lasted approximately forty minutes (one class period). The second session took place one to three days after the first session and the children played similar but different puzzles from the ones presented in the first session. The control group also played for two forty-minute sessions, but they played by themselves in both sessions. Before taking part in the study, all children were asked to complete a background questionnaire to assess their electronic game playing experience and preferences. The questionnaire was similar to the one shown in Appendix A . l . The three experimental conditions in the collaborative session were: (a) one-mouse shared, in which two children played together on one computer sharing a single mouse; (b) two-mouse give, in which two children played together on one computer, each with a separate mouse, using a give protocol to transfer control of the game between the two mice; and (c) two-mouse take, in which two children played together on one computer, each with a separate mouse, using a take protocol to transfer control of the game between the two mice. The control group played by themselves on a single computer with one mouse. The first experimental session consisted of welcoming remarks by the re-searcher, a brief introduction to the experimental study and to the game, a ten minute hands-on interface training session, and twenty-five minutes of time playing The Incredible Machine. The interface training session was intended to give the children experience in manipulating objects in the game to help eliminate problems with the user interface during the sessions. The interface training session was conducted individually so that all children became familiar with the interface. The training session demonstrated 117 to the children how to begin playing a puzzle, how to move objects from the parts bin onto the playing screen, how to hook objects together, how to flip objects, how to size objects, and how to run the machine by clicking on the appropriate icon on the screen. Next, each child was given a picture of a screen from The Incredible Machine with several objects placed on the playing screen (see Figure 5.5). The children were asked to duplicate this picture using the game to ensure that they were able to perform all operations required to play the game. The configuration of objects on the playing screen required the children to move objects from the parts bin to the playing screen, flip a pair of scissors, make a ramp smaller, make a ramp larger, and attach an elastic between two gears. Upon completion of the interface training session, the children were assigned a partner and an experimental collaborative condition or assigned to the control group. The children were given twenty-five minutes to solve puzzles in the game and were asked to solve as many puzzles as they could. They were told that they could play for as long as they desired up to a maximum of twenty-five minutes. When the children solved a puzzle, a researcher recorded the time at which the puzzle was solved and then helped the children advance to the next puzzle in the game. The second, solo session took place one to three days after the first session. In this session, all of the children played The Incredible Machine individually for twenty-five minutes. For this session, puzzles similar to those presented in the first session were created. These puzzles covered similar concepts found in the first session but were distinct and had different solutions. If a child completed a puzzle, the child was responsible for recording the time at which the puzzle was solved and for recording a password that the game provided. In this session, the game automatically advanced to the next puzzle each time a puzzle was solved. A researcher was available to help the children record the password and move to the next puzzle if necessary. 118 Figure 5.5: Task used for the interface training session to familiarize the children with the game and its mouse-driven user interface. Children had to achieve this configuration by moving objects from the parts bin to the playing screen as well as performing manipulations on the objects. After the solo play session, the children were asked to rank on a nine-point scale whether they would prefer to play The Incredible Machine alone or with a friend. To facilitate this procedure for children, a pinwheel was used. The pinwheel consisted of two different coloured cardboard circles, each divided into eight pieces. Both circles were slit and then placed together so that a portion of each circle could be seen (See Figure 5.6). Playing alone was assigned one colour and playing with a friend was assigned another colour. The children were required to turn the pinwheel to indicate whether they preferred to play alone or with a friend. If the children preferred to play The Incredible Machine alone, they would turn the pinwheel so that more of the alone colour was showing. If there was no preference for playing alone or with a friend, the pinwheel could be placed with equal amounts of both colours showing. The number 119 Figure 5.6: Photographs of the pinwheels used by children to rate their preference of playing either alone or with a friend on a nine-point scale. of pie-shaped pieces showing represents the ranking for that category. For example, the pinwheel on the left of Figure 5.6 ranks playing with a friend at six (six of the dark coloured, friend, pieces are showing) while the pinwheel on the right ranks playing with a friend at four (four of the dark coloured, friend, pieces showing). A higher ranking for friend reflects a greater preference for playing with a friend. The pinwheel on the left indicates a preference for playing with a friend with six dark coloured pieces and two dark coloured pieces. The pinwheel on the right indicates no preference for either playing with a friend or playing alone since four pieces of each colour are showing. This technique was used to make ranking easier for the children because of its visual nature [Borys and Perlman, 1985]. For our analysis, the ranking score represented the children's preference for playing with a friend; a child's preference score was the number of dark coloured, friend pieces showing. Experimental Variables The independent variables in this phase of the study were the same as in the first phase: cooperative condition and gender groupings. Again, the cooperative con-120 ditions included the three conditions of one-mouse shared, two-mouse give, and two-mouse take. A control group of children playing alone was also examined. The two gender groupings were pairs of girls and pairs of boys. Each group was analyzed separately and no mixed-gender pairs were studied. Table 5.4 shows the number of dyads in each experimental condition and gender grouping. Solo Play (control group) One-Mouse Shared Two-Mouse Give Two-Mouse Take Girls 18 dyads 18 dyads 18 dyads 18 dyads Boys 18 dyads 17 dyads 18 dyads 18 dyads Table 5.4: Number of dyads per experimental condition and gender grouping The dependent measures in this phase of the study could be grouped into four categories: achievement, learning, mouse sharing, and preference. Achievement in the game was measured by the number of puzzles the children were able to solve in the initial collaborative session. Learning is the label used to represent the possible transference of skills and knowledge from the collaborative session to the subsequent solo session and was quantified in this phase as the number of puzzles the children were able to solve in the followup solo session. We recognize that this measure includes many factors and that it may or may not be assessing the children's actual learning. It is an attempt to differentiate this variable from the pure performance-based assessment of children's achievement in the first session. Mouse sharing included two measurements: mouse control and mouse exchanges. Mouse control is the percentage of time each partner had control of the game. Mouse exchanges is the number of times control of the game was transferred between the two children. Preference was measured by whether the children stated a preference to play The Incredible Machine alone or with a friend. 121 r 5.2.2 Results and Discussion As in the previous phase, the data for girls and boys were analyzed separately. Non-parametric tests were used to analyze the achievement and learning data from this study because the assumptions of normal distribution and homogeneity of variance were violated. Achievement Table 5.5 shows the mean number of puzzles completed in the initial collaborative session for each of the experimental conditions (including the control group, Solo Play), by gender groupings. The Kruskal-Wallis test revealed a significant overall effect of experimental condition1 for girls, x 2 (3 , N = 72) = 11.08, p < .05. No significant differences were found for boys, x 2 (3 , iV = 71) = 2.59, ns. This finding supports the results from the first phase, where the experimental condition signifi-cantly affected girls' performance in their collaborative session but had no significant effect for boys. As in the first phase, girls solved the most puzzles on average in the two-mouse give condition. Posthoc analyses revealed a significant difference between girls in the Solo Play condition and girls in the Two-Mouse Give condition, p < .05. No other pairwise comparisons were significant for girls. Learning Table 5.6 shows the mean number of puzzles completed in the followup solo session for each of the experimental conditions (including the control group, Solo Play), by gender groupings. The Kruskal-Wallis tests revealed no overall effect of condition on learning for girls, x 2 (3 , iV = 72) = 3.53, ns, or for boys, x 2 (3 , iV = 71) = 3.91, ns. Caution must be taken when comparing the results of the three collaborative condi-tions to the control condition. This is because children who played initially with a partner may have taken some time to adjust to playing alone in the second solo ses-1This analysis included the control group, which may be questioned by some. 122 Condition Number of Dyads Mean SD Control (Solo Play) 18 0.61 0.98 Girls* One-Mouse Shared 18 1.67 1.71 Two-Mouse Give 18 1.94 1.36 Two-Mouse Take 18 1.22 1.67 Control (Solo Play) 18 2.44 2.06 Boys One-Mouse Shared 17 3.12 1.83 Two-Mouse Give 18 2.06 1.43 Two-Mouse Take 18 2.50 1.10 *an overall effect of condition was found for girls, p < .05 Table 5.5: Mean number of puzzles solved by children in each of the experimental conditions and gender groupings in the initial collaborative session. sion, whereas children who played both sessions alone could continue in the second session without needing any adjustment time. Condition n Mean SD Control (Solo Play) 36 1.78 2.13 Girls One-Mouse Shared 36 2.14 1.44 Two-Mouse Give 36 2.44 1.42 Two-Mouse Take 36 1.94 2.09 Control (Solo Play) 36 4.50 2.48 Boys One-Mouse Shared 34 4.47 2.46 Two-Mouse Give 36 3.25 1.55 Two-Mouse Take 36 3.92 1.11 Table 5.6: Mean number of puzzles solved by children in each of the experimental conditions and gender groupings in the followup solo session. A correlation was found between the number of puzzles a pair of children solved in the collaborative session and the number of puzzles the children solved in their later, solo session, for girls, r(124) = .46, p < .001, and for boys, r(122) = .53, p < .001. This suggests that there may be knowledge or skill gained during the first session that is beneficial to performance in a subsequent session. 123 Mouse Sharing Mouse sharing included mouse control and mouse exchanges. Mouse control was measured by how long each partner had physical control of the shared mouse in the one-mouse shared condition or logical control of the game cursor in the two-mouse give and take conditions. Mouse exchanges were measured by the number of times control was passed between the two children. In the one-mouse shared condition, an exchange of control was considered to have occurred whenever the single shared mouse was physically passed from one child to the other. In the two-mouse give and take conditions, an exchange of control was considered to have occurred whenever control of the game cursor was switched from one mouse to the other by pressing a right mouse button. Table 5.7 summarizes the results for mouse sharing and mouse exchanges in the three collaborative conditions for both girls and boys. The data for four pairs of children (two pairs of girls from the one-mouse shared condition, one pair of girls from the two-mouse give condition, and one pair of boys from the one-mouse shared condition) are missing because of problems experienced while collecting the mouse sharing data. The mouse control columns present the average percentage of time each partner had control of the mouse. The first number is the average percentage of time the mouse was controlled by the children within each pair who had control of the mouse less than their partners. The second number is the average percentage of time the mouse was controlled by the children within each pair who had control of the mouse more than their partners. The exchanges column (the third number) in Table 5.7 is the average number of times control was passed between the two children. Mouse Control For girls, a one-way analysis of variance revealed no significant effect of turn-taking protocol on the amount of time a partner controlled the mouse, .F(2,48) = 0.33, ns. 124 Mouse Control Condition n Less Time More Time Exchanges One-Mouse Shared 16 30% 70% 13 Girls Two-Mouse Give 17 33% 67% 25 Two-Mouse Take 18 30% 70% 29 One-Mouse Shared 16 24% 76% 18 Boys Two-Mouse Give 18 33% 67% 22 Two-Mouse Take 18 38% 62% 46 Table 5.7: Mouse control represents the average percentage of time each partner had control of the mouse. The first value represents the children within each pair who had control of the mouse for the shorter total time. The second value represents the children within each pair who had control of the mouse for the longer total time. Exchanges represents the number of times control was passed between the two children in a pair. For boys, a one-way analysis of variance revealed a significant effect of experimental condition on the amount of time a partner controlled the mouse, F(2,49) = 5.69, p < .05, with a medium to large effect size of .13 and a power of 76%. The post-hoc analysis revealed that boys in the two-mouse take condition had a significantly more equal distribution of mouse-control time than did boys in the one-mouse shared condition, p < .05. Table 5.8 shows the average number of puzzles solved in the solo phase by children who had control of the mouse less than their partners and by children who had control of the mouse more than their partners. Figure 5.7 represents this data graphically, along with the data for the pairs when they played in the initial collaborative session. A significant correlation was found for boys between the amount of time a partner had control of the mouse in the collaborative session and the number of puzzles he was able to solve in the subsequent solo session, r(102) = .30, p < .01. A one-way analysis of variance of having control of the mouse for a longer time versus a shorter time was performed. This revealed that boys who had control of the mouse 125 Mouse Control Condition n Less Time More Time One-Mouse Shared 16 1.94 2.31 Girls Two-Mouse Give" 16 2.13 2.88 Two-Mouse Take 18 1.33 2.56 One-Mouse Shared 16 3.25 5.63 Boys Two-Mouse Give 18 2.56 3.94 Two-Mouse Take 18 3.78 4.06 "the data for one two-mouse give pair (girls) is not included because the distribution of mouse control was exactly 50% for each girl Table 5.8: Mean number of puzzles solved in the second, solo phase by children who had control of the mouse less than their partners and by children who had control of the mouse more than their partners. for a longer period of time in the collaborative session solved significantly more puzzles than did their partners who had control of the mouse for a shorter period of time, F(\,102) = 8.44, p < .01, with an effect size of .08 and a power of 82%. Further analyses revealed that the differences between partners were significant for boys in the one-mouse shared condition, F(l, 30) = 5.36, p < .05, and for boys in the two-mouse give condition, F(l,34) = 4.49, p < .05, but not for boys in the two-mouse take condition, F(l,34) = .22, ns. The analysis for girls showed similar trends in that those who had control of the mouse longer than their partners solved more puzzles on average than did girls who had control of the mouse less than their partners, but this difference was not significant, F(l, 98) = 3.80, ns, with an effect size of .04 and a power of 49%. Mouse Exchanges One-way analysis of variance tests revealed significant effects of experimental con-dition on the number of times girls and boys exchanged control of the mouse, F(2,48) = 6.51, p < .01 and F{2,49) = 6.34, p < .01, respectively, with a large effect size of .17 and a power of 88%. The post-hoc analysis revealed that girls exchanged control of the mouse significantly more times in both the two-mouse give condition 126 S h a r e d G i v e T a k e E P h a s e 1: w i t h p a r t n e r • P h a s e 2: p a r t n e r s w h o Q P h a s e 2: p a r t n e r s w h o S h a r e d G i v e T a k e c o n t r o l o f t h e m o u s e l e s s c o n t r o l o f t h e m o u s e m o r e Figure 5.7: Mean number of puzzles solved by the children who had control of the mouse less than their partners and by the children who had control of the mouse more than their partners. and the two-mouse take condition than did girls in the one-mouse shared condition, p < .05. The post-hoc analysis for boys revealed that boys in the two-mouse take condition exchanged control of the mouse significantly more times than did boys in either the one-mouse shared condition or the two-mouse give condition, p < .05. A correlation was found for boys between the number of exchanges in a collaborative session and the percentage of time each partner had control of the mouse in the one-mouse shared condition, r(14) = .68, p < .01, and in the two-mouse give condition, r(16) = .56, p < .05. In these two conditions, a higher number of exchanges correlated with a more equal distribution of time that each partner had control of the mouse. No significant correlations between mouse control time and exchanges were found for boys in the two-mouse take condition or for girls in any of the conditions. Both girls and boys exhibited different sharing patterns using the various turn-taking protocols as shown by the number of times the children exchanged control. For both genders, the least number of exchanges occurred in the one-mouse 127 shared condition while the highest number of exchanges occurred in the two-mouse take condition. These results may be attributed to the ease of obtaining control of the game cursor. In the one-mouse shared condition, gaining control is typically more difficult because a child must physically obtain control of the shared mouse. In contrast, obtaining control in the two-mouse take condition is relatively easy because a child only needs to press a button on the mouse to gain control. For girls in the two-mouse give condition, the number of exchanges was about the same as the number of exchanges in the two-mouse take condition. This suggests that girls in the two-mouse give condition behaved similarly to girls in the two-mouse take condition in terms of turn-taking, in which case the girls may have had a relatively easy time obtaining control of the game. In contrast, the average number of exchanges for boys in the two-mouse give condition was close to the average for the one-mouse shared condition. This suggests that boys in the two-mouse give condition behaved similarly to boys in the one-mouse shared condition in terms of turn-taking, in which case the boys may have had a difficult time obtaining control of the game. Playing Preference Children's preference for playing The Incredible Machine either alone or with a friend was gathered using the pinwheels described in the previous Section 5.2.1. Only children who played in one of the collaborative groupings (not the control group) were included in this analysis because children in the control group never received an opportunity to play with a partner and therefore would have difficulty rating whether or not they would prefer this mode of playing. The results for girls' and boys' preference by condition of playing either alone or with a friend are shown in Table 5.9. A ranking represents the pinwheel score for a preference to play with a friend. For example, if a child adjusted the pinwheel to show three pieces of the friend colour and five pieces of the alone colour, the 128 preference ranking would be three. A one-way analysis of variance for girls revealed no significant effect of condition on preference rankings, F(2,105) = .68, ns. For boys, the one-way analysis of variance revealed a significant effect of condition on the boys' preference rankings, F(2,101) = 3.63, p < .05. Post-hoc analysis revealed that boys in the two-mouse give condition gave higher preference rankings (mean 5.56) than boys in the one-mouse shared condition (mean 4.03), p < .05. Condition n M a SD One-Mouse Shared 36 4.75 1.99 Girls Two-Mouse Give 36 5.08 2.27 Two-Mouse Take 36 5.36 2.38 One-Mouse Shared 32 4.03 2.62 Boys Two-Mouse Give 36 5.56 1.27 Two-Mouse Take 36 4.39 1.51 aThe rating scale ranges from strongly prefer to play alone (0) to strongly prefer to play with a friend (8) using the pinwheel method to select rankings. Table 5.9: Children's Average Ranking for Playing Preference by Condition Whether girls and boys preferred to play alone or with a friend was analyzed using a t-test for a single mean. The mean was tested against an expected value of four which represents the neutral point of equal preference for playing alone or with a friend. Girls in all three experimental conditions significantly preferred playing with a friend as opposed to playing alone, one-mouse shared: £(35) = 2.26, p < .05; two-mouse give: £(35) = 2.86, p < .01; and two-mouse take: £(35) = 3.43, p < .01. The analysis for boys playing in the two-mouse give condition found that boys significantly preferred playing with a friend as opposed to alone, £(35) = 4.11, p < .001 while boys in the one-mouse shared and two-mouse take conditions showed no preference for either playing style, £(32) = .07, ns and £(35) = .93, ns, respectively. The results for girls' and boys' preference of playing either alone or with a friend, across all experimental conditions are shown in Figure 5.8. 129 prefer to play alone prefer to play with a friend 30 i 25 • n J 0 1 3 4 5 6 7 8 Q G i r l s ' Preference • Boys 1 Preference Figure 5.8: Girls' and boys' ranking of whether they preferred to play with a friend or alone. The rankings ranged from 0 (strongly prefer to play alone) to 8 (strongly prefer to play with a friend). Children's playing preferences can be grouped into three nominal categories: prefer playing alone, no preference, and prefer playing with a friend. The results of this grouping are shown in Table 5.10. A ranking of zero to three would be placed in the prefer to play alone group, a ranking of four would be placed in the no preference group, and a ranking of five to eight would be placed in the prefer to play with a friend group. A chi-square analysis of preferences for playing alone or with a friend revealed that significantly more girls and boys preferred to play with a friend than preferred to play alone, x2(l>AT = 82) = 14.1, p < .001, and X 2 ( l , AT = 80) = 5.0, p < .05, respectively. A l l children who played in a collaborative group played first with their part-ner and then alone in the followup solo session. Because the order of session was not counterbalanced, the results may be biased towards a preference for playing alone 130 Play Alone Play with a Friend No Preference n % n % n % Girls 24 22% 58 Boys 30 29% 50 54% 48% 26 24 24% 23% Table 5.10: Percentage of Children who Preferred Each Mouse Interaction Style because this was the condition that the children had just completed playing. In addition, during the first session of play, the children were unfamiliar with the game and therefore had to figure out how to play the game. As a result, the first session may have been frustrating for the children and many were not as successful as they were in the second session. This could also cause the rankings to be biased towards playing alone. Previous Experience The children's previous experience in playing electronic games, or the presence of a computer at home, as reported in the background questionnaires, did not yield any significant correlations with children's achievement in either session of play. The percentage of time a child had control of the mouse was also not correlated to the amount of time the children reported they played video or computer games or used computers. Any other effect of previous experience was minimized by randomly assigning children to conditions. Follow-up analysis, using a one-way analysis of variance, did not find any significant differences in the children's previous experi-ences for each of the conditions. 5.3 Conclusion The results of this study show that seemingly similar ways of sharing limited re-sources (in our case turn-taking for mouse-driven control of a game cursor) in a collaborative environment can have a significant influence on the participants in 131 terms of achievement, learning and behaviour. The addition of a second mouse and the turn-taking protocol used affected girls' achievement as they played The Incredible Machine in their collaborative ses-sion. Girls playing in the two-mouse give condition solved significantly more puzzles than did girls in the one-mouse shared condition. Analysis of girls' results in the followup solo session revealed no significant effect of experimental condition on the number of puzzles the girls could solve on their own although a similar trend ex-isted where girls in the two-mouse give condition solved more puzzles on average than did girls in the other two conditions. Control of the mouse also had no effect on the girls' learning while playing in the initial collaborative session. No significant correlation was found between the amount of time a girl had control of the mouse and the number of puzzles she could solve in the followup solo session. For girls, no significant effect of experimental condition was found for the distribution of mouse time between the partners. This suggests that regardless of the turn-taking strategy employed, the girls exhibited similar mouse-sharing pat-terns. The qualitative observations support this results. Girls in all experimental conditions appeared reluctant to "take" the mouse (or control of the game cursor in the two-mouse condition) from their partner. Even when playing in the two-mouse take condition, many girls would ask their partners if it were OK to take control of the game before initiating the switch. As a result, the girls' methods for transferring control in all three experimental conditions were similar. The turn-taking strategy may not have affected how the girls shared control of the mouse but it may have affected the girls' interactions with their partners. Qualitative observation revealed that girls playing in the two-mouse give condition sometimes passed control to their partners even when the partners had not requested it. This may have had the effect of encouraging the other child to share her ideas, which she otherwise might have kept to herself. This behaviour was not as prominent in the other two experimental conditions. This sharing of ideas may have enhanced 1 3 2 the children's collaboration and could be one possible reason why girls in both phases of this study were able to solve the most puzzles in the two-mouse give condition. For boys, there was a positive correlation between the amount of time a boy had control of the mouse in the collaborative session and how many puzzles he was able to solve in the subsequent solo session. This indicates that control of the mouse and manipulation of the environment may be important for boys' learning. The results for the one-mouse shared and two-mouse give conditions show that the boys who had control of the mouse for a longer period of time during the collaborative session solved more puzzles in their solo session than did their partners who had control of the mouse for a shorter period of time. In contrast, the results for the two-mouse take condition showed that the performance of both partners in the solo session was very comparable. Because the results for boys playing in the two-mouse take condition were different than in the other two conditions, it strengthens our argument that the boys may have been learning something from their collaborative session. If the results had been the same for all three conditions, we may have attributed the correlation between mouse time and the number of puzzles solved in the followup solo session to the fact that the boys who knew how to solve the puzzles took control of the mouse. But if this were the case, then the number of puzzles the boys could solve in the followup solo session would be a factor of their innate ability as opposed to a transference of skills and knowledge from the collaborative session and all three experimental conditions would have produced the same result instead of the different behaviour we observed for the two-mouse take conditions. For boys, one way to help ensure a more equal distribution of mouse control and manipulation of the environment is by providing easy access to control of the game, such as in the two-mouse take condition. The ease of access caused more exchanges of control, which for boys correlated to a more equal distribution of mouse time. By providing boys with a mechanism to obtain a more equal distribution of 133 mouse time, as in the two-mouse take condition, both partners may better benefit from the experience. This type of setup can help to prevent situations where one boy dominates control of the game and the other partner is left out, which commonly happens in the one-mouse shared condition. We can relate these findings back to Cohen's [1994] focus on the relation-ship between interaction and productivity in a cooperative learning environment: "[If] interaction is critical for achievement gains for group tasks with ill-structured solutions, then factors that affect the amount and richness of interaction will af-fect productivity for such tasks" [p.20]. Providing alternative ways of turn-taking or more equal access to control of the game may have affected the "amount and richness" of the children's interactions in our study. One important observation that arose from this study was that boys and girls interacted quite differently using the various turn-taking protocols. Because of this, we must realize that the turn-taking protocol that best suits one gender may not be the best for all children; gender does appear to make a difference. It could be argued that the two-mouse take protocol was most suited for boys because it allowed for the most equitable access to control of the game cursor and thus was of mutual benefit for both partners given the way that boys play the game. But the girls performed best using the two-mouse give protocol. This generalization does not hold true for all girls or for all boys, but it does indicate that we should be careful when designing collaborative environments, especially if they are to be used in educational settings. If we want to ensure that the environment is equally appropriate for both girls and boys, we may have to provide more than one turn-taking protocol to support effective collaboration. As was noticed in the previous study, boys again solved more puzzles on average than did girls while playing The Incredible Machine in both phases of this study. This brings into question whether the nature of the game, or perhaps the task of solving puzzles itself is more suited to boys than girls. 134 A limitation of the second phase of this study was that the children only took part in the study for two forty-minute class periods. The behaviour reported here may change when children work in collaborative environments for a longer period of time. As children become familiar with the turn-taking protocols and with the software application, we may see different behaviour patterns arise. In addition, playing in a collaborative session for only twenty-five minutes is probably not long enough to show any strong effects on learning. Our study focused primarily on quantitative analyses of mouse control time and the number of exchanges that took place. As Cole's [1995] study suggests, it is also important to investigate children's behaviour qualitatively to gain further insight into children's collaborative behaviour. We used the IBM-compatible version of The Incredible Machine in both phases of this study because of the difficulties uncovered during the research in Chapter 4 with the drag-and-drop interaction style used in the Macintosh version of the game. The next chapter investigates the differences between the interaction styles drag-and-drop and point-and-click that are used in the two versions of The Incredible Machine. 135 Chapter 6 Drag-and-Drop vs. Point-and-Click Computer Brands in Public Schools 1995-1996 Macintosh Data reported by Quality Education Data, Educational Technology Trends, 1996. "Entirely appropriate," says Karla, "because Windows is more male, and Mac is more female." I felt defensive. "How so?" "Well, Windows is non-intuitive . . . counterintuitive, sometimes. But 136 it's so MALE to just go buy a Windows PC system and waste a bunch of time learning bogus commands and reading a thousand dialog boxes every time you want to change a point size or whatever . . . MEN are just used to sitting there, taking orders, executing needless commands, and feeling like they got such a good deal because they saved $200. WOMEN crave efficiency, elegance . . . the Mac lets them move within their digital universe exactly as they'd like, without cluttering up their human memory banks. I think the reason why so many women used to feel like they didn't understand computers was because PCs are so brain-dead . . . the Macintosh is responsible for upping not only the earning potential of women but also the feeling of mastering technology, which they get told is impossible for them. I was always told that." - Douglas Coupland microserfs, 1995 page 120 Many children today are exposed to computers from a very early age. The types of interaction techniques used in children's software are quite varied. In general, chil-dren appear to adapt to whatever interaction style is present, but this is not an ideal situation because some children may be intimidated using an interaction style with which they are not comfortable. In order to design effective interaction techniques, "we need to use a deeper understanding of task, device, and the interrelationship between task and device from the perspective of the user" [Jacob, Sibert, McFarlane, and Mullen, 1994]. Paying attention to the perspective of the user means that we cannot assume that children are just like adults. There has been significant research on interaction styles for adults, but until recently very little research has focused on children's interactions with computers. It is important to investigate these same issues with children. Some researchers have noted that children have difficulty using some com-mon mouse interaction techniques such as marquee selection [Berkovitz, 1994] and 137 maintaining pressure on the mouse button [Strommen, 1994]. Numerous other re-searchers [Hall and Cooper, 1991; Inkpen, 1997; Lockheed, 1985; Upitis and Koch, 1996; Wilder et al., 1985] have observed that gender differences often exist with respect to children's interactions with computers. In our previous research we observed that some children had difficulty per-forming a drag-and-drop action while playing in an interactive learning environment. The study described in Chapter 4 was conducted in two phases and each phase used a different computer platform. The first phase, in the school environment, used IBM-compatible computers. The second phase, in the Science World environment, needed four computers to run four sessions in parallel. Although four IBM-compatible com-puters were not available to us, it was possible to obtain several Macintosh LCIII computers. The Incredible Machine, the software used for that study, was available on both platforms, so we decided to use the Macintosh LCIII computers at Science World because this would significantly decrease the time required to complete the study. On the surface, the IBM-compatible and the Macintosh versions of The In-credible Machine looked identical to us and we did not realize until after the Science World phase of the study was underway that the two versions of the game utilized somewhat different interaction styles. The IBM-compatible version, used in the school environment, utilized a point-and-click mouse interaction style, whereas the Macintosh version of the game used in the Science World environment utilized a drag-and-drop mouse interaction style. During the study we observed that many children were having difficulty using the drag-and-drop interaction style on the Mac-intosh version. In addition, the number of puzzles that girls were able to solve while playing the game in the Science World phase of the study was far fewer than what children achieved during the preliminary phase of the study in the school environ-ment. While the differences in mouse interaction style certainly were not the only 138 factors that might have contributed to these differences, we recognized that it was important to examine this issue further to determine what effect interaction style had on the children's achievement and motivation. Adults' use of the two mouse interaction styles of drag-and-drop and point-and-click has been explored by other researchers who examined the differences in speed and accuracy between the two methods. One study, reported in 1991, con-cluded that a dragging task was slower than a pointing task and that more errors were committed during a dragging task than during a pointing task [MacKenzie, Sellen, and Buxton, 1991]. We decided to re-examine the issue of mouse interaction style for three reasons: (1) direct manipulation interfaces are more prevalent today than was the case in 1991 and therefore user experiences may have changed; (2) the previous research dealt only with adults and it cannot be assumed that the results will be the same for children; and (3) gender was not examined in the previous research. While many other input devices such as pen-based input are becoming common in the workplace, mouse-based input is still the dominant interaction de-vice used by children at both school and home. In order for this research to have an immediate impact on developers of educational software, mouse-based input was the method of interaction explored in this study. A study was designed to determine whether the previous results on adult's use of mouse interaction styles hold for children and whether or not gender differences affect these results. In addition, the interaction styles of drag-and-drop and point-and-click were explored in a complex game environment to help understand how the choice of interaction style can affect children's achievement and motivation within a software environment. Research Setting The research for the study described in this chapter took place at Science World, over two summers, involving several hundred children. 139 The research area for both phases of the study included a computer set up on a table in an open area, which was chosen for its visibility to visitors exploring other exhibits. The research area was set back slightly from the main traffic and sectioned off with small risers to provide some privacy for children taking part in the study. Children were positioned with their backs to the passers-by, which helped to reduce distractions during the study. 6.1 Phase 1: Mouse Interaction Style in The Incredible Machine The first phase in this study examined children using two versions of The Incredible Machine that utilized either a point-and-click interaction style or a drag-and-drop interaction style. The goal of this phase was to determine the impact these interac-tion styles have on children's achievement and motivation. 6.1.1 Method The experiment was modelled after our initial studies, employing the puzzle-solving game, The Incredible Machine. Subjects The subjects in the first phase of the study were 189 girls between the ages of nine and thirteen. The study took place at Science World during the month of August, 1995. A l l subjects were visitors to Science World who volunteered to take part in the study, and who had never played the computer game The Incredible Machine. Previous research has found that gender differences often exist with respect to children's interactions with computers and therefore it is important that our investigation be sensitive to the possibility of gender differences [Hall and Cooper, 1991; Inkpen, 1997]. Time constraints did not permit studying both girls and boys. 140 Because observations from our previous research indicated that girls in particular performed differently using a drag-and-drop interaction style as opposed to a point-and-click interaction style, only girls were used in this phase of the study. The age range of nine to thirteen was chosen because previous observations had shown that although children of all ages play and enjoy The Incredible Machine, in general, children under the age of nine have difficulty learning and playing the game without assistance. In addition, because the motivation for this study arose from analyzing results in earlier studies, the age range from the earlier studies was maintained. One hundred and fifty-five of the girls were observed using the Macintosh version of The Incredible Machine game, while the other 34 girls were observed using the IBM-compatible version of The Incredible Machine. The reason for the difference in the sample sizes was that the data for the girls playing the Macintosh version of the game were also being used for a separate study (see Chapter 4, Section 4.2) that required a larger sample size. For purposes of comparing means between the two conditions, a subset of 32 girls of the 155 girls playing on the Macintosh version was randomly selected (32 girls were selected because this was the number of girls who played using the IBM-compatible version for the full thirty minutes). Hardware and Software The two platforms used were a 386 IBM-compatible computer running Windows 3.1 with a three-button mouse and a Macintosh LCIII computer, with a one-button mouse. The game software used in this phase of the study was The Incredible Machine. Because operation of the game only required input from the mouse, the keyboard was not made available to the girls. The mouse and mouse-pad were initially placed in front of the monitor. The girls were allowed to move the mouse and mouse-pad to wherever they felt comfortable using them. 141 Procedure The sessions began with welcoming remarks from the researcher, followed by a brief verbal introduction to the experimental study and to the game The Incredible Machine, a short interface training session, and then thirty minutes of time to play The Incredible Machine. The girls were randomly assigned to a particular interaction style if both platforms were free, otherwise they were assigned to use whichever platform was available. In order to reduce user interface problems during the session, the interface training session was designed to teach the girls how to manipulate objects in the game using one of the interaction styles. The interface training session demonstrated how to begin playing a puzzle, how to move objects from the parts bin onto the playing screen, how to connect objects together, how to flip objects, how to re-size objects, and how to run the configured machine by clicking on the run machine icon. Following the interface training session, the girls were asked to complete as many puzzles as they could in the thirty minutes provided. Upon completion of a puzzle, the girls were required to record the time at which they finished and the password given by the game. Recording the password ensured that the girls did not move on to the next puzzle until they had completed the current puzzle. Upon completion of a puzzle, the game automatically advanced to the next puzzle. The girls were told that they could stop playing at any time they wished. Experimental Variables One independent variable was manipulated in this phase of the study: mouse inter-action style. The girls played using a version of The Incredible Machine that uti-lized either a point-and-click interaction style or a drag-and-drop interaction style. Two dependent variables were measured: achievement in the game and motivation. Achievement in the game was measured by the number of puzzles the girls solved in the thirty-minute playing period. Motivation was measured by whether or not the 142 girls played for the full thirty minutes. 6.1.2 Resul ts The mean number of puzzles solved by girls using both the IBM-compatible version and the Macintosh version are shown in Table 6.1. Only girls who stayed and played for the full thirty minutes were included in this analysis (32 girls using the IBM-compatible version, 123 girls using the Macintosh version). Of the girls who played on the Macintosh platform for the full thirty minutes, the data for 32 randomly selected girls were used in this analysis to correspond to the sample size of girls using the IBM-compatible version. The mean number of puzzles solved by children in the randomly selected sample was not significantly different from the mean of the population it was chosen from, F(l, 153) = 4.94, ns. Condition" n M SD P Point-and-Click Interface 32 1.66 1.56 Drag-and-Drop Interface 32b 0.84 1.17 p< .05 "only girls who stayed for the full thirty-minute session were counted in this analysis 6 a subsample of 32 girls was randomly selected from the 123 girls who played for the full thirty minutes using the Macintosh platform Table 6.1: Mean Number of Puzzles Solved Using Each Mouse Interaction Style An independent-sample t-test was used to examine the relationship between the interface used and the number of puzzles solved. Girls using the point-and-click interaction style solved significantly more puzzles than did girls using the drag-and-drop interaction style, £(62) = 2.36, p < .05. The percentage of girls who played for the full thirty-minute session but were not able to solve any puzzles was 24% for the point-and-click interaction style and 50% for the drag-and-drop interaction style. The analogous results for all girls who took part in the study, including those who left early, were 26% for the point-and-143 click interaction style and 60% for drag-and-drop interaction style. Examining all children who took part in the study, of the 34 girls using the point-and-click interaction style in the IBM-compatible version, only two girls left before the thirty minute time period was up, one of whom left after solving three puzzles, while the other left before solving any puzzles at all. Of the 155 girls using the drag-and-drop interaction style in the Macintosh version, 32 girls left before the thirty minute time period was up, one of whom left after solving one puzzle while the other 31 girls left before solving any puzzles at all. The difference between the number of girls who left early using the point-and-click interaction style (6%) versus those using the drag-and-drop interaction style (21%) was statistically significant, X 2 ( l , n = 189) =4.94, p< .05. 6.1.3 Discussion Girls using the point-and-click interaction style in the IBM-compatible version were more successful and more motivated to play the game than were girls using the drag-and-drop interaction style in the Macintosh version of the game. Success was measured by the number of puzzles the girls were able to solve. On average, girls using the point-and-click interaction style in the IBM-compatible version solved significantly more puzzles than did girls using the drag-and-drop interaction style in the Macintosh version. Motivation to play the game was measured by the percentage of girls who chose to play the game for the full thirty minute period. In this study, all but two girls (6%) using the point-and-click interaction style in the IBM-compatible version of the game stayed and played for the full thirty minute period, while 32 girls (21%) using the drag-and-drop interaction style in the Macintosh version left early. Whether or not the girls chose to stay and play for the full thirty minute period could have been affected by how successful they were in the game. Being unable to solve any puzzles can be frustrating and may have caused some of the girls to give up. Al l of the girls who left early, except for two (one in each condition), 144 were unable to solve any puzzles before they left. Sixty percent of the girls using the Macintosh version of the game were unable to solve any puzzles, compared to 26% of the girls using the IBM-compatible version. This demonstrates that the Macintosh version of the game may have had a more difficult and frustrating interface. Figure 6.1 shows the number of girls who left during each five-minute interval during the thirty-minute session. A chi-square analysis of girls using the drag-and-drop interaction style found no significant difference between the proportion of girls who left during each of the five-minute intervals, x 2 ( l , N = 32) = 6.625, ns. No analysis was performed on girls using the point-and-click interaction style because only two girls left before the thirty-minute time period was up. 0-4min 5-9min 10-14min 15-19min 20-24min 25-29min Session Time n P o i n t - a n d - c l i c k " D r a g - a n d - d r o p Figure 6.1: Time distribution graph illustrating the intervals at which girls left the session early. Notice that only two girls using the point-and-click interaction style left the session early. The results from this phase of the study demonstrate that a point-and-click interaction style, used in an interactive learning environment, can be more effective for girls in terms of performance and motivation than a drag-and-drop interaction 145 style. Because this phase examined girls using a rich, complex environment, im-plementation details and other issues related to playing the game itself may have interacted with the results. One implementation issue that arises when using the drag-and-drop interac-tion style is the fact that it is a two-step movement: the mouse button is pressed down to perform the first action and the mouse button is then released to perform the second action. Because manipulating some objects within The Incredible Ma-chine requires three steps, the designers' choice of how to implement a three-step motion using a two-step interaction style could impact how users interact with the system. The Macintosh version of The Incredible Machine handles three-step move-ments by first performing a click (press the mouse button down and then release) to pick up the object, and then performing a drag-and-drop between the two endpoints. Because of this, regular objects and connector objects in this version are moved from the parts bin in two different ways, which creates an inconsistency in the interface. Implementation details such as this may have contributed to the results so the sec-ond phase of the study was designed to examine children using the point-and-click and drag-and-drop interaction styles in isolation, removing other factors that may have been present in the interactive learning environment used in the first phase. 6.2 Phase 2: Mouse Interaction Style in a Simplified Setting The second experiment in the study was conducted in a much simplified software environment that presented a similar pair of mouse interaction styles to those used in the first experiment, but without the possibly confounding factors present in the more complex interactive learning environment provided by The Incredible Machine. Because the simplified environment did not have to provide all of the richness of a commercial computer game, the experimenters were able to configure the run-146 time software to include additional data collection that was not available using the commercial software. 6.2.1 Method The experiment was a more abstract version of the puzzle-solving game environment used in the first phase, isolating the movements of drag-and-drop and point-and-click. Subjects The subjects in the second phase of the study were 67 children (34 girls and 33 boys) between the ages of nine and thirteen. The study took place at Science World during the month of August, 1996. A l l subjects were visitors to Science World who volunteered to take part in the study. The age range of nine to thirteen was used to correspond with the age range used in the first phase of the study. The children who participated in the second phase of the study were not the same children who participated in the first phase of the study. Background data on the children participating in the study was retrieved through questionnaires1. A l l of the children who took part in this phase were familiar with computer and video games: 75% of the children had a computer at home and 73% had some type of video game player at home. Only three of the children had neither a computer nor a video game player at home. The children were asked to indicate how often they played video or computer games. The responses are summarized in Figure 6.2. Hardware and Software A Silicon Graphics workstation with a three-button mouse was used to conduct the experiment. Because operation of the game only required input from the mouse, 'Analysis of the children's background data revealed no significant correlations between com-puter or video game playing experience and the children's performance, motivation or preference. 147 G i r l s Boys (n = 34) (n = 33) • P l a y c o m p u t e r o r v i d e o g a m e s a f e w t i m e s a y e a r • P l a y c o m p u t e r o r v i d e o g a m e s a f e w t i m e s a m o n t h • P l a y c o m p u t e r o r v i d e o g a m e s a f e w t i m e s a w e e k • P l a y c o m p u t e r o r v i d e o g a m e s a l m o s t e v e r y d a y o P l a y c o m p u t e r o r v i d e o g a m e s e v e r y s i n g l e d a y Figure 6.2: How often girls and boys reported that they played computer or video games the keyboard was not presented to the children. Instead the mouse and mouse-pad were placed in front of the monitor and the children were able to place the mouse and mouse-pad wherever they felt comfortable using them. Two versions of a special-purpose 2-D graphics program written in OpenGL were developed to support the two different mouse interaction styles of point-and-click and drag-and-drop for this phase of the study. The program displayed two squares on the screen: a green, solid, source box and a red, outlined, target box (see Figure 6.3). The children were required to use a mouse to pick up the source box, move it over to the target box, and drop it inside the target box. The two boxes were displayed either 400 pixels or 800 pixels apart, and each box could be one of two sizes, 32 x 32 pixels or 64 x 64 pixels. This produced eight different possibilities of distance x source x target. There were approximately 35 pixels/cm on the monitor that was used in this phase of the study. Therefore the width and 148 height of the larger box was approximately 1.8cm and the width and height of the smaller box was approximately 1.1cm. Trial #1 Figure 6.3: The initial screen configuration produced by the software used in the second phase of the study. Children were required to move the solid (green) source box on the right to the outlined (red) target box on the left. The manipulation of distance between boxes as well as source and target size was used to confirm that the movement could be accurately modelled by Fitts's Law. The two sizes were chosen to approximate small and large objects found in The Incredible Machine. The two distances were chosen to approximate typical short distance and long distance movements performed while playing The Incredible Machine. Each possible combination of distance x source x target comprised a trial. Two instances of each combination, for a total of sixteen trials, comprised a block. The order of appearance of each distance x source x target combination was random within each block. When a successful drop of the source box occurred, the software produced an audible beep and a new set of source and target boxes appeared on the screen for the next trial. After each block of sixteen trials, the screen would turn black until a researcher pressed a key to progress on to the next block. The current 149 trial number was displayed on the top of the screen throughout a session. One version of the software used a point-and-click mouse interaction style while the other used a drag-and-drop mouse interaction style. The point-and-click interaction style required a child to click on the (green) source box to pick up the object (a click refers to pressing the mouse button down and then releasing it), then move the source box over to the (red) target box, and finally click the mouse button again to drop the source box inside the target box. The drag-and-drop interaction style required the child to press the mouse button down on the source box to pick it up (the mouse button was not released at this point), then move the source box over to the target box, and finally release the mouse button to drop the source box inside the target box. When the source box was successfully picked up, visual feedback was provided in both versions of the software by having the solid green source box turn into an outlined green box and a small iconified picture of the source box was attached to the cursor. During the movement of the source box, the iconified picture of the source box remained attached to the cursor until it was dropped, to provide visual feedback that the source box was picked-up and was being moved (see Figure 6.4). For both the point-and-click interaction style and the drag-and-drop inter-action style, a successful pick-up occurred when the tip of the cursor was inside the source box and the mouse button was pressed down. Whether or not the mouse button was released within the source box for the point-and-click interaction style was irrelevant. A successful drop of the source box occurred in the point-and-click interaction style when the tip of the cursor was in the target box and the mouse button was pressed down. Again, whether or not the mouse button was released inside the target box was irrelevant. For the drag-and-drop interaction style, a suc-cessful drop occurred when the tip of the cursor was inside the target box and the mouse button was released. When a child completed a trial (successfully picked up the source box and successfully dropped it inside the target box) an audible beep 150 Trial #1 Figure 6.4: Visual feedback during either a drag-and-drop or a point-and-click move-ment. The green filled source box turned into a green outlined box when it was picked up and a small iconified picture of the green source box was attached to the cursor as the source box was moved. was sounded to signify success and the system automatically advanced to the next trial. Two types of errors were possible during the task: pick-up errors and drop errors. A pick-up error occurred when the mouse button was clicked outside of the source box during a pick-up attempt. A drop error occurred in the point-and-click interaction style when the mouse button was clicked outside of the target box during a drop attempt. A drop error occurred in the drag-and-drop interaction style when the mouse button was released outside of the target box. If a pick-up error occurred for either the point-and-click interaction style or the drag-and-drop interaction style, neither the source box nor the cursor changed its appearance (i.e., the source box did not become outlined and an iconified picture of the source box was not attached to the cursor, which would have happened had the source box been successfully picked up). Following a pick-up error, the only possible action in either interaction style was to attempt to pick up the source box again. 151 If a drop error occurred in the point-and-click interaction style there were three possible design choices for how the system could respond: 1. the source box could return to its original position and the child would have to go back and pick up the source box again; 2. the source box could be placed in the incorrect place on the screen, requiring the child to pick up the source box from where it had been dropped and then move it to the target box; 3. the source box could remain picked-up, allowing the child another attempt at correctly dropping the source box into the target box. Using the drag-and-drop interaction style there were only two possible design choices for the system response: 1. the source box could return to its original position and the child would have to go back and pick up the source box again; 2. the source box could be placed in the incorrect place on the screen, requiring the child to pick up the source box again and then move it to the target box. The third option that existed for point-and-click did not make sense for drag-and-drop because the erroneous drop action had required the mouse button to be re-leased. The motivation for this phase of the study was to further explore the dif-ferences found in the first phase. Error handling choices were supposed to mimic what happened in the two versions of The Incredible Machine that were used in the first phase. Thus, the point-and-click version of the software was designed to follow the third option in order to mimic the IBM-compatible version of the game. In the event of a drop error, the source box would remain picked-up, attached to the cursor, allowing the child another attempt at correctly dropping the box inside the 152 target box. The state-transition diagram for movement using the point-and-click interaction style is shown in Figure 6.5(b) on the right. Similarly, the drag-and-drop version of the software was designed to follow the first option in order to mimic the Macintosh version of the game. In the event of a drop error, the source box would return to its original position and the child would have to go back and pick up the source box again. The state-transition diagram for movement using the drag-and-drop interaction style is shown in Figure 6.5(a) on the left. The diagrams show the necessary steps for completing the placement by moving from top to bottom in a diagram. The states are represented by the long rectangles. A state comprises a position on the screen and whether or not the object is picked up. The position on the screen could either be outside both the source and target boxes, inside the source box, or inside the target box. Transitions be-tween states are represented in the diagram by arrows and correspond to mouse movements, mouse clicks (in the point-and-click version), and mouse button-down or mouse button-up events (in the drag-and-drop version). Downwards arrows rep-resent forward progress towards completion of the placement while upwards arrows represent errors or backwards movement through the steps. Errors are represented by double-lined arrows. Analyzing the state transition diagram shows that while there are three pos-sible errors for each interaction style, the impact of these errors differs depending on the interaction style. When an error is committed using the point-and-click inter-action style the player remains in the current state, ready for another try. In two of the three errors for the drag-and-drop interaction style, when an error is committed the player is forced to move backwards to previous states. For one of the errors this means going back one state, but for the error originating in the fourth state, the player must go back three states all the way to the initial state (indicated by the long backwards double arrow). The player must repeat states one to four as a result 153 (a) Drag-and-Drop (b) Point-and-Click Outside of Source Source not nicked ut> U Inside of Source Source not picked up Inside of Source Source picked up Outside of Target and Source Source picked up Inside of Target Source picked up Inside of Target Source Dropped ^ ^ M o v e Click - ^ j ^ Button Down Button Up ^ Error Figure 6.5: State transition diagrams for both (a) the drag-and-drop interaction style on the left and (b) the point-and-click interaction style on the right. 154 of this error. The software recorded the time at which each mouse event occurred (both mouse button-down events and mouse button-up events) and the position on the screen where the mouse event occurred. In addition, times and positions were also recorded for mouse movement. The movement time for a trial began when the child first attempted to pick up the source box and ended when the source box was successfully dropped inside the target box. Procedure The sessions began with welcoming remarks from the researcher, followed by a brief, verbal introduction to the experimental study. The task involved a series of trials where children were required to pick up a source box on the screen, move it to a target box on the screen, and drop the source box inside the target box. Each child was required to use two different interaction techniques to complete the task: point-and-click and drag-and-drop. The order of the interaction style was counterbalanced within each gender to evenly distribute any practice effects. At the start of each interaction style, the software was demonstrated to the children and they were given one practice block of sixteen trials to become familiar with the interaction style. After the practice block, the children were asked to perform the same task as quickly as they could without making too many mistakes. The children performed four blocks of sixteen trials for each interaction style. At the end of each block, the screen would go blank until the researcher pressed a button. This gave the children a break between blocks. Each child completed the experimental session in one twenty-minute period. Upon completion of the session, each child was asked to rank his or her preference for interaction style on a nine-point scale. To facilitate this procedure for children, a pinwheel was used (see Section 5.2.1). The pinwheel consisted of two different coloured cardboard circles, each divided into eight pieces (see Figure 6.6). 155 Both circles were slit and then placed together so that a portion of each circle could be seen. Each interaction style was assigned a colour and the children were required to turn the pinwheel to indicate which interaction style they preferred and to what degree. If the children preferred the point-and-click interaction style, they would turn the pinwheel so that more of the point-and-click colour was showing. If the children preferred the drag-and-drop interaction style, they would turn the pinwheel so that more of the drag-and-drop colour was showing. If there was no preference of interaction style, the pinwheel could be placed with equal amounts of both colours showing. The number of pie-shaped pieces showing for a particular interaction style colour represents its ranking. A higher ranking reflects a greater preference for that interaction style. 0 Figure 6.6: Photographs of the pinwheels used by children to rate their preference of interaction style on a nine-point scale. The computer recorded the time for movements as well as the number of errors committed. The errors were classified into two categories: pick-up errors and drop errors. Timing in a trial began with the first attempt to pick up the source box and ended when the source box was successfully dropped inside the target box. 156 Experimental Variables The design for this phase of the study was a 2 x 2 x 2 x 2 repeated measures design with two types of interaction style (point-and-click and drag-and-drop), two distances between the source and the target (400 pixels and 800 pixels), two sizes of source box (32 x 32 pixels and 64 x 64 pixels) and two sizes of target box (32 x 32 pixels and 64 x 64 pixels). The results were analyzed independently for girls and for boys. The three dependent measures in this phase were movement time, errors and preference. Movement time was the time it took to complete a trial (pick up the source box, move it over and drop it inside the target box). Errors were the number of incorrect attempts at picking up the source box (pick-up errors) and the number of incorrect attempts at dropping the source box into the target box (drop errors). Preference was the children's ranking of which interaction style they preferred on a nine-point scale. 6.2.2 Results Movement Times The mean movement times for the two interaction styles are shown in Table 6.2. Only trials in which no errors occurred were included in the means. A significant main effect for interaction style was found for girls, with the point-and-click in-teraction style being faster than the drag-and-drop interaction style, .F(l,33) = 6.30, p < .05, with an effect size of .16 and a power of 68%. Boys were also faster using the point-and-click interaction style compared to using the drag-and-drop interaction style, although this main effect was only marginally significant, F(l,32) = 3.96, p = .055, with an effect size of .11 and a power of 49%. The Mauchley sphericity tests were not significant, therefore no adjustment to degrees of freedom was made for girls or for boys. 157 Drag-and-Drop Point-and-Click n M SD M SD F P Girls 34 1347ms 248 1279ms 213 6.30 .017 Boys 33 1301ms 248 1243ms 244 3.96 .055 Table 6.2: Average Movement Times by Gender for Each Mouse Interaction Style Errors The average number of pick-up and drop errors for the two interaction styles are shown in Table 6.3. A significant interaction was found between the type of error and other independent variables, so each category of errors was analyzed separately. The Mauchley sphericity tests were not significant therefore no adjustment to degrees of freedom was made for girls or for boys. Drag-and-Drop Point-and-Click Error Type n M SD M SD F P Girls Drop 34 4.03 3.54 2.53 2.79 8.18 .007 Pick-up 34 8.94 6.11 6.68 5.95 4.47 .042 Boys Drop 33 3.91 4.12 4.36 6.65 0.17 .680 Pick-up 33 9.27 7.48 7.64 7.59 1.62 .213 Table 6.3: Mean Number of Errors Committed for Girls and Boys Using Each Mouse Interaction Style The type of interaction style used produced a significant main effect for both pick-up and drop errors for girls, F(l,33) = 4.62, p < .05 with an effect size of .12 and a power or 55%, and F(l,33) = 8.18, p < .01 with an effect size of .20 and a power of 79%, respectively. Girls committed more pick-up and drop errors using the drag-and-drop interaction style compared to using the point-and-click interaction style. Varying the distance of the movement and the sizes of the source and target boxes didn't cause any significant differences for drop errors, but the size of the source box did produce a significant main effect for girls in terms of the number of 158 pick-up errors they committed. Girls committed significantly more pick-up errors with a source box of size 32 x 32 pixels compared to with a source box of size 64 x 64 pixels, F(l,33) = 19.95, p < .001. The type of interaction style boys used did not significantly affect the number of pick-up or drop errors they committed, F(l, 32) = 1.82, ns with an effect size of .05 and a power of 26%, and F(l,32) = 0.17, ns with an effect size of .005 and a power of 6%, respectively. Varying the distance of the movement and the sizes of the boxes did produce significant differences in the number of errors boys committed. Boys committed significantly more pick-up errors with the small source box than with the large source box, F(l,32) = 11.49, p < .01. Boys committed significantly more drop errors with the small target box than with the large target box, F(l, 32) = 29.52, p < .001. Distance of the movement also affected the number of drop errors boys made, F(l, 32) = 7.50, p < .05, with more drop errors committed in long distance moves than in short distance moves. Fitts's Law As expected, the distance of a movement and the size of a target strongly affected the average movement time of a trial for both girls and boys2 p < .001. The effect sizes for both movement distance and target size, for both girls and boys, was over .8 with a power of 100%. These results suggest that the movement might be modelled by Fitts's Law. Fitts's Law is an information processing model used to predict time to move to a target, where the movement time (MT) is dependent on the distance of the movement and the size of the target, MT = a + b log2{2A/W). A is the amplitude or distance moved to the target and W is the width of the target [Fitts, 1954]. Welford proposed a slight variation of Fitts's Law which was used in this study: MT = a + b log<2(A/W + 0.5). The logarithmic term is commonly referred to 2distance: girls F(l,33) = 917, p < .001, boys F(l,32) = 545, p < .001 target size: girls F(l,33) = 259, p < .001, boys F(l,32) = 140, p < .001 159 as the index of difficulty (ID), and the coefficients a and b are computed through linear regression. Fitts's Law is commonly used to compare mouse-based interaction techniques. The comparison measure is the index of performance (IP), which is simply the reciprocal of the coefficient 6 from the Fitts or Welford equations. The movement times from this phase of the study were analyzed to see if they were accurately modelled by Fitts's Law. In order to compensate for the variance in errors between conditions, Welford's computation of effective target width was used to normalize the results based on the children's observed error rates for each interaction style [Welford, 1968, p.147]. High correlations were found (r) between the time to complete the movement (MT) and the index of task difficulty (ID) for each of the interaction styles for both girls and for boys, indicating that the movement was accurately modelled by Fitts's Law. The indices of performance (IP) as computed through linear regression are shown in Table 6.4. Regression Coefficients Intercept, a Slope, b IP = 1/b (ms) (ms/bit) (bits/s)6 Girls drag-and-drop 0.996 -69 396 2.5 point-and-click 1.000 -40 354 2.8 Boys drag-and-drop 0.994 -278 435 2.3 point-and-click 0.994 -188 401 2.5 an = 4, p < .01 6/P(index of performance) = 1/b Table 6.4: The regression analysis for Fitts's Law based on the prediction model: MT = a + b ID, where ID = log2(A/W + 0.5) The performance indices, IP for girls demonstrate that girls performed better using the point-and-click interaction style than with the drag-and-drop interaction style. Boys also had a higher rate of performance using the point-and-click interac-tion style versus the drag-and-drop interaction style. 160 Preference The results for girls' and boys' preferences of mouse interaction style are shown in Figures 6.7 and 6.8. A rating value represents the pinwheel score for the point-and-click interaction style. For example, if a child adjusted the pinwheel to show three pieces of the point-and-click colour and five pieces of the drag-and-drop colour (as was shown on the left in Figure 6.6), the preference value would be three. The results presented are also categorized by which interaction style the children used first. The difference in preference ratings by order in which the children performed the interaction styles was marginally significant for both girls, .F(l,32) = 3.85, p = .058, and boys, F(l,31) = 3.84, p = .059. Children who used the point-and-click interaction style first were more likely to state a preference for the point-and-click interaction style than were children who used the drag-and-drop interaction style first. Children's preference of interaction style was analyzed using a t-test for a sin-gle mean. The mean was tested against an expected value of four which represents the neutral point of equal preference for the drag-and-drop and the point-and-click interaction styles. The results of this analysis are presented in Table 6.5. Girls signif-icantly preferred the point-and-click interaction style, £(33) = 4.65, p < .001, while no significant preference of interaction style was found for boys, £(32) = 0.68, ns. Children's preference of interaction style can be grouped into three nominal categories: prefer drag-and-drop, no preference, and prefer point-and-click. The results of this grouping are shown in Table 6.6. A ranking of zero to three would be placed in the prefer drag-and-drop group, a ranking of four would be placed in the no preference group, and a ranking of five to eight would be placed in the prefer point-and-click group. A chi-square analysis of preferences for drag-and-drop and point-and-click revealed that significantly more girls preferred the point-and-click mouse interaction style than preferred the drag-and-drop mouse interaction style, X2(1,AT = 31) = 11.65, p < .01. Although more boys did prefer the point-and-161 8 7 6 0 & 4 1 o 2 1 0 prefer drag-and-drop 0 1 n prefer point-and-click • G i r l s ' Preference • Boys' Preference Used Drag-and-Drop Interaction Style F i r s t Figure 6.7: Girls' and boys' ranking of preferred interaction style for children who played using the drag-and-drop interaction style first. The rankings ranged from 0 (strongly prefer drag-and-drop) to 8 (strongly prefer point-and-click). • Boys' Preference Used Point-and-Click Interaction Style F i r s t Figure 6.8: Girls' and boys' ranking of preferred interaction style for children who played using the point-and-click interaction style first. The rankings ranged from 0 (strongly prefer drag-and-drop) to 8 (strongly prefer point-and-click). 162 n Ma SD tb Girls 34 5.21 1.51 4.65* Boys 33 4.30 2.57 0.65 "the rating scale ranges from strongly prefer drag-and-drop (0) to strongly prefer point-and-click (8) using the pinwheel method to select rankings. 6Single-mean £-tests were conducted to test the obtained means again the expected value of four, which represents the neutral point of equal preference for drag-and-drop and point-and-click interaction styles. *p < .001 Table 6.5: Children's Average Ranking for Preference of Interaction Style click method over the drag-and-drop method, this difference was not found to be significant, x 2 ( l , AT = 32) = 1.13, ns. Drag-and-drop Point-and-click No Preference n % n % n % Girls 6 18% 25 73% 3 9% Boys 13 39% 19 58% 1 3% Table 6.6: Percentage of Children who Preferred Each Mouse Interaction Style 6.2.3 D i s c u s s i o n The results for girls in our study were similar to the results found in previous research on adults [MacKenzie et al., 1991] in that movement using the point-and-click interaction style was significantly faster than movement using the drag-and-drop interaction style. Girls also made significantly fewer errors using the point-and-click interaction style compared to the drag-and-drop interaction style. Finally, 73% of girls preferred to use the point-and-click interaction style compared to the drag-and-drop interaction style. Using Fitts's model of performance, which modelled movement time and took into account the number of drop errors using Welford's 163 correction, the point-and-click interaction style had a higher index of performance (IP = 2.8) than did the drag-and-drop interaction style (IP = 2.5) for girls. Boys, on average, were faster and made fewer errors using the point-and-click interaction style compared to the drag-and-drop interaction style, although these differences were not statistically significant. More boys preferred the interac-tion style of point-and-click (58%) compared to the drag-and-drop interaction style (39%), although again this difference was not significant. The Fitts's performance index for the point-and-click interaction style (IP = 2.5) for boys was higher than the index for the drag-and-drop interaction style (IP = 2.3) for boys. Impact of Errors The movement time results from the repeated measures analysis in Section 6.2.2 in-cluded only those trials in which no errors were made. This approach has been used in other studies [Boritz, Booth, and Cowan, 1991; Card, English, and Burr, 1978]. While these results demonstrated that the point-and-click interaction style was faster than the drag-and-drop interaction style, the children in this study made errors on approximately 15% of the trials. By including the trials in which errors occurred, differences between the movement times for drag-and-drop and point-and-click in-creased significantly, F(l,67) = 78.03, p < .001 and F(l,67) = 61.46, p < .001, respectively. Instead of an average difference of 65ms between the two interaction methods, the average difference increased to approximately 250ms. Pick-up Errors The number of pick-up errors children committed was surprisingly high in this study. For both girls and boys, the number of pick-up errors was, in most cases, more than double the number of drop errors. One reason for the high number of pick-up errors could be the presentation of the task: the children were required to pick up a source box and move it to a target box. If the children focused primarily on the dropping portion of the task as opposed to the pick up portion, they may have been looking 164 ahead to the task of reaching the target while they were still attempting to pick up the source box and therefore they may have been careless. A pickup error significantly increased the movement time for both the drag-and-drop interaction style, F(l,63) = 535, p < .001, and the point-and-click in-teraction style, F(l,63) = 394, p < .001. The time implication of a pick-up error was on average 965ms for the drag-and-drop interaction style and 760ms for the point-and-click interaction style. It is interesting that a pick-up error was slightly more costly using the drag-and-drop interaction style than using the point-and-click interaction style. One possible reason could be that getting re-oriented to perform a drag-and-drop motion takes more time than for a point-and-click motion. Also, with relation to the time it takes to recognize that an error has occurred, the movement in each interaction style is different. For a drag-and-drop movement, the children may have begun a dragging movement, maintaining pressure on the mouse button, but in the point-and-click movement, the children would be moving towards the target without maintaining pressure on the mouse button. Because movement while maintaining pressure on the mouse button can be slower, this could cause the recovery period after a pick-up error using the drag-and-drop interaction style to be longer than for the point-and-click interaction style. We did not perform an analysis on the detailed mouse trajectory data. Such an analysis might provide further insight into performance differences. Drop Errors The software handled drop errors quite differently between the point-and-click in-teraction style and the drag-and-drop interaction style. Because the point-and-click movement comprised two discrete actions (one click to pick up the object, one click to drop the object), failure during only one of the actions did not require repeating both actions. In contrast, error recovery during the drag-and-drop movement was more complicated because the action was one physical motion (press the button 165 down to pick up the object, release the button to drop the object). If the target was missed while releasing the button, it was not possible to release the button again until it was depressed again. This phase of the study implemented the assumption that it was not appropriate to leave the source box in an incorrect position because during similar circumstances in the puzzle-solving game environment from the first phase of the study, some objects were not left in incorrect positions by the game software. Because of this, the only alternative was to return the source box back to its original location, before the motion began. This required a child to go back to the very beginning of the trial and pick up the source box again, before re-attempting the drop action. The average time for a drop error was 468ms using the point-and-click in-teraction style and 2,463ms using the drag-and-drop interaction style. On average, one drop error significantly increased the movement time, 38% for point-and-click, F(l,48) = 69.98, p < .001, and 185% for drag-and-drop, F = (1,45) = 372.53, p < .001. Obviously the error handling mechanism used in the drag-and-drop interaction style contributed to the substantial time penalty for drop errors. This highlights one of the difficulties of using a drag-and-drop interaction style. By combining two subtasks into one compound gesture, difficulty arises when an error is made during one of the subtasks. Often, the whole gesture must be repeated instead of just one of the subtasks, causing a substantial increase in the time for the overall movement. Preference The children were all asked why they preferred one method of interaction over another. Many of the children who preferred the point-and-click interaction style explicitly stated that point-and-click was easier than drag-and-drop and they com-plained that the drag-and-drop interaction style made their fingers or hands tired from keeping the mouse button pressed down. Other researchers have also reported that children have difficulty maintaining pressure on the mouse button [Strommen, 166 1994]. Most children who preferred the drag-and-drop interaction style over the point-and-click interaction style explained that it was because they were more fa-miliar with drag-and-drop and they commonly used software at home that involved dragging. Another positive benefit of the drag-and-drop interaction style, mentioned by some of the children, was the tactile feedback it provides. The children explained that they knew they were dragging the box when their finger was maintaining pres-sure on the mouse button. This notion is supported by Buxton, who explains that a kinesthetic connectivity can help to reinforce the conceptual connectivity of the subtasks within a compound gesture [Buxton, 1987]. We elaborate on this in the following subsection. Pros and Cons of Drag-and-Drop Previous research has indicated that a drag-and-drop movement may have an advan-tage over a point-and-click movement because it maps a conceptual task to a physical task [Buxton, 1987]. Our study looked at moving an object from one position to another, which is conceptually a single task. Using a point-and-click interaction style the task is performed in two discrete actions, one click to pick up the object and one click to drop the object. Buxton [1987] refers to these two discrete actions as atomic tasks and suggests that there is a benefit to phrasing or chunking these tasks together into a single action, called a gesture, as in the drag-and-drop move-ment. Phrasing enables connected concepts to be expressed by connected physical gestures. Using the drag-and-drop interaction style, the task can be performed in one fluid action of pressing the mouse button down, dragging the object to its new position, and releasing the mouse button. A drag-and-drop movement also provides the benefit of physical (proprio-ceptive) feedback. In the movement of an object from one position to another, two different states exist: one in which the object is not picked up and one in which 167 the object is picked up. Using a drag-and-drop interaction style, physical feedback provides knowledge of which state a person is currently in through an awareness of muscle tension. If the mouse button is pressed down, the object is in the picked-up state; if the mouse button is not pressed down, the object is not in the picked up state. While phrasing definitely has advantages, many children encounter difficul-ties while performing the drag-and-drop movement. When tasks are chunked to-gether into gestures their connections are reinforced, but separating these tasks later may prove difficult. When an error occurs in a chunked task, error recovery is diffi-cult if the atomic tasks have been fused together. In some cases, instead of having to repeat the portion of the gesture in which the error occurred, the whole gesture has to be repeated. This can be problematic, especially when dealing with children or novice users whose error rates may be high. In addition, the benefit of physical feedback that a drag-and-drop motion provides for children may be small compared to the physical difficulty that children experience while maintaining pressure on the mouse button for long-distance moves. Observing the interaction technique of drag-and-drop from the perspective of children, we recognize that the problems associated with this movement may in fact outweigh the benefits. 6.2.4 Issues Related to Experimental Design The experiment in the second phase of this study was designed to isolate the two mouse interaction styles. There may still have been some confounding factors which we discuss in the following subsections. Number of Trials Children in this study performed one practice block of sixteen trials and four ex-perimental blocks (64 trials in total) for each interaction style. It is possible that 168 the children's performance was still improving from trial to trial and from block to block. The reason for having short practice and experiment sessions was two-fold. First, the motivation for this study was to examine how an interaction style could affect children's use of an unfamiliar piece of software. In addition, we were interested in children's interactions in a school environment. Children using com-puters in a school setting may not be given the opportunity to use a piece of software for extended periods of time, which limits the chance that the children will become proficient at using the software. Moreover, if the interaction style is frustrating for children, they may not persist with the software and consequently may not become proficient. Therefore, the focus was on novice users as opposed to expert users. It is important to note that the children in this study were not novice computer users; 75% of them had computers at home and most played computer or video games frequently. They were novices in the experimental task. A second reason for short sessions was because the subjects were children. The children who participated worked very diligently during their twenty-minute session even though the task was not as exciting as most of their other interactions with computers. Extending the duration of the sessions, with children as subjects, could cause problems because some of the children might not have been motivated to perform a lengthy task for the mere purpose of someone's research study. Figure 6.9 shows the average movement time for each interaction style over the four session blocks. A repeated measures analysis for movement time over the four blocks revealed that the children's speed of movement in the point-and-click interaction style significantly improved over the four blocks, F(3,189) = 5.75, p < .01, while no such improvement was found for the drag-and-drop interaction style, .F(3,192) = 0.50, ns. The Greenhouse-Geisser correction was used to adjust the degrees of freedom for all F-tests because the sphericity assumption was violated. The lack of significant differences for the drag-and-drop interaction style presents two possible alternatives. One alternative is that children were performing 169 1340 1180 -I 1 1 , 1 2 3 4 B l o c k Number —#—Drag-and-Drop —*—Point-and-Click Figure 6.9: Average movement time by session block for drag-and-drop and point-and-click. at their optimal level and, therefore, no improvements were present, nor would they be in additional blocks. A second alternative is that the drag-and-drop interaction style had a much longer learning time and four blocks of trials did not provide enough practice to produce any significant improvements. In this case, additional blocks might have improved the children's performance. Modeling a Compound Task The second phase of this study modelled the compound task of picking up an object and moving it to a target position. The description we gave prior to the practice trials helped the children form a clear mental model of the task and they all seemed to comprehend it easily. This approach differs somewhat from previous research on Fitts's Law in the field of Human-Computer Interaction, which often examines a single movement towards a target from a home position [Card et al., 1978] or a serial tapping task [Fitts, 1954]. Because there was a conceptual framework for our 170 task, the movements we observed may have been different than those for tasks that involve only clicking between multiple targets or selecting a single target on a screen. We believe that our compound task provided a good match to the interactions we observed when children used The Incredible Machine in the first phase of the study, which was our goal in designing the task. One result that supports the claim that the children's movements in the second phase were part of a cognitively more complex task than merely selecting one target then another was the high number of pick-up errors we observed with both the point-and-click and drag-and-drop styles. Children in the second phase of this study had an unusually high number of errors while attempting to pick-up the source box, substantially higher than the number of errors committed while dropping the source box into the target box. The fact that more errors occurred during source acquisition compared to target acquisition highlights the fact that the children's movements were different during the two stages of the task. This might be because the children were already planning their second movement when they attempted to pick up the source. Right to Left Movement An issue of concern is the fact that the movement from source to target in this phase of the study was always in a right-to-left direction. It is possible that this design biased the results for children who may perform better in a left-to-right movement. Handedness may also interact with direction of movement although we did not control for handedness in our study. The choice of limiting this study to one direction of movement was to reduce complexity and to model the environment used in the first phase of the study which was The Incredible Machine. In the game, the parts bin is always located on the right hand side of the screen so that children are normally moving objects from the right hand side of the screen to the left. Further studies would be needed to determine whether the direction of movement influenced 171 our results. Implementation of Point-and-Click The point-and-click interaction style in this phase of the study was implemented using a flexible protocol where the mouse button-down event triggered the action. When the system recognized a mouse button-down event, the location of the cursor and whether or not the source box was picked up was checked and a pick-up or drop was then performed accordingly. With this scenario, the location of the mouse button-up event was irrelevant. The children could perform a drag-like motion to pick-up the source box as long as the mouse button-down event occurred in the source box. When using a drag-like motion, dropping the source box involved releasing the mouse button at some point and then pressing it again within the target box. A traditional point-and-click movement would succeed in this, but so would a drag-and-drop motion if an extra click were added at the end. This provided extra flexibility in the point-and-click interaction style that might have affected our results. In a pilot study conducted as part of the design stage for this phase of the study, the implementation of point-and-click was stricter, requiring both the mouse button-down event and the mouse button-up event to occur within the designated area (i.e., within the source box during a pick-up attempt or within the target box during a drag-and-drop attempt). This requirement was constraining for the chil-dren and caused jerky movements because the mouse would almost have to come to a stop during the pick-up and drop actions. Many children attempted a more fluid movement where the mouse button-up event may have occurred after the pick-up, during the move to the target, which caused an error to occur with the stricter imple-mentation of point-and-click used in the pilot study. This demonstrates that even a slight difference in implementation can have a strong impact on the results. We chose to implement the more flexible version of point-and-click to simulate the interface 172 used in the first phase of our study, because The Incredible Machine also permitted a lazy button-up during a click. The state-transition diagrams that compare the more flexible implementation of point-and-click with the restrictive implementation of point-and-click are shown in Figure 6.10. Analyzing the state transition diagrams for two possible implementations of point-and-click we notice that there are four possible states in which an error could occur in the restrictive version, compared to only three states in the flexible version. Even worse, two of the errors in the restrictive version cause a player to move back multiple states, where players using the flexible version remain in the same state after committing an error. Finally, the restrictive version has two additional states that do not appear in the flexible version. As was the case for the two interaction styles of The Incredible Machine described in Chapter 3, detailed analysis of interaction styles through examination of the state transition diagrams reveals subtle but important differences, especially with respect to error handling. E r r o r Handling How a system handles interaction errors can impact the effectiveness of the software. In the second phase of this study, error handling was implemented to resemble the error handling in the software used in the first phase. The point-and-click interaction style had a forgiving manner of dealing with drop errors: the source box remained picked up, attached to the cursor, ready for another drop attempt. This alternative was not available in a drag-and-drop style of interaction because of the inherent semantics. This left two possibilities for error handling: 1. the system could start the trial over again, forcing a child to go back and pick up the source box again; 2. the system could partially complete the task, dropping the source box in an incorrect position, requiring the child to pick-up the source box from this new position and then attempt to drop it in the target box again. 173 (a) Restrictive * * * * * * Point-and-Click Point-and-Clicl Outside of Source Source not picked up M 1W Inside of Source, Object picked up Inside of Source Source not picked up 3 _ r Inside of Source Source picked up Outside of Target and Source Source picked up | Inside of Source, Object picked up Inside of Target Source picked up ___ _ Inside of Target Source Dropped ^ ^ M o v e Click Button Down Button Up ^ Error Figure 6.10: State transition diagrams for (a) the restrictive point-and-click inter-action style on the left and (b) the flexible point-and-click interaction style on the d g h t - 174 Our implementation required that the whole movement be repeated, as is the case when an object is placed in an invalid position in The Incredible Machine. If instead the system allowed a partial completion, dropping the source box in a position that would usually be closer to the target box, this could have significantly impacted the children's preferences. Our system was frustrating for the children when they had to go back to the starting position to pick up the object again. Examining how a soft-ware environment handles interaction errors should be an important consideration when deciding which interaction style to use, and explicit consideration should be given to the class of intended users. Children may have different needs than adults when it comes to error handling. 6.3 Conclusion We draw both specific conclusions about the two mouse interaction styles used in the studies and about our research strategies, and more general conclusions about the design of children's software. Drag-and-Drop vs. Point-and-Click The results of this study show that utilizing a point-and-click interaction style in children's software can be more effective than using a drag-and-drop interaction style, especially for girls. Girls were able to perform point-and-click interactions significantly faster and with significantly fewer errors than drag-and-drop interac-tions and most girls preferred the point-and-click interaction style. The results for boys showed similar trends but the results were not statistically significant. This study also dealt with the issue of whether or not the choice of mouse interaction style impacts children's motivation and achievement in a learning envi-ronment. Do a few milliseconds or a couple of extra errors once in a while really make a difference for children's use of an interactive learning environment? The results of our study demonstrate that for girls the choice of interaction style can significantly 175 impact both their motivation and their achievement. Girls using the point-and-click version of The Incredible Machine were more motivated to continue playing and were more successful in the game than were girls using the drag-and-drop version of the game. Our results with children contrast with previous research by Lim, Benbasat and Todd [1996] on adults' use of direct-manipulation (drag-and-drop) interfaces and menu (point-and-click) interfaces [Lim, Benbasat, and Todd, 1996]. They observed similar results to ours for the isolated movements of drag-and-drop and point-and-click: the point-and-click interface required less motor time than did the drag-and-drop interface. On the other hand, the use of these two interaction styles within a complex task did not affect performance in the complex task. No statistically significant difference in the total time to complete the task was observed. One explanation for the difference could be attributed to the errors committed by children while playing The Incredible Machine in our study. Errors were frequent and this may have contributed to the childrens' difficulties with the drag-and-drop interaction style that we measured in our study. In the Lim, Benbasat and Todd study, adults made very few errors. While the second phase of the study was designed to followup on the observa-tions made in the first phase, it is difficult to directly compare the results from the complex interactive environment used in the first phase to the controlled, simple tasks the children performed in the second phase. In the second phase, the chil-dren's sole focus was to move the source box over to the target box. In the complex puzzle-solving game environment of the first phase, the children had many other things to focus on: trying to decide where to place an object in order to satisfy the goal; thinking about the next move; imagining what the machine will do when it is run. Al l of these tasks would be competing with the cognitive load needed to perform the appropriate mouse interactions within the game. It is possible that the increased cognitive load required to play the game also increased the impact that an 176 interaction style had on children's ability to perform the interactions and therefore it affected their motivation and success in the learning environment. For all of these reasons, the results of this study are limited by the exper-imental design choices and the software, including the implementation of the two interaction styles and the tasks performed. What this study does reveal is that the choice of mouse interaction style can have a strong effect on children's interactions and that it is important to fully understand the effect an interaction style may have. Research Strategies This study highlights the importance of using multiple research strategies to max-imize generalizability, precision, and realism [McGrath, 1995]. The justification of this study arose from the lack of strict control in our previous research: two different computer platforms were used in two different phases of a study (see Chapter 4). This lack of control may have appeared to be an inherent flaw in the design of the earlier study, but it did provide realism and it uncovered an important interaction issue for children which may have otherwise been overlooked. The study reported in this chapter used multiple research methods to explore the issue of mouse interaction style. The first phase of the study was a field study which focussed on investigating children's natural interactions while they played the computer game The Incredible Machine, controlling for the type of platform used. This phase provided realism but was limited in terms of precision because the differences observed between the computer platforms may not have been completely attributable to the interaction style used. The second phase of this study used a controlled experiment to gain more precise knowledge of children's use of both interaction styles but lacked realism because the task the children performed only represented one action from the complex game environment used in the first phase. By combining the information obtained from both phases of the study we gain a better understanding of how the choice of interaction style can impact children's use 177 of interactive learning environments. Impact on the Design of Children's Software The results of this study are significant for designers of children's software because often such software is developed without much testing involving children. Moreover, there is little research on effective support of children's interactions with computers, and what research does exist is often ignored by software developers. For example, previous researchers have observed that some children have difficulty performing a dragging motion because of the physical requirements needed to maintain constant pressure on the mouse button. Research on adults has shown that a dragging task is slower and that more errors are made as compared to a pointing task. Despite this knowledge, children's software is often implemented to utilize a drag-and-drop interaction style. Bringing solid research and strong results, such as the study discussed in this chapter, to the forefront may help make designers of children's software think more about the implications of their design choices. As the use of computers becomes more prominent in schools we need to be even more sensitive to how children interact with computers. Computers and soft-ware are no longer being used only by those who wish to play games in their spare time. While many children are quite proficient in their interactions with computers and many have adapted to electronic game interfaces, this is not true of all children. The designers of children's software must be careful not to assume too much about children's ability to adapt to their systems, especially in the design of educational software because, as shown in this study, design choices do significantly affect chil-dren's interactions with the software. In a school environment, as a learning tool, computers will be used by all students, so the software must be accessible to all students. Finally, this research confirms previous research that shows that gender dif-ferences exist in children's interactions with computers. Examining the interaction 178 styles of point-and-click and drag-and-drop in the second phase of the study showed that statistically significant differences existed for girls but not for boys. This sug-gests the possibility that the choice of interaction style may matter more for girls than for boys. Rather than focusing on why the results were different for girls and boys, we choose instead to simply accept that girls and boys may interact differently with computers. Our goal then is to find alternatives that can benefit both girls and boys. In this study the choice is easy. Point-and-click was a significantly better interaction style for girls than was drag-and-drop, and it was preferred by most girls. The results for boys showed that the point-and-click interaction style was at least as good if not better than the drag-and-drop interaction style and that the majority of boys preferred the point-and-click interaction style over drag-and-drop. 179 Chapter 7 A Research Agenda [W]e are trying to conceive a new way of thinking about computers in the world, one that takes into account the natural human environment and allows the computers themselves to vanish into the background. - Mark Weiser The Computer for the 21st Century, 1991, page 94. 180 Results from our research have uncovered a number of additional areas for future study. The discussion of these research directions will be from varying per-spectives and are relevant to researchers from multiple disciplines. Advancing the technology is of little use if the learning environments are inadequate or if the the-ories are never put into practice in the classroom. 7.1 How Many Computers? As computers become more prevalent in schools and we move closer to the possibility of having one computer for every student, we need to question whether or not this is a desirable goal. Even in schools today, when classes go to the computer lab the teacher is often forced to decide how to distribute the limited number of computers. Our research reported in Chapter 4 showed that in some environments having children work together is better than: having them work either alone or on side-by-side computers. Furthermore, the results from Chapter 5 indicated that modifying the turn-taking protocol children use in a collaborative session can provide additional benefits. Our research also impacts the view that one computer for every child is ideal. The results in Chapter 4 showed that girls playing on side-by-side computers may have more difficulty than if they played totally by themselves! Considering that children using computers in a school lab are often asked to work on side-by-side computers, this is an important result. 7.1.1 Research Goal #1: Situated Observations The controlled study reported in Chapter 4 focused on pairs of children playing in an empty classroom for one class period. It is important to re-examine this issue in a more realistic school setting such as a classroom or computer lab over a longer period of time. Focusing on qualitative observation would help provide a good understanding of the children's complex interactions in this rich environment. 181 7.1.2 Research Goal #2: Shared Virtual Spaces When we explored children collaborating on side-by-side computers in Chapter 4, each child was running their own version of the game software. One of the difficulties encountered was the lack of motivation to help a partner because it had no direct effect on their own success in the game. A future study could re-examine children working either together on one computer or side-by-side on two computers working in a virtual space. In this scenario, the children would both be working on the same puzzle but they would each have their own computer to interact with and their individual views would show what their partner was doing. 7.1.3 Research Goal #3: Collaboration at a Distance Face-to-face communication appeared to be a strong factor in the children's success while playing with a partner on the same computer in the research reported in Chapter 4. Examining whether this same level of interaction could be achieved when children are collaborating over a network at a distance is another possible extension of our work. 7.2 How Many Input Devices? One of the difficulties of having children collaborate on a single computer is the problem of having only one input device. As observed in our research described in Chapter 4, especially in the videotape data, sharing a single mouse can sometimes lead to physical struggles over who gets to operate the mouse. By providing multiple input devices we may be able to better facilitate the children's collaboration. Our research not only demonstrated that conflicts can arise when children share a single mouse, it demonstrated that the addition of a second mouse, even if the access is still sequential, can impact children's behaviour, achievement, learning, and motivation. Chapter 5 also indicated that the protocol used to share control of 182 the game can also impact the outcomes of a collaborative session. A give protocol appeared to have a positive impact on achievement for girls while a take protocol appeared to have a positive impact on learning for boys. 7.2.1 Research Goal #4: Concurrent Collaboration In our research on multiple input devices in Chapter 5 the conditions that utilized two mice still required turn-taking on the part of the children. The game had only one cursor and only one mouse was active at a time. The difference came from pressing a button on the mice to transfer control between the two mice as opposed to physically passing a single mouse. The next step would be to provide the children with an environment in which both mice were active at the same time so they could interact concurrently with the game. It would be important to explore whether this change would enhance the children's collaboration because the children could be more efficient without having to wait for their partner, or if it would hamper their collaboration because without the need to share a limited resource the necessity for communication no longer exists. 7.2.2 Research Goal #5: Longitudinal Studies Another area for future work includes examining the two-mouse configurations from the research in Chapter 5 in a realistic classroom environment over a prolonged period of time. The children in our study were not familiar with our setup of two mice and their interactions may have changed once they became familiar with the setup. 7.2.3 Research Goal #6: Two-Mouse Protocols at a Distance The two-mouse conditions and turn-taking strategies used in Chapter 5 relied heav-ily on the children's face-to-face verbal interactions. A future direction could be to determine whether the turn-taking protocols of the two-mouse give and two-183 mouse take configurations would be as effective when children were not co-located and determining what forms of communication would be necessary to make this collaboration effective. 7.3 How Do We Measure Learning? Measuring the amount of knowledge that children gain while playing in a computer environment is difficult. This difficulty is compounded when groups of children work together. When groups of children collaborate to produce a single outcome it is difficult to determine if all children contributed or if one child dominated the work. This issue of individual accountability has been shown to be a critical aspect of cooperative learning. Our research on supporting children's collaboration in an educational learn-ing environment examined children's performance on two levels which we refer to as achievement and learning. Achievement was the number of puzzles the children could solve while playing The Incredible Machine collaboratively. Learning was the amount of knowledge and skills the children could transfer from this collaborative session to a followup solo session, which we quantified as the number of puzzles the children solved in the followup solo session. These simplistic measures of achieve-ment and learning along with our qualitative observations indicated that, while difficult for us to quantify, the children appeared to benefit from the collaboration and from our attempts to more effectively support this collaboration. Children playing together on a single machine appeared to gain a better understanding of the concepts in the game than did children playing alone or on side-by-side machines. This was also enhanced by adding a second mouse to the computer, using a give turn-taking protocol for girls and a take turn-taking protocol for boys. 184 7.3.1 Research Goal #7: Alternative Measures of Achievement and Learning Our research in Chapter 4 and Chapter 5 focused on the number of puzzles the children could solve in a collaborative session as well as a followup solo session. Evaluating the children's achievement and learning though alternative measures would help to validate the results from our study. In addition, analysis of the children's verbal communication could provide insight into the effectiveness of the children's collaboration session. 7.3.2 Research Goal #8: Extended Collaborative Sessions Our research only touched the surface with respect to exploring children's learning while playing in various collaborative configurations. One of the limitations of our study described in Chapter 5 was that the children only played in their collaborative session for twenty-five minutes before being evaluated in a followup solo session. Re-investigating these issues while extending the children's playing time in their collaborative sessions would provide stronger results on whether the collaborative configuration had any effect on the children's learning. 7.3.3 Research Goal #9: Structured Collaboration The children in our research in Chapter 4 and Chapter 5 were asked to play to-gether in a specific collaborative configuration and we made no further attempts to structure the interaction. The goal was to observe children as they collaborated "naturally". No control was provided for individual accountability or goal, resource or reward interdependence. In addition the children were not trained in cooperative procedures. A future direction would be to explore the collaborative setups from our research in a study that ensures that all the necessary components of cooperative learning are present. 185 7.4 How Do We Facilitate Children's Interactions With Computers? As shown throughout our research, children's achievement and learning are sensitive to factors at many levels. Whether it be the social environment, the computer environment, or the user interface, it is important that we understand all factors that may affect children's interactions with computers. In the past this has been a strong research focus on adult's use of computers in the workplace. For the future it is important that research also focus on other user groups, such as children, and other environments, such as schools. The development of usability guidelines for children's use of computers in education may help to ensure that the technology we place in the classroom is accessible and beneficial for all children. Our research in Chapters 4 through 6 demonstrated that issues such as the way children are assigned to computers; the limitations of the computer systems to support multiple children's interactions; or the lack of sensitivity to the physical and cognitive abilities of children when designing interaction styles, can all have a positive or a negative impact on children's use of technology. The research de-scribed in Chapter 6 also clearly shows that in some cases children's interactions with computers are different than those of adults. As a result, it is important that we not blindly apply our existing knowledge of adults in the workplace to children in school. 7.4.1 Research Goal #10: Alternative Input Devices We explored children's use of two mouse-based interaction styles in Chapter 6 to determine if one interaction style was superior to the other in terms of the children's performance and preference. Although the mouse is one of the most common input devices there may be other more effective input devices for children. Investigation of other input devices for children is needed to determine if they are beneficial or 186 detrimental for activities in the classroom. 7.4.2 Research Goal #11: Naturalistic Observations Our research on children's use of the drag-and-drop and point-and-click interaction styles in Chapter 6 arose accidentally when observing children in a different study. We can learn many things about how to effectively design children's software by casual observations of children using existing software. 7.4.3 Research Goal #12: Ubiquitous Computing Our research throughout this dissertation focused on slight modifications to the computer technology found in schools today. By constraining ourselves to our cur-rent view of computers we may limit our horizons. An important research direction would be to look at new technologies to support children's collaboration such as a Liveboard or hand-held notebooks. Sometimes it is necessary to discard our precon-ceived ideas about how computer technology should be used so that truly innovative solutions can be uncovered. 7 . 5 How Do We Deal With Gender Differences? It is important that any of the research agendas described above be sensitive to potential gender differences and that we continue to strive to recognize and respond to these differences to ensure that computer technology placed in an educational environment is useful for both girls and boys. Al l of our research studies discussed in this dissertation uncovered gender differences in the way children approach and interact with computers. In Chapter 4 we found that girls were able to solve the most puzzles playing with a partner on the same computer. In Chapter 5 we found that the addition of a second mouse, using a give protocol to transfer control between the two mice, enabled girls to 187 solve even more puzzles. Finally, in Chapter 6 we found that the interaction style utilized in a learning environment can also contribute to girls' successes or failures. In contrast, the impact of playing configuration and interaction style was less for boys. Playing with a partner or alone, on one machine or two, had no significant effect for boys but the turn-taking strategy utilized when boys played together on the same machine may have impacted their learning. Boys were most successful in a followup solo session after playing together on the same machine, using a take protocol to transfer control between the two mice. Also in contrast to girls, the choice of mouse interaction style did not show any significant effects on boys' performance in a learning environment. 7.5.1 Research Goal #13: Observations in Varied Environments Besides being sensitive to gender issues in our research, an important direction is to spend time observing girls and boys interacting with computers in a variety of environments. These environments could include a public school classroom, an all-girl or all-boy classroom, an informal learning environment such as a museum, a technology-rich rich and a technology-poor school, and children's homes. This type of research could help us to better understand how girls and boys interact with the technology and provide insight into how we can develop environments that are equally beneficial for both girls and boys. 188 Bibliography Anderson, A., Tolmie, A., McAteer, E., and Demissie, A. (1993). Software style and interaction around the microcomputer. Computers and Education, 20(3), 235-250. Baecker, R. M . (1993). Readings in Groupware and Computer-Supported Cooper-ative Work. San Mateo, CA: Morgan Kaufmann. Baecker, R. M . , Glass, G., Mitchell, A., and Posner, I. (1994). SASSE: The col-laborative editor. In CHI '94: Human Factors in Computing Systems, Boston, MA, pp. 459-460. Conference Companion. Becker, H. J. (1992). A model for improving the performance of integrated learning systems: Mixed individualized/group/whole class lessons, cooperative learn-ing, and organizing time for teacher-led remediation of small groups. Educa-tional Technology, 32(9), 6-15. Becker, H. J. (1994). Analysis and trends of school use of new information tech-nologies. [Prepared for the U.S. Congress Office of Technology Assessment and Posted on the World Wide Web]. Retrieved June 22, 1997 from the World Wide Web: http://olympia.gse.uci.edu/EdTechUse/C-TBLCNT.HTM. Bederson, B. B., Hollan, J. D., Perlin, K. , Meyer, J., Bacon, D., and Furnas, G. (1986). Pad+-H A zoomable graphical sketchpad for exploring alternate interface physics. Journal of Visual Languages and Computing, 7, 3-31. Berkovitz, J. (1994). Graphical interfaces for young children in a software-based mathematics curriculum. In CHI '94: Human Factors in Computing Systems, Boston, MA, pp. 247-248. Conference Companion. Bier, E. A. and Freeman, S. (1991). M M M : A user interface architecture for shared editors on a single screen. In UIST '91: ACM Symposium on User Interface Software and Technology, pp. 79-86. Blank, D. E. (1982). Pac Man™ goes to school: Teaching problem solving in the electronic age. The Computing Teacher, 10(1), 50-53. Boritz, J., Booth, K. S., and Cowan, W. B. (1991). Fitts' law studies of directional mouse movement. In Graphics Interface '91, pp. 216-223. 189 Borning, A. (1986). Graphically defining new building blocks in ThingLab. Human-Computer Interaction, 2(4), 269-295. Borys, S. and Perlman, D. (1985). Gender differences in loneliness. Personality and Social Psychology Bulletin, 11, 63-74. Bowman, J. R. F. (1982). A "Pac-Man" theory of motivation: Tactical implica-tions for classroom instruction. Educational Technology, 22(9), 14-16. Braun, C. M . J. and Giroux, J. (1989). Arcade video games: Proxemic, cognitive and content analyses. Journal of Leisure Research, 21(2), 92-105. Bricker, L. J., Tanimoto, S. L., Rothenberg, A. I., Hutama, D. C , and Wong, T. H. (1995). Multi-player activities that develop mathematical coordination. In CSCL '95: First International Conference on Computer Support for Col-laborative Learning, pp. 32-39. Buxton, W. (1987). There's more to interaction than meets the eye: Some is-sues in manual input. In W. S. B. Ronald M . Baecker (Ed.), Readings in Human-Computer Interaction: A Multidiscliplinary Approach, pp. 366-375. San Mateo, CA: Morgan Kaufmann. Card, S. K., English, W. K., and Burr, B. J. (1978). Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a crt. Ergonomics, 21, 601-613. Carrier, C. A. and Sales, G. C. (1987). Pair versus individual work on the ac-quisition of concepts in a computer-based instructional lesson. Journal of Computer-Based Instruction, i ^ ( l ) , 11-17. Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-459. Clinton, President W. J. (1996). State of the Union Ad-dress. Retrieved June 25, 1997 from the World Wide Web: http: / / www2.whitehouse.gov / WH/New / other / sotu.html. Cockburn, A. and Greenberg, S. (1995). Turbo turtle: A collaborative microworld for exploring Newtonian physics. In CSCL '95: First International Conference on Computer Support for Collaborative Learning, pp. 62-66. Cohen, E. G. (1994). Restructuring the classroom: Conditions for productive small groups. Review of Educational Research, 64(1), 1-35. Cole, K. (1995). Equity issues in computer-based collaboration: Looking beyond surface indicators. In CSCL '95: First International Conference on Computer Support for Collaborative Learning, pp. 67-74. 190 Consumer Reports (1995). Softwaxe for kids more than fun and games. Coupland, D. (1995). microserfs. Toronto, ON: HarperCollins. Dalton, D. W., Hannafin, M . J., and Hooper, S. (1989). Effects of individual and cooperative computer-assisted instruction on student performance and attitudes. Educational Technology Research and Development, 37(2), 15-24. David, A. and Ball, M . P. (1986). The video game: A model for teacher-student collaboration. Momentum, 17(1), 24-26. Dorval, M . and Pepin, M . (1986). Effect of playing a video game on a measure of spatial visualization. Perceptual Motor Skills, 62, 159-162. Druin, A., Stewart, J., Proft, D., Bederson, B., and Hollan, J. (1997). Kidpad: A design collaboration between children, technologists, and educators. In CHI '97: Human Factors in Computing Systems, Atlanta, GA, pp. 463-470. Durndell, A., Glissov, P., and Siann, G. (1995). Gender and computing: Persisting differences. Educational Research, 37(3), 219-227. Elrod, S., Bruce, R., Gold, R., Goldberg, D., Halasz, F., Janssen, W., Lee, D., McCall, K. , Pedersen, E., Pier, K. , Tang, J., and Welch, B. (1992). Live-board: A large interactive display supporting group meetings, presentations and remote collaboration. In CHI '92: Human Factors in Computing Systems, Monterey, CA, pp. 599-607. Emihovich, C. and Miller, G. E. (1988). Learning LOGO: The social context of cognition. Journal of Curriculum Studies, 20(1), 57-70. Fink, C. M . (1990). Cooperating with computers: An educational strategy for students with behavior disorders. Preventing School Failure, 34(4), 20-24. Fitts, P. M . (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychol-ogy, 47(6), 381-391. Gagnon, D. (1985). Videogames and spatial skills: An exploratory study. Educa-tional Communication and Technology, 33(4), 263-275. Gillan, D. J., Holden, K., Adam, S., Rudisill, M . , and Magee, L. (1990). How does Fitts' law fit pointing and dragging? In CHI '90: Human Factors in Computing Systems, Seattle, WA, pp. 227-234. Goldberg, R. (1979). The Best of Rube Goldberg, compiled by Charles Keller. Englewood Cliffs, NJ: Prentice-Hall. 191 Goldman, S., Moschkovich, J., and through Applications Project Team, T. M. -S. M . (1995). Environments for collaborating mathematically: The middle-school mathematics through applications project. In CSCL '95: First In-ternational Conference on Computer Support for Collaborative Learning, pp. 143-146. Greenfield, P. M . (1984). Mind and Media: The Effects of Television, Video Games, and Computers. Cambridge, MA: Harvard University Press. Grudin, J. (1991). CSCW. Communications of the ACM, 34(12), 30-34. Grudin, J. (1994). Groupware and social dynamics: Eight challenges for develop-ers. Communications of the ACM, 57(1), 93-105. Guzdial, M . , Turns, J., Rappin, N. , and Carlson, D. (1995). Collaborative support for learning in complex domains. In CSCL '95: First International Conference on Computer Support for Collaborative Learning, pp. 157-160. Hall, J. and Cooper, J. (1991). Gender, experience and attributions to the com-puter. Journal of Educational Computing Research, 7(1), 51-60. Hawkins, J., Sheingold, K. , Gearhart, M . , and Berger, C. (1982). Microcomputers , in schools: Impact on the social life of elementary classrooms. Journal of Applied Developmental Psychology, 3(4), 361-373. Hodes, C. L. (1995). Gender representation in mathematics software. Journal of Educational Technology Systems, 24(1), 67-73. Huff, C. and Cooper, J. (1987). Sex bias in educational software: The effect of designers' sterotypes on the software they design. Journal of Applied Social Psychology, 17(6), 519-532. Hymel, S., Beaumont, S. L., McDougall, P., and Zarbatany, L. (1997). Children's communication styles in cooperative and competitive contexts: An alternative explanation of sex differences. Article Submitted for Publication. Hymel, S., Zinc, B., and Ditner, E. (1993). Cooperation versus competition in the classroom. Exceptionality Education Canada, 5(1-2), 103-128. Inkpen, K. M . (1997). Three important research agendas for educational multi-media: Learning, children, and gender. In World Conference on Educational Multimedia and Hypermedia '97, Calgary, AB, pp. 521-526. Inkpen, K. M. , Booth, K. S., Gribble, S. D., and Klawe, M . M . (1995). Give and take: Children collaborating on one computer. In J. Bowers and S. Benford (Eds.), CHI '95: Human Factors in Computing Systems, Denver, CO, pp. 258-259. A C M . Conference Companion. 192 Inkpen, K. M . , Booth, K. S., and Klawe, M . M . (1995). Playing together beats playing apart, especially for girls. In CSCL '95: First International Conference on Computer Support for Collaborative Learning, pp. 177-181. Inkpen, K. M . , McGrenere, J., Booth, K. S., and Klawe, M . (1997). The effect of turn-taking protocols on children's learning in mouse-driven collaborative environments. In Graphics Interface '97, Kelowna, BC, pp. 138-145. Inkpen, K. M . , Upitis, R., Klawe, M . , Lawry, J., Anderson, A., Ndunda, M . , Sedighian, K., Leroux, S., and Hsu, D. (1994). "We have never-forgetful flowers in our garden:" Girls' responses to electronic games. Journal of Computers in Math and Science Teaching, 13(A), 383-403. Ishii, H. and Ohkubo, M . (1990). Design of Team Workstation: A realtime shared workspace fusing desktops and computer screens. In S. Gibbs and A. Verrijn-Stuart (Eds.), Multi-User Interfaces and Applications, pp. 131-142. North-Holland: Elsevier Science Publishers. Jacob, R., Sibert, L., McFarlane, D., and Mullen, M . (1994). Integrality and separability of input devices. ACM Transactions on Computer Human Inter-action, 1 (1), 3-26. James, C. B. (1989). Cooperative learning in the classroom. The Social Stud-ies, 80(3), 98-101. Johnson, D. W. and Ahlgren, A. (1976). Relationship between student attitudes about cooperation and competition and attitudes toward schooling. Journal of Educational Psychology, 68(1), 92-102. Johnson, D. W. and Johnson, R. T. (1986). Computer-assisted cooperative learn-ing. Educational Technology, 26, 12-18. Johnson, D. W., Maruyana, G., Johnson, R., Nelson, D., and Skon, L. (1981). Effects of cooperative, competitive, and individualistic goal structures on achievement: A meta-analysis. Psychology Bulletin, 89(1), 47-62. Johnson, R. T., Johnson, D. W., and Stanne, M . B. (1985). Effects of cooper-ative, competitive, and individualistic goal structures on computer-assisted instruction. Journal of Educational Psychology, 77(6), 668-677. Johnson, R. T., Johnson, D. W., and Stanne, M . B. (1986). Comparison of computer-assisted cooperative, competitive, and individualistic learning. American Educational Research Journal, 23(3), 382-392. Kim, S. (1990). The Art of Human-Computer Interface Design, pp. 31-44. Addison-Wesley. 193 Klawe, M . and Phillips, E. (1995). A classroom study: Electronic games that engage children as researchers. In CSCL '95: First International Conference on Computer Support for Collaborative Learning, pp. 209-213. Klawe, M . , Westrom, M . , Davidson, K., and Super, D. (1996). Phoenix quest: lessons in developing an educational computer game for girls ... and boys. In ICMTM 96. Klawe, M . M . (1992). Bringing mathematical research to life in the schools. Tech-nical Report 92-33, Department of Computer Science, University of British Columbia, Vancouver, British Columbia, V6T 1Z4. Koschmann, T., Newman, D., Woodruff, E., Pea, R., and Rowley, P. (1993). Technology and pedagogy for collaborative problem solving as a context for learning: Report on a CSCW '92 workshop. SIGCHI Bulletin, 25(4), 57-60. Koschmann, T. D., Feltovich, P. J., Myers, A. C , and Barrows, H. S. (1992). Implications of CSCL for problem-based learning. SIGCUE Outlook, 21(3), 32-35. Krendl, K. A. and Lieberman, D. A. (1988). Computers and learning: A review of recent research. Journal of Educational Computing Research, 4 {4), 367-389. Kulik, J. A., Bangert, R. L., and Williams, G. W. (1983). Effects of computer-based teaching on secondary school students. Journal of Educational Psychol-ogy, 75(1), 19-26. Lawry, J., Upitis, R., Klawe, M . , Anderson, A., Inkpen, K., Ndunda, M . , Hsu, D., Leroux, S., and Sedighian, K. (1995). Exploring common conceptions about boys and electronic games. Journal of Computers in Math and Science Teach-ing, 14(4), 439-459. Lepper, M . R. and Malone, T. W. (1987). Intrinsic motivation and instructional ef-fectiveness in computer-based education. In R. E. Snow and M . J. Farr (Eds.), Conative and Affective Process Analyses, pp. 255-286. Hillsdale, NJ: L. Erl-baum. Lim, K. H., Benbasat, I., and Todd, P. A. (1996). An experimental investigation of the interactive effects of interface style, instructions, and task familiarity on user performance. A CM Transactions on Computer-Human Interaction, 3(1), 1-37. Lim, K. H., Ward, L. M . , and Benbasat, I. (1997). An empirical study of com-puter system learning: comparison of co-discovery and self-discovery methods. Information Systems Research,, In Press. 194 Lockheed, M . and Hall, K. (1976). Conceptualizing sex as a status characteristic: Applications to leadership training strategies. Journal of Social Issues, 32(3), 111-124. Lockheed, M . E. (1985). Women, girls and computers: A first look at the evidence. Sex Roles, 13(3/4), 115-122. Long, S. M . and Long, W. H. (1984). Rethinking video games: A new challange. The Futurist, 18(6), 35-37. Lowery, B. R. and Knirk, F. G. (1982). Micro-computer video games and spatial visualization acquisition. Journal of Educational Technology Systems, 11(2), 155-166. MacKenzie, S., Sellen, A., and Buxton, W. (1991). A comparison of input devices in elemental pointing and dragging tasks. In CHI '91: Human Factors in Computing Systems, New Orleans, Louisiana, pp. 161-166. Male, M . (1986). Cooperative learning for effective mainstreaming. The Comput-ing Teacher, U(l), 35-37. Mantei, M . (1989). Observations of executives using a computerized supported meeting environment. International Journal of Decision Support Systems, 5, 153-166. Marzio, P. (1973). Rube Goldberg: His Life and Work. New York, NY: Harper & Row. McGrath, J. E. (1995). Methodology matters: Doing research in the behavioral and social sciences. In R. M . Baecker, J. Grudin, W. A. S. Buxton, and S. Greenberg (Eds.), Readings in Human-Computer Interaction: Towards the Year 2000, pp. 152-169. San Francisco, CA: Morgan Kaufmann. McGrenere, J. (1996). Design: Educational electronic multi-player games, a lit-erature review. Technical Report 96-12, Department of Computer Science, University of British Columbia, Vancouver, British Columbia, V6T 1Z4. McGrenere, J. and Inkpen, K. (1996). Experimental design: Input device proto-cols and collaborative learning. Technical Report 96-11, Department of Com-puter Science, University of British Columbia, Vancouver, British Columbia, V6T 1Z4. Mevarech, Z. R., Stern, D., and Levita, I. (1987). To cooperate or not to cooperate in CAI: That is the question. Journal of Educational Research, 80(3), 164-167. Mitchell, A., Posner, I., and Baecker, R. (1995). Learning to write together using groupware. In J. Bowers and S. Benford (Eds.), CHI '95: Human Factors in Computing Systems, Denver, CO, pp. 288-295. A C M . 195 Nastasi, B. K. and Clements, D. H. (1993). Motivational and social outcomes of cooperative computer education environments. Journal of Computing in Childhood Education, ^(1), 15-43. National Council of Teachers of Mathematics (1989). Curriculum and evaluation standards for school mathematics. Reston, VA. Newman, D. (1988). Sixth graders and shared data: Designing a L A N environ-ment to support collaborative work. In CSCW '88: Proceedings of the Confer-ence on Computer-Supported Cooperative Work, Portland, Oregon, pp. 291-305. Norman, K. L. (1997). Teaching in the Switched On Classroom. Published on the World Wide Web. Retrieved July 5, 1997 from the World Wide Web: http://www.umd.edu/soc/. Okey, J. R. and Majer, K. (1976). Individual and small-group learning with computer-assisted instruction. Audio- Visual Communication Review, 24(1), 79-86. O'Malley, C. and Koschmann, T. (1993). Collaborative learning environments. Tutorial Notes HCI '93. Papert, S. (1980). Mindstorms: Children, Computers and Powerful Ideas. New York, NY: BasicBooks. Papert, S. (1993). The Children's Machine: Rethinking School in the Age of the Computer. New York, NY: BasicBooks. Phoenix Quest Web Site (1996). Retrieved June 25, 1997 from the World Wide Web: http://www.cs.ubc.ca/nest/egems/pq.html. Provenzo, E. F. (1991). Video Kids: Making Sense of Nintendo. Cambridge, MA: Harvard University Press. Quality Education Data (1996). Educational Technology Trends, 9th Edition. [Re-search Results Posted on the World Wide Web]. Retrieved July 5, 1997 from the World Wide Web: http://www.qeddata.com/instbase.html. Repman, J. (1993). Collaborative, computer-based learning: Cognitive and affec-tive outcomes. Journal of Educational Computing Research, 9(2), 149-163. Rheingold, H. (1983). Video games go to school. Psychology Today, 17, 37-46. Roschelle, J. (1992). What should collaborative technology be? A perspective from Dewey and situated learning. SIGCUE Outlook, 21(3), 39-42. Ross, J. A. and Raphael, D. (1990). Communication and problem solving achieve-ment in cooperative learning groups. Curriculum Studies, 22(2), 149-164. 196 Scardamalia, M . , Bereiter, C , McLean, R. S., Swallow, J., and Woodruff, E. (1989). Computer supported intentional learning environments. Journal of Ed-ucational Computing Research, 5, 51-68. Sedighian, K. (1996). Super tangrams: A child-centered approach to designing a computer supported mathematics learning environment. In International Conference on Learning Sciences '96, Evanston, IL, pp. 490-495. Sharan, S. (1980). Cooperative learning in small groups: Recent methods and effects on achievement, attitudes, and ethnic relations. Review of Educational Research, 50(2), 241-271. Sierra On-Line, Inc. (1993). The Even More Incredible Machine. Computer Soft-ware. Silvern, S. B. (1986). Classroom use of videogames. Educational Research Quar-tly, 10(1), 10-16. Simpson, J. (1986). Computers and collaborative work among students. Educa-tional Technology, 26(10), 37-44. Slavin, R. E. (1980). Cooperative learning. Review of Educational Research, 50(2), 315-342. Slavin, R. E. (1984). Students motivating students to excel: Cooperative incen-tives, cooperative tasks, and student achievement. The Elementary School Journal, 85(1), 53-64. Slavin, R. E. (1990). Cooperative Learning: Theory, Research, and Practice. En-glewood Cliffs, NJ: Prentice Hall. Smith, R. B. (1987). Experiences with the alternate reality kit: An example of the tension between literalism and magic. IEEE Computer Graphics and Applications, 7(9), 42-50. Software Publishers Association (1995). Report on the Effectiveness of Technol-ogy in Schools, 95-96: Executive Summary. [News Release Posted on the World Wide Web]. Retrieved June 22, 1997 from the World Wide Web: http: //www.spa.org/project / edu.pub / pbtop.htm. Steiner, K. E. and Moher, T. G. (1994). A comparison of verbal interaction in literal and virtual shared learning environments. In CHI '9\: Human Factors in Computing Systems, Boston, MA, pp. 97-98. Conference Companion. Stewart, J. E. (1997). Single display groupware. In CHI '97: Human Factors in Computing Systems, Atlanta, GA, pp. 71-72. Conference Companion. 197 Strommen, E. (1994). Children's use of mouse-based interfaces to control virtual travel. In CHI '94: Human Factors in Computing Systems, Boston, MA, pp. 405-410. Strommen, E. F. (1993). "Does yours eat leaves?" Cooperative learning in an ed-ucational software task. Journal of Computing in Childhood Education, ^(1), 45-56. Trowbridge, D. (1987). An investigation of groups working at the computer. In Applications of cognitive psychology: problem solving, education, and comput-ing. Hillsdale, NJ: Lawrence Erlbaum Associates. Upitis, R. and Koch, C. (1996). Is equal computer time fair for girls? Poten-tial internet inequities. In INET '96: 6th Annual Conference of the Internet Society, Montreal, Quebec. Watson, J. (1991). Cooperative learning and computers: One way to address student differences. The Computing Teacher, 18(A), 9-15. Webb, N. M . (1984). Microcomputer learning in small groups: Cognitive require-ments and group processes. Journal of Educational Psychology, 76(6), 1076-1088. Weiser, M . (1991). The computer for the 21st century. Scientific American, 265, 94-104. Welford, A. T. (1968). The Fundamentals of Skill. London: Methuen. Wilder, G., Mackie, D., and Cooper, J. (1985). Gender and computers: Two surveys of computer-related attitudes. Sex Roles, 13(3-A), 215-228. Yager, S., Johnson, D. W., and Johnson, R. T. (1985). Oral discussion, group-to-individual transfer, and achievement in cooperative learning groups. Journal of Educational Psychology, 77(1), 60-66. 198 Appendix A A . l Sample Questionnaire - Kerrisdale 1994 QUESTIONNAIRE N a n u M • Age Note: Electronic games include both video and computer games. Stftxifi) 5;r-ng!;. ' • •.-••< Diva$r« i K . v r t '•>>'% 1. I love playing electronic games with my ! 2 3 5 friends. 2. Two piayer electronic games are no fun because you always have to sit and wait for your turn 0 • 3. I like talking white I play electronic 1 2 3 4 (T^ games. 4. I wouJd rather play electronic games f I } 2 3 4 5 like The Incredible Machine by myself. 5. Having people around while f play 2 3 4 5 electronic games makes me nervous. 6. Electronic games like The Incredible 1 2 3 /4 I EfflK Machine are easier to play with a ^25^ partner. 7. I only like playing electronic games by myself. 8. Electronic games are boring when playing by yourself. 9. 1 don't like having to share my electronic games with my fneads. 10. I like electronic games where my friends and I can work on it together. 11. 1 think you can icarn more in an electronic game when you play by yourself. 12. I like having a friend around when I play electronic games to help rnc get through the hard parts. 0 1 1 2 0 • 1 2 • 9 0 & 199 Page2 13. Would you have liked to play The incredible Machine longer? 14. Who would you rather play computer or video games with: 15. How do you prefer to play computer and video games' © Girl? Bovs Either Neither Byfcfa Vithjp No Preference 16. How often do you play video or computer games: Ne_* ~ _ C^ A few times a Yeary A few times a Month A few times a Week Almost Every Day 17, How often do you use a computer: Never ,--AJewjimesjn_r 'A few times u Month) SWrrTffitcs a WccIT Almost Every Day 18. Circle all the things you have at home: 19. Would you participate in this study again if it was held outside of school time? ; Comp siintendo^ Super Nintendo Entertainment System Sega Genesis 20. What did you like best about Tlie ~ incredible Machine* W * ^ ' U rt •>r>f".v/ c' w f <V<f; 21. What did vou like least about The . Incredible Machine! S^rtve TUtt<^> X CJOIJ [fjrt T S f r C f k « o.rfatv- 'Par- Vk^- sfWrttL, 200 Appendix B B . l Sample Video Transcript - Kerrisdale 1994 Day 7A #1 Integrated Play Female/Female Start Time: 9:07 Nicki, white sweatshirt on rig h t . Mary, white T-shirt on l e f t . The two remained t o t a l l y focused on the game throughout the t h i r t y minutes. Mary dominated the mouse for the entire period. 9:07 Mary takes the mouse. Mary: "Do you know how to use the mouse?" Nicki: "Yeah, of course!" 9:08 Mary puts the mouse down, the two begin to look at the manual. Shortly after, Mary takes the mouse again. 9:09 Nicki p u l l s the mouse away from Mary. 201 Seconds later, Mary prys Nicki's hand off the mouse and starts using it herself. Nicki stops looking at the screen and starts looking at the manual. Soon after, Nicki tries and fails to pull the mouse out of Mary's hand. 9:10 Nicki starts looking at the manual again. 9:11 Nicki grabs the mouse from Mary. Before long, Mary has grabbed it back. 9:12 Nicki goes back to the manual. Mary: "This is hard." 9:13 Nicki grabs the mouse from Mary. A bit later, the two start looking at the manual together. 9:14 Mary starts using the mouse again. 9:15 Nicki makes a grab for the mouse, but Mary pushed her hand back. 9:18 Mary: "What's this? Oh, it's a l i t t le mouse to run the belt! Oh 202 that's so sweet!" 9:19 The two turn their attention from the game to the instruction manual. When they go back to the game, Nicki takes control of the mouse while Mary continues to look at the instruction book. Soon after, Mary pulls Nicki's hand off the mouse and starts to use it herself. 9:20 Mary leaves the mouse and starts looking at the instruction book. Nicki takes the mouse. A few seconds later, Mary grabs the mouse back. 9:21 Nicki tries and fails to pull the mouse from Mary. Mary has managed to position her body and left arm so that they separate Nicki from the mouse. 9:22 Nicki reaches with both arms for the mouse and is pushed back by Mary. 9:23 Saying "let me try", Nicki successfully grabs the mouse from Mary. Mary puts her hand on Nicki's and soon has control of the mouse. 9:24 Mary: " . . . you're going to tire the mouse to death!" 203 Nicki: "It's only a computer Mary." Nicki: "Let me use that thing." Nicki grabs the mouse. 9:26 Mary grabs the mouse. 9:28 The two applaud themselves, having just finished the first puzzle. 9:31 The two complete puzzle number two. Mary: "Yah! We did it!" A balloon pops. Mary: "0000! I hate that noise!" 9:33 The two complete the third puzzle. Mary & Nicki: "YESS!" 9:36 The two complete the fourth puzzle. 9:38 Bell rings, completing the session. 204
Thesis/Dissertation
eng
Computer Science
Vancouver : University of British Columbia Library
For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.
Graduate
Adapting the human-computer interface to support collaborative learning environments for children
Text