UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Supporting learners in a remote computer-supported collaborative learning environment: the importance… Graves, David 1997

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
831-ubc_1999-0042.pdf [ 7.64MB ]
Metadata
JSON: 831-1.0051306.json
JSON-LD: 831-1.0051306-ld.json
RDF/XML (Pretty): 831-1.0051306-rdf.xml
RDF/JSON: 831-1.0051306-rdf.json
Turtle: 831-1.0051306-turtle.txt
N-Triples: 831-1.0051306-rdf-ntriples.txt
Original Record: 831-1.0051306-source.json
Full Text
831-1.0051306-fulltext.txt
Citation
831-1.0051306.ris

Full Text

Supporting Learners in a Remote Computer-Supported Collaborative Learning Environment: The Importance of Task and Communication by David Graves B.Sc. (Honours, Psychology) University of New South Wales 1991 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF Master of Science in THE FACULTY OF GRADUATE STUDIES (Department of Computer Science) we accept this thesis as conforming to the required standard The University of British Columbia December 23, 1997 © David Graves, 1997 In presenting this thesis in partial fulfilment of the requirements for an advanced degree at the University of British Columbia, I agree that the Library shall make it freely available for reference and study. 1 further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for financial gain shall not be allowed without my written permission. Department of The University of British Columbia Vancouver, Canada DE-6 (2788) Abstract This paper describes novel research in the area of remote Computer-Supported Collabora-tive Learning. A multimedia activity (Builder) was designed to allow a pair of players to build a house together, each working from his or her own computer. Features of the activity include: interactive graphical interface, two- and three-dimensional views, sound feedback, and real-time written and spoken communication. Mathematical concepts, including area, perimeter, volume, and tiling of surfaces, are embedded in the task.' Afield study with 134 elementary school children was undertaken to assess the learning and collaborative potential of the activity. Specifically, the study addressed how different modes of communication and different task directives affected learning, interpersonal attitudes, and the perceived value and enjoyment of the task. It was found that playing led to academic gains in the target math areas, and that the nature of how the task was specified had a significant impact on the size of the gains. The mode of communication was found to affect attitudes toward the game and toward the player's partner. Gender differences were found in attitude toward the game, perceived collaboration and attitude toward partner. Contents Abstract ii Contents iii List of Tables vii List of Figures viii 1 Introduction 1 1.1 The Game Builder- A Preview 1 1.2 Motivations 2 1.3 The Learning Activity 5 1.4 Research Questions and Structure of Approach 5 2 Research Context 8 2.1 Cooperative Learning 8 2.2 Computer-Aided Learning 12 2.3 .Computer-Supported Collaborative Learning (CSCL) 14 2.4 Computer Games 22 2.5 Gender Issues 24 3 Situating the current research 26 3.1 Research Focus 26 3.2 The Learning Setting '. 27 3.3 Variables Investigated 30 iii 3.4 Outcomes Measured 32 4 Builder 34 4.1 Overview 34 4.2 The Task 35 4.3 Communication 38 4.4 . Sequence of Play and Interface 39 4.4.1 "Challenge-Selection" (Screenshot 1) 39 4.4.2 "Challenge-Info" (Screenshot 2) 40 4.4.3 "Top-View" (Screenshot 3) 41 4.4.4 "Side-View" (Screenshot 4) 44 4.4.5 "3D-View" (Screenshot 5) . . 46 4.5 The Client-Server Structure 46 5 Tools for Assessing Outcomes 48 5.1 Academic Measures •..:48 5.1.1 Target Areas >. • 48 5.1.2 Tests : " 49 5.2 Socio-motivational Measures 49 5.3 Log Files 50 5.4 Observations 51 6 Pilot Studies 52 6.1 Description of Pilot Study 1 53 6.1.1 Goals 53 6.1.2 Procedure 54 6.1.3 Results of Software Testing 55 6.1.4 Assessment of the Interface 56 6.1.5 Findings Regarding Communication and Gender 59 6.1.6 Difficulty and Engagement 60 6.1.7 Findings Regarding the Test Materials 61 iv 6.1.8 Procedure-related Findings 61 6.2 Other Pilot Studies 63 6.2.1 The "Unintuitive" Interface (Pilot 3) 63 7 Study Design and Methodology 66 7.1 Design 66 7.2 Participants 67 7.3 Materials 68 7.4 Procedure 68 8 Results 71 8.1 Achievement Outcomes 72 8.1.1 Results on Academic Tests 73 8.1.2 Game Performance 75 8.1.3 Relationship Between Game Performance and Academic Gain . . . . 77 8.2 Sociomotivational Outcomes 79 8.3 Game-play 84 8.4 Reported Home Computer Use 86 9 Discussion 89 9.1 Summary of Findings 89 9.1.1 Achievement 89 9.1.2 Attitudes 90 9.2 Relating Findings to the Literature 91 9.2.1 Remote Collaborative Learning?. . . . 91 9.2.2 The Role of the Task 93 9.2.3 Supporting Interaction 94 9.2.4 Gender Differences 95 9.3 Future Research 96 Appendix A Implementation Details 99 v Appendix B Client-Server Messages 100 Appendix C Academic Tests 102 C l Pre-test 102 C.2 Post-test 102 Appendix D Questionnaire 103 Appendix E Observation Form 105 Appendix F Builder Screenshots 106 Bibliography 107 v i List of Tables 8.1 Academic improvement across three independent variables 74 8.2 Number of challenges completed across SEX and C O M M for SGM players . 76 8.3 Number of challenges completed across SEX and C O M M for M G M players 76 8.4 Challenge score, improvement and pre-test as a function of number of chal-lenges completed for SGM 77 8.5 Challenge score, improvement and pre-test as a function of number of chal-lenges completed for M G M . . . 77 8.6 Factor loadings and means of questionnaire items 80 8.7 Means for Factor 1 (Perception of Collaboration) showing SEX*COMM in-teraction 82 8.8 Means for Factor 2 showing SEX*GOAL interaction 83 8.9 Means for Factor 3 showing GOAL*COMM interaction 84 8.10 Written message count across SEX, GOAL and C O M M 84 8.11 Volume of spoken communication across SEX and GOAL 84 8.12 Uses of computer across gender 88 vii List of Figures 1.1 Changing the size of a wall in Builder 1 4.1 Three examples (a,b,c) of room layouts 37 4.2 Corner of a room that does not enclose an area 44 viii Chapter 1 Introduction This thesis presents the results of a study about the effectiveness of a multiuser computer act ivi ty for mathematics learning in the intermediate grades. In particular, the study looked at the impact of different modes of communication and different types of tasks on learning and att i tude in same-sex pairs. The thesis begins with a discussion of the motivations behind the elements investigated in the study, followed by an overview of the structure of the thesis. A s a prelude to the discussion of motivations, a brief introduction to the act ivi ty that was used in the study is given below. 1.1 The Game Builder — A Preview-Figure 1.1: Changing the size of a wall in Builder 1 The act ivi ty Builder was designed and implemented as part of the work contr ibut ing to this thesis. Builder consists of a series of challenges, each of which asks players to construct a house according to specifications relating to floor area and volume (e.g., " B u i l d a house wi th a floor area of 80 square units") . It is designed for two players, who work together in a shared graphical v i r tua l space to achieve their goal. Players see the results of each others' actions (such as building a new wall) , and can communicate wi th each other either by spoken or wri t ten messages (depending on the mode of the game they are playing). In Figure 1.1 we see that Dave is in the process of changing the size of a wall (Screenshot 3 in Append ix F shows an enlarged, colour version of Figure 1.1). The max imum possible perimeter of the house is l imited by the fixed number of bricks allocated for each challenge. Players can add windows and doors to their walls, which frees bricks according to how much area the window or door covers. A s wi th bricks, the number of windows and doors is l imited by the set of frame pieces allocated for the challenge. The goal in the design of the act ivi ty was to teach mathematical concepts relating to area, perimeter, volume, and t i l ing of surfaces. Chapter 4 gives a complete description of Builder. 1.2 Motivations Tradi t ional Distance Educat ion ( D E ) allows geographically-separated or non-mobile stu-dents to take centrally-administered courses. W i t h the increasing use of computers in D E , such as W W W - b a s e d projects like W e b C T [Gol96], the t radi t ional model has been enhanced in several ways. The learning material , for example, can be easily distr ibuted and updated from the central location, alleviating the inconvenience of postal delays and retrieving ma-terial from libraries. Furthermore, in contrast to the regular text materials used in D E , internet-based systems allow material to be of a mul t imedia nature, including video and sound, and can also include interactive elements providing immediate feedback, such as on-line quizzes [Gol96]. Addi t ional ly , these W W W - b a s e d facilities allow communicat ion between students and instructors v ia bulletin boards, email and sometimes real-time chat facilities. The dual roles of the computer in DE Reflecting on the qualities brought to D E by computers suggests there are two types of func-tions involved. O n the one hand we see computers used in the organization and presentation of content, while on the other they provide a medium of communicat ion between those in-volved in the learning. The central questions of this study, which is of an exploratory nature due to the lack of previous research in the field, are prompted by consideration of these two roles. Current computer-based D E courses show significant benefits over the t radi t ional methods, and also employ technology just wi th in the capacity of a typical home computer system. 1 However, as advances in technology allow progressively greater functionality and flexibility in networked computer environments, so the potential role for computers broad-ens wi th in the D E setting. Supporting communication ' > One of the pr imary issues addressed in this study was the influence of the type of com-munication on the quality and experience of learning in a remote Computer-Supported Col laborat ive Learning ( C S C L ) activity. A s stated above, in D E computers may provide asynchronous communication in the form of email and bulletin boards [Lev92, Rie92], or synchronous communication in the form of real-time chat [Gol96]. These channels of com-municat ion suffer the restriction that they are composed purely of A S C I I characters, so a user is unable to draw pictures, for example, to describe what s/he is th inking . Such l im-ited forms of communication may provide supplementary help to students taking a course, but i f the goal is for interaction to play a more central role in learning, a richer type of communicat ion may be required. T w o fields of research bear upon this issue. The most important of these is Cooperat ive Learning ( C L ) , since the positive academic and social re-sults in the C L literature [ J M J + 8 1 , J J 8 3 , e.g.] provide the incentive to bring collaboration to the D E setting. Secondly, we can consider extensions to D E in light of the parallel field of Computer-Supported Cooperat ive Work ( C S C W ) , where the goal is to enable a group of remote users to work together on a project in a v i r tua l shared space. M a n y C S C W *It would be unreasonable, for example, to include a real-time video-conferencing capability in a system intended for use by students with a modem connection. 3 projects have developed technology to simulate in-person communicat ion through the use of audio and video channels [IM91, e.g.], based on the assumption that enriched channels of communicat ion wi l l provide stronger support for the cooperative activi ty. In this study simple wri t ten communicat ion was compared wi th a richer form of communicat ion, which included wri t ten and spoken messages, as well as a graphical representation of the other user ("vir tual presence" 2 ) . Designing content The second pr imary issue addressed in the study was the influence of the nature of the task on the learning experience. In computer-supported D E , we have said that the computer is used to present and organize teaching material . L imi ted interaction and feedback may also be provided in the form of automatical ly-marked on-line questions that ask rote-style, mem-ory questions about the material . Given that it has been typical for tertiary-level teaching to be concerned more wi th the content of the material than how it is taught, this type of feedback may be considered appropriate for the university-level students at which'projects such a s W e b C T are aimed. For the current study, aimed at elementary-level students, the role of the computer was extended to allow learners to explore wi th manipulables in a task in which the learning material was embedded, rather than presented. A g a i n , relevant work in the fields of Cooperative Learning and Compute r -Aided Learning ( C A L ) , which is concerned wi th how to effectively use computers as educational tools, provides motivations and suggestions. For example, researchers in C L , and Computer-Supported Col laborat ive Learning ( C S C L ) , have placed cr i t ical importance on the nature of the task, finding that rote-style tasks are not as effective as less-structured tasks wi th in a collaborative learning setting. In the current study, the role of the task in C S C L was investigated by manipulat ing the specification of the goal in the activity. 2Also referred to as "telepresence" in the CSCW literature. 1.3 The Learning Activity The specific instructional setting used in this study was an interactive, mul t imedia problem-solving act ivi ty (Builder), where two learners worked together over a local network from physically-separated computers, communicat ing wi th wri t ten and spoken messages over a network. Mathemat ica l constructs involving area and volume were embedded in the familiar real-world task of building a house. Given the embedded nature of the task and that the act ivi ty employed an interactive graphical interface wi th sound effects, the act ivi ty was game-like in nature. Th is was reinforced by having clearly-specified challenges wi th scores, and keeping records of high scores between different sessions. The design of such a game involves several Human-Computer Interaction (HCI) considerations, some of which are common to all C A L applications, while some are specific to remote C S C L . In the former case there is the question of what special demands a game or an educational act ivi ty places on a computer interface, wi th the added issue of age-appropriateness. For the latter case, as in the field of C S C W , the demands relate to how an interface should present shared'and private work, and how identity, behaviour and communicat ion of remote learners should be represented: The use of electronic games in learning, either of an individual or group nature, is a relatively new endeavour, and has been the subject of research of the E - G E M S (Electronic Games in Educat ion for Mathemat ics and Science) group at U B C . E - G E M S ' findings have shown potential both for effectively-presented C A L content promoting learning [SK96b], and for successful co-present collaborative play [ IBK95] . In these and other electronic game studies, significant gender differences have been found, so in addit ion to the pr imary questions concerning task and communicat ion, gender was also considered as an independent variable in the current study. 1.4 Research Questions and Structure of Approach To summarize the discussion above, the motivat ing question behind this study was: Can the positive outcomes of Cooperative Learning also be achieved in a remote CSCL environment? 5 The research questions addressed in the study were: • W h a t type of communicat ion is necessary to support remote collaborative learning? • How does the nature of the task affect the learning experience? • A r e the effects of communicat ion and nature of task the same for male and female learners? Other research A s discussed above with regard to the motivations of the study, several fields of research bear upon the questions addressed in this study. The fact that so many different fields are relevant is part ly due to the mult idiscipl inary nature of the field of C S C L , and part ly because there is so litt le literature dealing wi th the specific C S C L model investigated in this study. This lack of research is evident in a recent extensive literature review on educational multiplayer computer games which, despite being 8 8 pages long, spends only' three pages describing the literature on educational multiplayer games [McG96] . Even removing the game restriction, there is relatively li t t le research in the area of remote C S C L , : most C S C L studies being in the co-present domain. Therefore, the literature review touches upon many different but sometimes overlapping fields, drawing from them the most pertinent issues for this study. The review of other research (Chapter 2), begins wi th the discussion of Coopera-tive Learning ( C L ) , C A L and C S C L . It is appropriate to look at these fields together, as C S C L is essentially a marriage of C A L and C L . C L provides a good start ing point because it is a well-established field, and many of the motivations, expectations, and assessment techniques, both for C S C L in general and this study in particular, are to be found there. The C A L section introduces the topic of the general use of computers in education. 3 The discussion of C S C L begins by defining four different models wi th in the C S C L field. M u c h of the work has been done wi th co-learners in the same room, either at one or more than one computer, which is a substantially different environment to that of the current study. These studies have dominated the field for several reasons: they are easier to do; up unti l 3For the purposes of this paper, the term CAL includes similar labels that have been applied to the use of computers in education, e.g. Computer-Based Instruction (CBI), Computer-Assisted Instruction (CAI). 6 recently the available technology was too l imited to support collaboration in rich v i r tua l learning environments; and researchers have impl ic i t ly or explici t ly believed that the nec-essary interaction for C L is not supportable remotely. Since this study uses the setting of a multiplayer game to investigate collaborative learning, the discussion of C S C L is followed by a section specifically dealing with educa-tional games. A l though educational computer games are a sub-category of C A L , they are worth considering separately to evaluate their relative efficacy in instructional settings, and because they entail a somewhat different set of design guidelines, which should be addressed in relation to the Builder activity. A substantial proport ion of the design of educational computer applications (the proportion not directly concerned with the content of the learn-ing material , though the content itself should greatly influence design) is the interface with the user, which falls into the field of Human-Compute r Interaction ( H C I ) . The literature review addresses H C I wi thin the discussions of C A L , C S C L and games, which are all fields that involve H C I issues. The final section considers gender differences, which are.a s tr iking feature of many studies in the computer game and C A L literature. The goal in addressing gender differences and other learner characteristics is to ascertain whether the effect of different aspects of the learning environment depends on learner characteristics, which has implications for design. The current study Following the literature review, Chapter 3 summarizes the relevance of the previous re-search to the current study. This includes discussion of the decisions made in the design of the Builder activity, and suggestions for hypotheses regarding the manipulations of com-munication and task structure based on the literature review. The remainder of the thesis presents the current research project, beginning wi th a description of Builder (Chapter 4) and the assessment tools designed for use in the study (Chapter 5), followed by an account of the pilot studies conducted prior to the study itself, which led to modifications to the act ivi ty and the planned study procedure (Chapter 6). The study itself and its implications are described in the final three sections: M e t h o d (Chapter 7), Results (Chapter 8), and Discussion (Chapter 9). 7 Chapter 2 Research Context M u c h of the promise and excitement of Computer-Supported Col laborat ive Learning ( C S C L ) rests wi th the fact that it combines Cooperat ive Learning wi th Computer-Assis ted Learning ( C A L ) , two fields which each bring their own different advantages to C S C L . The descrip-tion of related research wi l l begin wi th a brief discussion of these two fields and how they come together in C S C L . Fol lowing this discussion, other specific research areas of relevance for the current study wi l l be described, specifically computer games and computer-related gender issues. 2.1 Cooperative Learning Roots of Cooperative Learning Cooperat ive Learning is not a new idea, being wri t ten about as early as the 1920's; but studies that have evaluated its potential as an educational model have mostly taken place in the last 20 years [Sla80]. Researchers in the field draw upon theoretical perspectives from educational philosophy and social psychology to stimulate and guide experimental work. A l l p o r t ' s work on the positive effect of interaction on race relations, for example, provides motivat ion for the application of C L in school classrooms, wi th the goal of alleviating preju-dice amongst heterogeneous learners [HH91]. The suggestion that it may also be an effective approach to learning in general emerges out of more philosophical theories such as John Dewey's s i tuated learning, in which he saw learning emerging out of purposeful commu-8 nal act ivi ty and transactions wi th others [Ros92]. Th is is compatible wi th the modern view of constructivism in which knowledge is seen as emerging out of sociocultural interaction rather than something to be revealed or t ransmit ted [ S B M + 8 9 ] . It should not be implied that these theories reflect the consensus views in the educational psychology literature, as some writers have argued that situated learning and constructivism are academically inef-ficient and incomplete [ARS97]. Nevertheless, while it is true that collaborative learning models may never entirely replace the more t radi t ional methods of instruct ion, the positive outcomes in the field suggest that they may play an important role, both academically and socially, in a complete education system. What is Cooperative Learning? A central element in the definition of C L is the reward structure. The typical model of educationMs competit ive, wi th evaluation ranking students on a scale of better-to-worse. The continuing dominant presence of competitive-style education in the classroom, despite the grand claims emanating out of the C L literature, is supported by the common-sense belief that competitive-style education is necessary if learners want to be successful in the "real wor ld" [HZD93]. The reward structure in such competitive systems has been labelled negative reward interdependence, because one student's success depends on the failure of others [Sla80]. Cooperat ive Learning researchers such as Slavin and Johnson and Johnson have conducted extensive studies on the differences between competit ive, cooperative and individual is t ic goal structures [JJ74, Sla90, e.g.]. In contrast to competi t ive goal structures, cooperative goal structures are defined as "a si tuation in which the only way group members can attain their own personal goals is i f the group is successful" [Sla90, p. 13]. Cooperat ive Learning may or may not involve intergroup competi t ion. The individual is t ic paradigm is that in which reward does not depend on others at a l l . The work of Slavin and Johnson and Johnson indicates that positive reward interdependence is one of a set of necessary factors for C L . The other factors suggested include: face-to-face interaction, individual accountability, social skills training, and group evaluation opportunities [HZD93]. What's so good about Cooperative Learning? 9 A s mentioned above, one of the early goals in C L research was the breaking down of cross-racial prejudice. Several early studies showed positive effects of C L on interpersonal re-lationships [JJ83, Sha80]. Johnson and Johnson, for example, found that C L led to an increase in positive atti tude towards peers and a higher value placed on working wi th het-erogeneous peers [JJ83]. Th i s effect was observed not only in inter-racial groups but also in cross-sex and cross-handicap relationships [JJSR85] . O n the basis of such findings, C L has been recommended for classrooms where there is mainstreaming of students wi th disabil i-ties [Mal86]. In addit ion to the positive social findings, many studies have also found positive effects on achievement [ JMJ+81 , ST79 , e.g.]. 1 For example, a meta-analysis of 122 studies by [ J M J + 8 1 ] found that cooperation and cooperation with intergroup competi t ion led to greater academic gain than either interpersonal competi t ion or individual is t ic goal struc-tures. There have, however, been conflicting findings in the achievement literature, wi th some findings of competi t ion being more successful [Mic77, e.g.], and debate over the exact conditions in which Cooperat ive Learning is beneficial. M o r e refined research has suggested that successful C L depends on a number of factors. 1 * What factors are most important in Cooperative Learning? In several studies Webb has looked at the effect of interaction on achievement, finding that only certain types of interaction, such as giving an elaborate answer to a peer, are pos-itively related to achievement [Web82b, Web82c]. In response to the debate over which type of learners and what type of group composit ion profit most from C L , Webb's find-ings suggest that the conflicting data may depend on how often such groups or learners engage in this type of creative, task-related interaction. Homogeneous groups, for example, which have been shown in some studies to benefit less from C L , show less of this type of interaction, while heterogeneous groups and extroverted learners show more. In contrast to Webb's results, Cohen and her associates have found that simple frequency of interac-tion predicted achievement if the problem was inherently a group task and the task was il l-structured [Coh94]. A n il l-structured task is one which has no single right answer. Neg-1 Achievement is meant in terms of task-related learning, most commonly measured by pre- and post-tests on the material being covered. 10 ative support for Cohen's argument is found in [ J M J + 8 1 ] ' s meta-analysis of C L findings, which found that rote-learning tasks had less effect than other types of task on achievement measures. Despite the apparent conflict in the literature, Webb and Cohen's arguments may be compatible in that i l l-structured tasks could lead to co-learners exchanging more explanations. The importance of explanation in achievement results helps i l luminate the cognitive processes which make C L effective. In some of Webb's studies, for example, it was found that high-level learners profit more in heterogeneous groups than they do in homogeneous groups. This is hypothesized to be due to there being more of a need for explanation between the high and low-level learners, and that this explanation involves rehearsal and cognitive restructuring in the explainer. Low-level learners also profit more in heterogeneous groups, perhaps because they find it easy to understand the explanations of the higher-level learners. Middle-level learners, on the other hand, profit more in homogeneous groups, which Cohen suggests may be because they don' t get a chance to explain when there are higher-level learners in the.group [Coh94]. 1 ' v. • I The notion of an "inherent group task" has been studied mainly in terms of giving group members specific roles, or l imited tools or knowledge that must be used in combination to solve a problem. Repman, for example, found that groups using roles scored higher on achievement than those without roles, though there was no parallel positive difference shown in sociomotivational results [Rep93]. To summarize the findings of what makes C L successful (in addit ion to those factors previously defined as the "necessary" components of C L ) the literature proposes 2 use of the following ingredients: • high levels of interaction, part icularly task-related, explanatory and conceptual in-teraction; • training in cooperative relationships; • small groups - two or three members work best [JMJ+81]; • enforcing the group nature of the task by the assignment of responsibilities, roles or divided resources; 2Not unanimously, of course. 11 • problems that are i l l - s t ruc tured . 2.2 Computer-Aided Learning In a meta-analysis of 51 studies on computer-based teaching, [ K B W 8 3 ] found that computer-based groups did better overall than conventional learning groups according to academic measures, and that use of the computer substantially reduced learning time. In addition to academic gains, classroom computer activities have been found to lead natural ly (i.e., wi th -out teacher intervention) to higher levels of student interaction compared to non-computer activities, such as solving a jigsaw puzzle [ H S G B 8 2 , N C 9 3 , JJ86] . Several researchers have observed that students wi l l naturally spend more time, even sometimes outside normal class hours, working on computer tasks than on non-computer tasks [Sol91, Cle81]. Th is suggests that there may be something intrinsically motivat ing or engaging about using computers. There is much disagreement in the literature, however, over the true value of C A L . The research findings have not been universally positive [ W M 9 4 , e.g.], and the record wi th in actual classrooms has not been as powerful as some of the effects found in the research literature [Rob94]. There are also social concerns that C A L might lead to less interaction wi th teachers and classmates and hence not allow for sufficient social modelling or building of social skills and healthy social attitudes [JJ86, H D H 8 7 ] . Proponents of C A L , however, claim that the problem resides in the nature of the C A L applications that have been used and the attitudes towards their use. Hannafin et al [HDH87] argue that the resistance to computers in schools is based on a misunderstanding of the true potential of C A L . O n the basis of the studies cited above finding computers led to increased classroom interaction, they propose that the image of the isolated computer user is a myth . Furthermore, they argue that existing commercial C A L applications support the myth that computers are appropriate only for learning low-level skills. Such applications are criticized for emphasizing flashy graphics and sound, while consisting purely of repetitive dr i l l ing on rote-style questions [Pap93]. Instead, Hannafin et al argue that C A L has the potential to provide valuable aid in areas that t radi t ional learning cannot provide. The roles suggested include: 12 • use in one-on-one instruction with a flexible level of difficulty, allowing students to proceed at their own pace; • use in simulations of scientific phenomena that are impossible to demonstrate in the classroom; and • use in visualization and manipulat ion of large sources of da ta or other information. Papert and others have also proposed the use of computers as tools for creation, rather than as fixed question-askers. One example of this is is a study by Soloway where students used computers to create their own mul t imedia packages on a chosen topic [Sol91]. Th is study led to increased attendance and part icipation by "problem" students. Another example, described below in the section on C S C L , is the series of Logo studies by Nastasi and Clements in which students were engaged in computer programming exercises. ; The use of computers in education includes computer games, which are discussed separately in Section 2.4. HCI concerns in CAL The field of H C I is concerned wi th the usability of software, wi th the typical goal being to design interfaces that are "transparent" to the user. Th is t radi t ional approach to H C I is centered around the expert user, aimed at improving product ivi ty in computer-based work. M o r e recent thinking in H C I has identified the need for different types of interfaces for dif-ferent users and applications [Car87, e.g.]. For example, when novice users are learning to use an application, the demands on the interface are entirely different than those arising in expert use [TM94] . To support the learning process for novice users, researchers in the field have employed techniques such as scaffolding, in which the full functionality of a system is only gradually revealed, rather than confusing the user wi th a huge set of unfamiliar functions from the outset [Sol93]. Th i s technique can also be applied to C A L applications, where there is a need to guide learners through the system, at least in the early stages of use. A s more is learnt, and the act ivi ty becomes more difficult, more tools could become available within the interface. 3 3This is potentially confusing, since scaffolding in some applications may consist of removing tools so that the user has to think more. The technique of progressively adding tools might be used when training users on complicated business applications, for example. 13 Some researchers, however, have questioned the relevance of "ease-of-use" and "trans-parency" wi th in an instructional setting. In C A L , as in electronic games, it is often de-sirable to have some element of challenge in the activity, which may be incorporated into the interface. Research conducted by K a m r a n Sedighian using the E - G E M S game Super Tangrams has investigated the role of the interface in learning transformational geome-t ry [SK96a, Sed97]. The design of the interface was based on the goal of promoting re-flection by intentionally avoiding direct manipulat ion of objects (as in "drag-and-drop") in favour of manipulat ion of formal representations of the transformations being used (e.g., the length and direction of a 2D vector). It was found that making learners focus more on the important properties of transformations led to significant academic improvement on pencil-and-paper post-tests [SK96b]. 2.3 Computer-Supported Collaborative Learning (CSCL) The use of computers in a collaborative educational setting is a natural progression from C L and C A L because: • we want to retain the benefits of C A L , such as its flexible level of difficulty, and usefulness in visualization; • use of computers appears to promote interaction, compared to other classroom activ-ities; • C S C L alleviates the concern of computer use encouraging isolation; • computers provide a means to bring collaborative learning to remote settings, such as in Distance Educat ion . Types of CSCL A s stated previously, much of the work done in C S C L is wi th in the co-present model, which is easier to study and also provides a natural way to allow communicat ion between co-learners which is appropriate in many learning settings. Co-present C S C L entails a substantially different set of concerns than remote C S C L , since interaction in the in-person model includes the language, facial expressions and gestures of everyday life (i.e., the same type of communicat ion as in C L ) . In addit ion to standard co-present C S C L , we identify 14 three other possible types of C S C L . The four types are distinguished by two factors: the type of communicat ion used, and the nature of the shared space. Accordingly, the types are defined as follows. • S tandard co-present C S C L : In this model, all learners sit at one computer, sharing one input device. We refer to this as a "l i teral" shared-space even though the shared-space is electronic rather than physical, because there is l i terally one screen that learners share. A vi r tua l shared-space is defined as one where multiple electronic environments (physically presented on different screens) are v i r tual ly integrated into a single work-space. 4 • M u l t i p l e - I n p u t C S C L : This entails use of one computer as in the standard model, but each learner uses her/his own input device. This model is typical in video games. Since learners are co-present, communicat ion is the same as in the standard model. The shared-space is st i l l l i teral , but may be more complicated than in the standard co-present model because users may be represented separately in the interface. The simplest example of this is the use of multiple cursors differentiated by shape or colour. In video games multiple users are generally represented by avatars. 5 • M u l t i p l e - C o m p u t e r C S C L : Learners work at a separate computers, but are all in the same room. Communica t ion is as in the standard model, but the shared-space is v i r tua l . • R e m o t e C S C L : In the remote model, both communicat ion and shared-space are computer-mediated. Before describing the studies that have investigated each of these C S C L types, two preliminary issues wi l l be addressed. F i r s t we consider the potential advantages and dis-advantages in the comparison between C S C L and non-computer C L , which vary according to the type of C S C L being considered. Second, we address H C I issues in designing C S C L applications, which likewise vary according to the C S C L model. 4 Another way to clarify this is to say that the "virtual" refers to the sharing rather than the space. 5 "Avatar" is the term used for a graphical representation, often human-like, of a user in an electronic environment. 15 Differences between CSCL and CL General differences that exist between all types of C S C L and C L stem from the addit ion of the computer into the learning setting. Since the computer is supplementary to the in -teractions going on between learners, and can be incorporated into the process as much or as l i t t le as desired, there are no clear disadvantages of C S C L compared to C L . The poten-t ia l advantages of C S C L overlap wi th the discussion of C A L above, such as the provision of variable levels of difficulty to accommodate learners of different levels. A n addit ional advantage claimed by some researchers on the basis of findings of students staying late or arr iving early to work on C S C L projects, is that C S C L instruction places more responsibil-ity for learning on the students [Bat92]. It has also been argued that C S C L ameliorates the problem identified in regular C L instruction of students straying onto off-task behaviours, because the computer can be used to set lesson pace [Dal90]. In multiple-computer or multiple-input C S C L , the computer can enhance the collab-oration between learners if the v i r tua l shared-space is well designed, because it allows equal access to all participants, the abil i ty to use flexible symbol representation, and easy integra-tion of individual work wi th shared work. In remote C S C L there is the disadvantage that users can not communicate wi th each other in the normal way. Given Goldman ' s [Gol92] findings that physical proximity was related to the highest level of conversation during a C L activity, this absence of physical contact could be detrimental to the learning experience. There may, however, be unexpected advantages to remote C S C L . Batson [Bat92], for ex-ample, has proposed that the absence of face-to-face contact, even without true anonymity, encourages free expression in interaction wi th classmates and teachers. It might also be hy-pothesized that, at least for younger learners, the novelty of communicat ing electronically makes the experience more appealing. HCI issues in CSCL The majori ty of co-present C S C L studies involve the pair or group si t t ing around one computer wi th one input device, meaning that the interface issues are similar to those in individual C A L . There is no need to consider how the presence of users should be indi-16 cated in the v i r tua l space, nor to control simultaneous modification of shared material . In multiple-computer and remote C S C L , special design concerns arise about how to display a workspace that is being modified, perhaps simultaneously, by other users. The remote C S C L designer has to deal wi th the addit ional problem of how to incorporate communi-cation between users into the interface. Educat ional thinkers have suggested that symbols and gestures are social tools for mutual learning [Ros92]. In co-present interaction, these gestures and symbols are drawn from the rich physical and social environment shared by the co-learners. The challenge in remote collaborative activities is to know what symbols to provide and how to best simulate gestures and other forms of communicat ion v ia electronic media. Representing the actions of remote users has been a central concern in C S C W re-search. In the most simple, command-line shared editing systems, the behaviour of other users may be indicated only by the appearance of the text they type. M o r e recent C S C W applications have added icons to represent each user and labelled cursors which can be used in gesturing [ G R W B 9 2 ] , wi th some of the more "high-tech" systems providing full video and audio linkage between participants [IM91, e.g.]. Ishii and other C S C W researchers have emphasized the importance of the shared view. The motivation for the application Clea rBoard , for example, which includes super-imposed video footage of the remote user, is allowing workers to follow the eye-movements of their colleagues as they discuss group work [IK92]. The notion of the shared view may also be cr i t ical in C S C L applications. G o l d -man's study of children interacting during a face-to-face collaborative learning exercise, for example, found that when learners were engaged in conceptual conversation they spent more t ime looking at the shared workspace (in this case a computer screen) than at their own work-sheets, a pattern which was reversed during more routine on-task conversation [Gol92]. Co-present CSCL studies Based on studies comparing cooperative-style learning using a computer game wi th compet-itive and individual learning styles, Johnson and his associates have proposed that C S C L has many of the same relative advantages of Cooperat ive Learning [JJ86, JJS86] . The coop-erative style led to higher achievement on a post-test and was characterized by conceptual 17 and task-oriented interaction. It was also found that those in the cooperative condition had more positive attitudes toward peers; e.g., males were more positive toward females [JJ86]. They argue that the agent of the effect is the positive mood produced by the C S C L activity, which in turn influences motivat ion and span of attention. A s was the case for C L , it is important to understand the exact conditions under which C S C L is most effective. A r e gains uniform across al l learner types? A r e some tasks part icularly suited to the C S C L teaching style? Some studies have addressed these questions, but as yet the variables are not understood as well as in C L . Da l ton and. associates [DHH89, HH91] , for example, studied the relative gains across learners of different academic level. They sought to explain the mixed results in the literature concerning gains made by high-level learners in heterogeneous groups, hypothesizing that the level-of-learner effect may depend upon content of learning material . They found some effects of content, and also that interaction was higher in heterogeneous than homogeneous groups, but they did not find the expected interaction between content and level-of-learner. In Webb's studies of C S C L she found that the key • variables identified in her C L studies were not as important wi th in the C S C L setting [Web84]. .. : A study of elementary-grade dyads by Nastasi and Clements [NC93] looked beyond the broad question of the effect of C S C L vs. n o n - C S C L and considered the effect of varying the situation or task in which the learners were engaged. The main dependent variable considered was motivat ion, measured according to behaviours displayed by children during the task (e.g., positive self-statements). They compared students working on Logo wi th students working on more dril l- l ike C A L activities. The Logo activities consisted of stu-dents choosing their own problems and defining simple computer programming procedures to achieve them. Logo led to more higher-order th inking, greater motivat ion and conflict resolution, and more cognitive change. Al though it was also found that Logo students experienced more failures than the other C A L groups, this d id not lead to more negative self-statements. W h a t appeared part icularly important was that the Logo task allowed stu-dents to develop divergent ideas, irrespective of whether or not this led to conflict. These results are consistent wi th Cohen's suggestion that i l l-structured tasks are more likely to promote the sort of cognitive act ivi ty that leads to effective learning. The data on feedback, 1 8 however, provided an exception to the other results. The reward system in the Logo act ivi ty was largely learner-regulated, which the researchers thought would be an effective way to motivate learners. Instead it was found that the more standard feedback, used in the non-Logo group, where the computer s imply announced player success, was more motivating. Multiple-Input CSCL studies Bricker and her associates at the Universi ty of Washington studied collaboration in a multiple-mouse C S C L act ivi ty involving colour-matching and chord-matching. In colour-matching, for example, three learners, each with their own mouse, were asked to manipulate R G B values to approximate a target colour. Some sense of roles was supported in the task in that each learner was responsible for one colour setting, hence success depended on the contributions of all participants. It is worth noting some of the rationale for Bricker et al's choice of the co-present C S C L model, as the points suggest challenges for implementations of remote C S C L . Fi rs t ly , they argue that "physically separating students to work individ-ually on computers tends to discourage communicat ion" [BTR+95 , p. 2]. 6 : Secondly, they question why it is that existing C S C W and remote C S C L applications fail to engage users as much as co-present video games. A s Bricker et al point out, the weaknesses in previous attempts at remote C S C L are part ly explained by l imitat ions in the existing technology. They also suggest, however, that collaboration is hampered because users are not sharing the same physical view. The challenges for remote C S C L are therefore to support com-munication sufficiently such that the physical separation does not discourage learners from communicat ing, and to provide a greater sense that the view of the workspace is shared by the whole group. The results of the Bricker studies were mixed. There was no difference in academic gain between cooperative and individual learning styles, nor on how much stu-dents enjoyed the collaborative nature of the task, wi th some participants saying that they would prefer to do the act ivi ty alone. Based on the C L literature it might be hypothesized that the decrease in conflict facilitated by having multiple input devices could lessen the effect on learning. It is possible, however, that the act ivi ty was not appropriate for C S C L since there was li t t le need for consultation among learners. 6It is not clear whether this is directed at remote CSCL or the standard single-user CAL model. 19 Multiple-mouse collaborative learning has also been investigated by K o r i Inkpen of the E - G E M S group using the educational game, The Incredible Machine [OL93]. Mot iva ted by ini t ia l findings that pairs working together on one computer wi th one mouse finished more puzzles than either individuals or pairs playing next to each other (but not playing the same game) [IBK95], learners' collaboration was further supported by the addition of an extra mouse. It was found that the use of multiple input devices had a positive effect on achievement, and further that the protocol used to exchange control between learners influenced success in the game [IBGK95]. Multiple-Computer CSCL studies The design of C S C L settings where learners are working on the same task from different machines requires consideration of several addit ional design concerns. These include: v iew control , representat ion of the user, conflict prevent ion and the suppor t -o f user roles [SM94]. Implementations of v iew cont ro l in C S C L and C S C W applications com-monly consist of a shared-space window, and sometimes include areas where the user may edit or view private material . Steiner and Moher ' s implemention of v iew cont ro l in a cre-ative wri t ing C S C L application followed the typical W Y S I W I S model, where " W h a t You See Is W h a t I See". Conf l ic t prevent ion, in Steiner and Moher ' s model, refers to the lock-ing of shared material while one worker is editing them, which is mostly an implementation issue but should also be indicated in the interface. Suppor t of user roles, which was not implemented in the creative wr i t ing application, is familiar from the discussion of the Cooperat ive Learning literature, and relates more to the task definition than the interface. A study on use of the creative wri t ing tool found that partners playing on separate computers in the same room communicated less than partners playing at a single machine. Steiner and Mohle r were actually interested in remote collaboration, but put partners in the same room to emulate full video and audio communicat ion. They took the results to indicate that even wi th perfect communicat ion support for learners, the physical separation of the users in a "vi r tua l ly" shared workspace fails to support collaboration as well as a "l i teral" shared workspace. Al ternat ive explanations for the results include the fact that the children in the study were very young (kindergarten), had had no previous exposure 20 to multiple-computer activities, and may not have understood the concept of the vi r tua l shared space. Furthermore, in Steiner and Moher ' s study there was no representat ion of the remote user to signify the other user in the vi r tua l environment. It could also be argued that mediating the communicat ion through the computer might allow learners to interact more easily while remaining focussed on the activity. Remote CSCL studies Scardamalia et a l . designed a computer-supported collaborative tool named C S I L E (Com-puter Supported Intentional Learning Environments) which allowed groups of learners to build and use a database of domain-related learning material - including assignments, reference materials, and notes between users expressing learning goals and open ques-tions [SBM+89]. The intention was to create a knowledge-building community, resting on the constructivist belief that knowledge is constructed rather than revealed, and that this construction emerges out of sociocultural interaction. Projects such as C S I L E can be distinguished from remote C S C L projects like Builder in that C S I L E is asynchronous rather than synchronous. In C S I L E the computer performs the roles of mult imedia bulletin-board and library, both of which are tasks that computers excel in . Given the fact that network speed is not an issue in asynchronous settings, the techniques of C S I L E can easily be applied to current Distance Educat ion endeavours [Uni97, e.g.]. Distance Education (DE) D E systems, as discussed in the Introduction, are also examples of remote C S C L . Develop-ers of these systems argue that networked computers provide a link between users leading to a community of learners, rather than the isolation in t radi t ional distance learning sys-tems [Gol96]. The real-time interaction available through chat facilities in such systems differs from more advanced collaborative systems such as those seen in C S C W [IK92, e.g.] because they are not integrated into a more general shared environment. It is not possi-ble, for example, to "point" at something, or to drag a diagram from a v i r tua l text-book into the shared-space. Furthermore, the content of a D E system generally does not involve group tasks, indicating that cooperation between learners is not a central concern. Riel 's 21 case study on Learning Circles [Rie92], however, was intended as a collaborative learning project, using the same asynchronous tools found in D E and C S I L E . Riel 's position is the antithesis of the more common approach that strives to simulate in-person collaboration in the remote setting. She argues that asynchronous email-based systems allow each learner temporal flexibility in presenting divergent ideas, in contrast to the time-pressure in syn-chronous collaboration, which may result in some learners being left out. 2.4 Computer Games Computer games have had a mixed reception in terms of both their social impact in gen-eral and their perceived potential in education. The active violence (as opposed to the passive violence in television) and gender stereotyping found in many popular games has been crit icized as psychologically harmful [Pro92], wi th some studies showing relationships between violent video games and violent behaviour [SW87, e.g.]. Provenzo has also ques-tioned whether electronic games are useful in educational settings given their "programmed" nature - i.e., they come constrained by the creator's world view [Pro92]. M a n y researchers in the fields of H C I and C A L , however, have looked at the impact of electronic games wi th great interest. Malone , one of the earliest computer game researchers, has suggested that the intrinsic motivation 7 s t imulated by games may be replicable in non-game applications [Mal82]. Even elements of computer games that appear contrary to what is desirable in a non-game application (such as challenge and difficulty to master) may be relevant because these factors are based on increasing levels of difficulty, which can be useful for user-tailored applications in any domain, and can contribute to sustaining user motivation and engagement. Malone 's investigations have attempted to establish what it is that makes computer games so motivat ing. He found that the presence of a goal was the most important factor in determining the populari ty of a game. This was followed by automatic scorekeeping, audio effects, randomness, and speed. M o r e formally, Malone ' s findings can be summarized into the following list of requirements for a motivat ing game. • C lea r goal: Th is should include performance feedback about how close the player is 7 A n intrinsically motivating activity is one which does not depend on reward from outside of itself. 22 to achieving the goal. • Uncertain outcome: This is often achieved v i a the use of variable difficulty levels and successive layers of complexity. • Fantasy: Does the game employ emotionally-appealing fantasies? Does it embody metaphors from physical or other systems that the user already understands? • Curiosity: Th is is achieved by providing sufficient informational complexity, includ-ing audio and visual effects, and elements of randomness. • Progressively-revealed information: Does the game introduce new information as players advance to higher levels? To this list other researchers have added: novelty, complexity, surprisingness, il lusion of control, goal formation, and competit ion [TM94]. Computer games in education Given the motivat ional aspect of games, it is tempting to explore their potential in edu-cation, especially for younger students. The findings of academic success of games have been mixed, but show enough promise to warrant further investigation. In a review of 68 studies of games in the classroom, for example, Randel reports that most studies found game-based learning equivalent to or better than t radi t ional learning according to academic measures [ R M W W 9 2 ] . Randel also suggests that games may be more useful in mathematics and science instruction than in the social sciences or arts, and that games may help wi th retention of rote-like material . A s might be expected, students were more positive about games than t radi t ional learning activities [ R M W W 9 2 ] . There are two comments to make about this comparison of games wi th t radi t ional learning. F i rs t ly , the quali ty of the game must be questioned. M a n y computer games are made by commercial companies where the main goal is to sell the product, and critics have argued that the l imited learning activities that they offer are generally "tacked on" to the game, rather than being an integrated part of it [Bro93]. Secondly, as [KP95] have suggested, it may not be appropriate to compare electronic games wi th t radi t ional learning. The easiest educational goal to achieve wi th elec-tronic games is s t imulat ing interest in the topic. In well-designed games, students may test hypotheses, develop problem-solving strategies and increase their understanding of complex 23 concepts [SK96b, e.g.], but the most effective use of computer games occurs when their use is supported wi th in a broader instructional environment. Multi-User Dungeons (MUDs) M U D s , which have appeared in countless forms on many internet servers since their incep-tion in 1980, represent one of the earliest attempts at multiplayer games. M U D s are based on fantasy and role-play, and were not originally designed to be educational, but recently some researchers have designed M U D - l i k e learning environments, seeing great potential for exploratory and creative educational activities [Res92]. A large part of the attraction of M U D s is the collaborative nature of the play, though until recently the domain has be-longed chiefly to computer-oriented males [KR93] . The l imited populari ty of M U D s may be due to the fact that most versions of the game are played entirely on the command-line. Interestingly, attempts to move M U D s into the interactive graphical domain have met wi th l imited success [MF91 , e.g.], but this may be due to current l imits in technology or the imagination of the creators, and interest appears to be steadily growing. The broader game Island wi thin which the Builder act ivi ty is set is an example of a graphical, educational M U D . Builder itself, which in the study was played without reference to the broader context of Island, is not typical of a M U D environment because the intention is for players to undertake specific tasks which begin and end wi th in one session of play. 2.5 Gender Issues The gender imbalance identified above in relation to M U D s has also been observed in other types of computer games and in the recreational use of computers in general. Surveys quoted in the popular media, for example, have estimated that 75% of video games are bought for boys [Per94]. Other studies referenced in the press have found that women tend to use computers for specific tasks and are more likely to ask for help using computer applications, while men are more likely to "muck around" wi th computers [Bul94]. Th i s a t t i tudinal difference is argued to impact on perception of mastery over computers and thereby proficiency in use [Bul94]. 24 Research on children's attitudes to electronic games has also found gender differences. In general, boys appear to be more interested in electronic games than girls, though this may be attr ibuted to a male bias in the design of games, many of which feature male-oriented themes of action and violence, and project stereotyped images of strong males and helpless females [Pro92]. The imbalance is self-perpetuating, since the at tract ion of young males to computers v ia these games helps provide the next generation of the software developers, thereby maintaining the male dominance of the industry. Studies have shown that the gender differences in att i tude towards computer games depend on the type of activity. Boys tend to enjoy competi t ive games where the aim is to get the highest score [ O K d V 9 6 ] , whereas girls prefer games that revolve around relationships [SU97], and are more likely to want to play games that involve collaborat ing with others [ I U K + 9 4 b ] . In the E - G E M S study on multiple-input C S C L cited above, gender differences were also found in collaborative game-play [ IBK95] . It was found that collaborative play had a greater effect on achievement in girls than boys, and that the effect of different interface styles depended on gender. Specifically, a "give" style, where one player passes control of the cursor to the other player, was more effective for girls playing together, while a "take" style, in which the user not currently in control of the cursor can take control , was more effective for boys [ IBK95] . Relatedly, in a study looking at control over speed of presentation of lesson material , Da l ton [Dal90] found that males preferred it when they were in control , while females preferred it when the speed was set. 25 Chapter 3 Situat ing the current research 3.1 Research Focus To reiterate the goals of the current study, the following broad issues were considered: • C a n the positive outcomes of Cooperat ive Learning also be achieved in a remote C S C L environment? • How can we best facilitate collaboration wi th in the remote C S C L setting? The prior question was addressed part ly by a comparison of academic tests between learners who used the C S C L activity, Builder, and a no-instruction control group, which allowed us to ascertain whether the act ivi ty led to any learning improvement. M o r e generally it was anticipated that the experience of designing the act ivi ty and observing students working wi th it would provide helpful insights on the potential for remote collaboration. In terms of the experimental design and what we can statistically conclude from the study's results, however, the study was focussed more on the latter question than the prior. There was, for example, no at tempt to compare the cooperative structure wi th a competit ive or indi-vidualist ic structure, nor was there any comparison between remote and co-present C S C L styles. The literature reviewed in the previous chapter provides several justifications for the approach taken. To begin wi th , based on the large number of studies comparing coopera-tive, competit ive and individualist ic learning styles [JMJ+81], we can consider cooperative learning established as an effective method of instruction, and turn our focus more to ex-26 actly how it is used. The literature also suggests that direct comparisons between learning styles, such as C S C L versus tradit ional classroom learning, may be ill-advised because of the mult i tude of additional factors in a C S C L environment that are difficult to control for, such as the effect of interactive graphics on enjoyment [Bat92]. Th i s view was echoed in [KP95] 's suggestion that it is inappropriate to compare games to other instruction because their intended use should be as a s t imulat ing supplement to other learning, perhaps rein-forcing material already learned or prompt ing interest in further investigation of a domain. Furthermore, recalling the introductory discussion regarding Distance Educat ion , it is in-appropriate to compare remote and co-present C S C L if the goal is to bring collaboration to a D E setting. Rather, assuming that learners do not have the option of in-person collabo-ration, how can we best support their collaboration remotely? This approach has practical applications because it generates potential guidelines for the development of D E systems. The literature review also provides justification for a study explici t ly concerned with remote collaborative learning in that very few previous studies have been performed. This suggests that the granularity of investigations in the area should be larger than that, for example, in a current study in the field of C L . We should also expect some difficulty in the interpretation of results due to the involvement of factors whose effects are as yet poorly understood (e.g., the effect of interactive graphics on motivat ion) . The study should therefore be seen as exploratory in nature, and used as a s tart ing point for later, more refined inquiries. 3.2 The Learning Setting The software developed for this study was a multiplayer, mul t imedia game, which allowed players to communicate using real-time spoken or wri t ten messages while performing the task of building a house. The game context was considered appropriate as a learning style for the target age group because of the intrinsic motivat ion games can provide [Mal82], and because of the success found in game studies by the E - G E M S group [SK96b, I B G K 9 5 ] . Pre-vious E - G E M S studies investigated individual games and single-computer multiplayer games where the students were co-present and working at a single computer [Sed97, I B G K 9 5 ] . The 27 current study expands on this work by turning to an investigation of multiple-computer, multiplayer games where players are physically separated. Previous research, particularly in the fields of C L and educational games, suggests many factors that should be considered in the design of the activity. Fi rs t ly , each of the key ingredients identified in the C L literature wi l l be addressed in relation to Builder. • Positive reward interdependence: A s discussed previously, positive reward struc-tures are those in which an individual 's success depends on the success of the group. In the Builder activity, two players share building materials and work simultaneously on the same house. Hence all actions of an individual affect the game status of her part-ner (i.e., the amount of remaining materials and how close the pair are to the goal). W i t h i n the game, scores are assigned to pairs, so there is no concept of individual success. 1 • Face-to-face interaction: A l l remote C S C L and C S C W depends on the assumption that face-to-face interaction is not essential i n . a li teral sense, but that it can be ap-proximated by the use of appropriate computer technology. This issue was addressed in the Builder study by manipulat ing the mode of communicat ion available between the two computers. A s described further below, one mode allowed only wri t ten com-munication, while the other provided a more enhanced form of communicat ion, which included speech. • Individual accountability: Th is was included in the Builder study through the inclusion of academic pre- and post-tests which students completed on an individual basis. Players were reminded before they started the act ivi ty that they would be si t t ing a post-test, and that their scores on the test would be individual . • Small groups: Builder is played in groups of two, a size that has been found to be effective in C L . • Roles and/or divided resources: Th i s is one element of C L that was not incor-porated into the Builder activity. We felt that it would be worthwhile to see whether 1 The fact that Builder keeps a record of high scores that are displayed at the beginning of each session suggests that, strictly speaking, the model used was that of cooperation with intergroup competition. There was, however, little emphasis placed on trying to beat the high scores. 28 or not, and how, learners collaborated in the absence of such techniques, which aim to structure and enforce the collaboration. The danger of omi t t ing roles is that one learner wi l l do all the work, and there is nothing to prevent this from happening in Builder. Th is problem might not, however, be as prevalent wi th in the game setting, since it is more likely both players wi l l want to contribute for the sake of their own en-joyment. It might also be hypothesized that a multiplayer, multiple-computer format provides greater potential for each user to contribute as much as they wish, rather than a si tuation in which only one learner can be in control of the collaborative piece of work at a t ime (as in single-input C S C L ) . A final point on this issue is that the vi r tual shared workspace of Builder makes it impossible for learners to keep their work separate, given that they work at a l l . • Social skills training and group evaluation opportunities: Tra in ing was incor-porated to a l imited degree in the study by instruct ing players how to communicate with each other and encouraging them to do so as much as possible. There was no scope in the study for more formal social skills t raining, nor was there an attempt to employ formal evaluation techniques. The need for the latter is partly alleviated by elements in the game setting, such as the presence of goals and previous high scores, according to which players can judge their performance. • Ill-structured tasks: Th is was addressed in the study by manipulat ing the type of task that pairs were assigned within the Builder activity. Some pairs worked on a task with a very clearly defined goal, while for others the goal was stated more generally. Th i s is described further under "Variables Investigated" (Section 3.3), and in the succeeding chapter describing Builder. Secondly we wi l l consider each of Malone ' s cri teria for a motivat ing game with respect to Builder. • Presence of a goal: A r e a and volume goals, wi th scores providing constant feedback on performance during play, are central to Builder. • Uncertain outcome: Malone suggests that uncertainty can be achieved through the use of variable levels of difficulty. Th is was incorporated in Builder by providing five 29 increasingly difficult challenges for players to attempt. • Fantasy: Builder's, use of the real-world metaphor of building a house is a good example of what Malone means by "fantasy". • Curiosity: Th is is related to the informational complexity of the game and is incor-porated in Builder through the use of interactive graphics and sound. The novelty of allowing players to view a 3-D representation of the house they have built also contributes to satisfying this cri terion. • Progressively-revealed information: Th is was not appropriate for the version of Builder used in the study since there was only a single 30-minute session of play. Final ly , to revisit the crit icisms in the literature of flashy, low-content educational games, the design of Builder balances entertainment and education by fully embedding the learning elements wi th in the central act ivi ty of the game. 3.3 Variables Investigated To better understand the role of some of the key factors in remote C S C L , two variables wi th in the Builder act ivi ty were manipulated in the present study. The first of these was the interaction between the users, which was addressed in terms of both the medium of communicat ion between the users, and also how they were represented in the vi r tua l space. Communica t ion between co-learners is thought to be the key to many of the gains seen in Cooperative Learning: "Increased verbalization forces cognitive restructuring and re-processing of information, as well as rehearsal and practice of relevant information and skil ls" [ H L K M 9 2 , p.258]. O n one hand, based on C L researchers' emphasis on face-to-face interaction, it might be argued that in a remote, computer-supported setting, communica-tion should come as close to face-to-face as possible. Indeed, this is the assumption made in video-supported C S C W applications which project images of remote users as they com-municate with each other over the network. O n the other hand, some studies reviewed in the previous chapter [Rie92, e.g.] have argued that removing face-to-face communicat ion can have positive effects (e.g., by allowing temporal flexibility in the learners' exchange of ideas). It is evident that much research remains to be done on what type of communicat ion 30 is necessary or ideal in the context of educational environments for children. M o s t current D E systems employ only wri t ten communicat ion between users, either synchronous or asyn-chronous. This study added spoken communicat ion to this model, a closer approximation to an in-person setting, to ascertain whether the learning and atti tude outcomes were dif-ferent from those in the wri t ten communicat ion mode. A p a r t from communicat ion per se, face-to-face interaction entails an awareness of the presence and behaviour of other learners, which becomes a non-tr ivial issue when learners are interacting remotely. Aga in this is an issue that is addressed in the C S C W literature, though one whose solution is part icularly domain-dependent. A s a first step in considering how users should be represented in a educational act ivi ty for children this s tudy investigated the effect of adding graphical rep-resentations of users. To sum up, the present study compared two communicat ion modes: wri t ten communication and "enhanced" communicat ion, wi th in which players could both speak and write, and also saw graphical representations of each other. The second variable manipulated was the nature of the task. Cooperative Learning studies have looked at the influence of content [DHH89] , nature of instructions [ H L K M 9 2 ] and structure of task [Coh94] on learning and other outcomes. In our study, the role of the task was investigated by varying the goal specification: some subjects were given a very specific goal (a magnitude of area or volume), such that they would know immediately when they had reached it, while others were given a general goal which was part ly open to their interpretation (they were told to maximize the area or volume). 2 The literature review provides conflicting suggestions for hypotheses regarding the comparison between these two conditions. O n one hand, Cohen's [Coh94] work, which provided the ini t ia l motivation for the different goal modes examined in this study, suggests that i l l-structured tasks lead to more successful learning. We might predict that learners have to communicate more when t ry ing to maximize the area or volume, and that this may lead to more reflection or cognitive change. O n the other hand, those given a specific goal know immediately when they have performed well. Based on the game literature [Mal82, e.g.], and also the findings of Nastasi and Clements [NC93] that direct external positive feedback from the computer was more motivat ing than t ry ing to promote an internal sense of motivat ion, we might predict that 2Further details are given in Section 4.2. 31 the more direct feedback of the specific goal outweighs the value of the more il l-structured task. F ina l ly , much research on Cooperat ive Learning and other forms of learning has addressed how the characteristics of the learner affect what is achieved through the learning act ivi ty [Web82a, J J S R 8 5 , e.g.]. One much researched characteristic is gender, which is part icularly relevant in the fields of C A L and C S C L due to the apparent gender differences in interest in , and mastery of, computer applications [ I U K + 9 4 a , Per94, Bul94] . Thus , in the present study, we had learners play Builder in same-sex pairs, allowing us to compare outcomes for boys and girls. The motivat ion for this comparison is not to judge achievement according to gender, but rather to be sensitive to the fact that some environments may favour part icular types of learner, which has to be addressed in instruct ional design. 3.4 Outcomes Measured The pr imary outcomes measured in the study were academic improvement and sociomoti-vational attitudes, specifically, perceived collaboration, persistence of interest in the game (a typical measure of motivation) and att i tude toward the task and the partner. These dependent variables are borrowed directly from the field of C L , where most studies have looked at either one or both of these areas. Pre- and post-tests on the mathematical con-cepts embedded in the act ivi ty (e.g., area, volume, perimeter, t i l ing of surfaces) were used to assess academic improvement. Fol lowing the example of previous E - G E M S studies [ IBK95, e.g.], performance in the game was used as an addit ional measure of achievement. The so-ciomotivat ional outcomes were evaluated wi th an a t t i tudinal questionnaire. Whi l e there are alternative methods to assess sociomotivational outcomes, such as the behavioural observa-tion used by Nastasi and Clements [NC93], the questionnaire is perhaps the most common method and is slightly less open to the problem of researcher bias. We were interested not only in the effects of communicat ion, goal and gender on achievement and att i tude (i.e., the "main effects"), but also in the effect of different combi-nations of these variables (i.e., the "interaction effects"). For example, it may be that the value of a communicat ion or task mode is gender-dependent. Look ing at the interrelation 32 of task and communicat ion is also interesting. The fact that there are advantages to either of the task styles, for example, prompts the question of whether the most appropriate goal specification might depend on the employed mode of communicat ion. 33 Chapter 4 Builder 4.1 Overview Work on the C S C L act ivi ty Builder has been done wi th in the context of the E - G E M S (Electronic Games for Educat ion in M a t h and Science) group at the Universi ty of Br i t i sh Co lumbia . E - G E M S is a collaborative effort involving computer scientists, mathematicians, educators, professional game developers, classroom teachers and students, aimed at mo-tivating children to learn and explore mathematical and scientific concepts with the aid of computer games. A m o n g E - G E M S ' current projects is the multiplayer game Island, in which the Builder act ivi ty is set. Island is a graphical , educational Mu l t i -Use r Dungeon ( M U D ) in which players solve mathematical puzzles to collect materials to build houses on the island. Island was created for the Macintosh platform in the programming language C + + using a client-server model. It runs over an AppleTa lk network, using the NetSprocket l ibrary and OpenTransport . The 3-D renderer wi th in the Builder act ivi ty uses Macintosh ' s Q u i c k D r a w 3 D library. Further details on the tools used and programming credits are given in Append ix A . Builder allows two players to design a house using various 2-D layouts and view it in 3-D. In the 2-D design phase players can switch between top-view and side-view, placing and resizing walls, windows and doors. W h e n players have finished designing their house, or at any intermediate point, they can enter the 3-D view and navigate around to inspect 34 the rendered representation of their house. The following discussion of the views in Builder should be read along wi th inspection of Screenshots 3, 4 and 5 in Append ix F . The screenshots show Dave's screen as he plays Builder wi th Sonia, who is in another room playing on a different computer. In Screenshot 3 (Top-View) they are laying out walls to define the rooms of their house. In Screenshot 4 (Side-View), Dave is placing a window in one of the walls. The final shot is Dave's view in the 3-D rendered environment. He can see Sonia's avatar moving in front of the walls of their semi-constructed house. In the version of the game they are playing, Sonia and Dave can communicate v ia either spoken or wri t ten messages. The two white boxes at the bot tom of the screens are for sending and receiving wri t ten messages. Microphones and speakers are used for the input and output of spoken messages. 4.2 The Task We wil l begin the more detailed discussion of Builder wi th a description of the content of the activity, which centers around the task of building a house. A single session of playing the game may consist of several separate "challenges", wi th each challenge corresponding to the building of one house. In the target age group (grades 5-9) the mathematics syllabus includes material on calculating perimeter, area and volume for various shapes, as well as t i l ing of surfaces. These concepts are embedded in the Builder activity, as described in detail below. The learning target is that conceptual understanding wi l l be improved in the following areas: addit ion and subtraction of areas and volumes; t i l ing of surfaces; and the relationship between perimeter and area (e.g., the fact that a square encloses a greater area than a thin rectangle of the same perimeter). A s one of the goals of the current study was to investigate the role of task in a C S C L setting, two task modes are implemented in Builder which differ in terms of how the area or volume target is stated. Players in specific goal mode ( S G M ) are given a numeric target which they have to reach exactly for successful completion of a challenge, while players in maximize goal mode ( M G M ) are instructed to build a house with the largest possible area or volume given the materials available. In the latter condition it is entirely up to 35 the players to decide when they have satisfied their goal. 1 In the former there is direct feedback indicating when the challenge has been sucessfully completed. The two modes are otherwise identical. B o t h consist of five challenges, wi th the first three measuring house size in area and the other two in volume. The size of the house that can be built in each challenge is constrained by the available resources. For walls the l imit is set by the number of bricks allocated to the players at the start of each challenge. For windows and doors the constraint is not in regard to the material used for the window or door itself, but on the wood pieces that are used to frame the window or door. A s for the bricks, a l imited supply of these pieces (which are of various length) is allocated at the start of each challenge and shared between the two players. Inserting windows and doors in the walls frees underlying bricks according to the surface area covered by the window or door, which in turn is set by the horizontal and vertical frame pieces they choose. Players are further constrained by a maximum allowable floor area per room so that they cannot s imply build one big room. The mathematical concepts are integrated into the act ivi ty in the following ways. • Relationships between perimeter, area and shape: W h e n arranging walls to make a room in Top-View (as described below in Section 4.4.3) players can create houses of greater area [volume] if the room is approximately a square [cube]. For example, two walls of length 20 wi th two walls of length 4 can define an area of 40 square units (or 72, depending on placement), while four walls of length 10 can define an area of 80 (see layouts in Figure 4.1). Because of the smaller perimeter of the latter room it uses fewer bricks (80 less if the walls are of height 10, which is the default). When the house consists of more than one room the opt imizat ion becomes more complicated. Similarly, in Side-View, players should discover that choosing vertical and horizontal frames of roughly equal size wi l l release many more bricks than choosing disproportionate frames. For example, two 1x9 windows release 18 bricks while one 9x9 plus one l x l release 82 bricks. • Addition and subtraction of areas and volumes: W h e n placing walls in Top-View to define a house, the wid th of the walls is not included in the calculation of floor-area 1 Players in M G M are given an indication of expected performance by the high-score records (described in Section 4.4.1) 36 72 20 40 l o " 10 80 10 Figure 4.1: Three examples (a,b,c) of room layouts and volume; i.e., the measurement is of internal area. For example, i f players make a house by laying out 4 walls of length 10 as indicated in Figure 4.1 c, the floor area wi l l not be 100 (as might be expected from the calculation 10x10) but 80 square units. It is almost impossible to ignore this issue when aiming for a specific area. Though it is less obvious, this concept is also involved in the placement of windows and doors in Side-View. For example, if the player makes a square window wi th sides of length 5 in a square wall wi th sides of length 10, the surface area of the wall not covered by the window can be calculated by, subtract ing the area of the window from that of the wall (i.e., [10 x 10] - [5 x 5] = 75 square units). Calcula t ing the remaining area like this is useful because it allows the user to know how many bricks the wal l requires. Since the bricks are square, wi th sides of length 1, the number of bricks used is in fact the same as the area of the wall not covered by doors or windows. A s these calculations are not str ict ly necessary in at taining the goals, they are performed by the program and displayed to the user beneath the currently active wall in Side-View (described further under Side-View in Section 4.4.4). • Tiling: The concept of t i l ing is natural ly embedded in the use of bricks in the activity. W h e n a player needs a certain number of bricks to make a new wall or enlarge an existing wall , s/he is motivated to calculate the door or window dimensions needed to free this number of bricks. To make t i l ing of surfaces a stronger focus of the activity, harder challenges could be introduced wi th bricks of varying dimensions. Given the short playing time for the current study we chose to keep bricks as 1 square unit to keep the t i l ing calculations simple. Based on previous research, it is not clear which of the two task modes should be expected 37 to generate greater understanding of these concepts. In S G M , we might expect that at least l imited awareness of the concepts is assured assuming that the player approximates and then achieves the numeric target. However, once the target is reached players are unlikely to experiment further with alternative strategies. O n the other hand, in M G M the degree of exposure to the concepts is set more by the ambit ion and curiousity of the players. Players might settle for a relatively low area or volume and hence discover less than those in S G M . Alternat ively, they might work exhaustively at the job of maximiz ing the size of their house, and hence discover more than is needed to meet the specific targets in S G M . Thus the relative ut i l i ty of S G M vs. M G M in enhancing learner awareness of part icular mathematical concepts remains an empirical question, and no specific hypotheses are provided regarding the superiority of ine over the other in the present context. 4.3 Communication T w o forms of communicat ion have been implemented in Builder, wri t ten and spoken. Wr i t -ten messages require typing in a Send-Message box at the bot tom of the game window, and then clicking on the "Send Message" but ton. A l l messages, including one's own, appear in a scrollable Receive-Message box also at the bot tom of the game window. The message boxes can be seen in all of the screenshots in Append ix F . Spoken messages require the user to hold down the C o n t r o l key while speaking into the microphone. Sound compression allows speech to be t ransmit ted wi th minimal delay. For research purposes, two modes of communicat ion wi th in Builder are defined. The basic communication mode ( B C M ) allows only wri t ten communciat ion. The enhanced communication mode ( E C M ) allows both wri t ten and spoken communicat ion as well as an element of "vi r tual presence". The term, v i r tua l presence, refers to the simulated presence of other participants in shared v i r tua l spaces, and has been a focus of several C S C W ap-plications [IM91, G R W B 9 2 , e.g.]. In Builder, v i r tua l presence is implemented in two ways. F i r s t , wi th in the 2-D building environment a small icon representing each player appears on top of the wall s/he is working on (see Screenshot 3). Second, when both players are exploring the 3-D model of the house they have constructed, each of them can see the 38 other's avatar moving around (see Screenshot 5). Due to constraints on time and subjects, it was not possible to examine the effects of speech and v i r tua l presence separately. The relative lack of previous research, however, makes a coarse-grain comparison such as this an excellent start ing point upon which to base more refined future investigations. 4.4 Sequence of Play and Interface This section describes the sequence of screens and components of the interface for each screen and should be read along wi th inspection of the screenshots (Appendix F ) . Dave is the player whose screen we are looking at; Sonia is the other player. The message windows are present throughout all Builder screens. 4.4.1 "Challenge-Selection" (Screenshot 1) This screen shows five buttons marked "Challenge 1 " , "Challenge 2 " etc. The main purpose of the screen is to allow players to choose which challenge they want to do, and to provide feedback on challenges already completed. If Dave tries to select a challenge before Sonia has entered the game, the message "wait ing for partner" appears. 2 Once both players have entered the game, either player can choose a challenge, which results in both players being sent to the Challenge-Info screen described below. If both players choose a challenge, the choice of challenge is determined by which message reaches the server first. T w o lists of scores are provided on the Challenge-Selection screen. The first column (adjacent to the buttons) gives the scores for the current session, while the second gives the record scores for each of the challenges. 3 There are two condition-specific differences for this screen. F i rs t , the scores are different according to the goal condit ion. In M G M (maximize goal mode), the scores indicate the area that was obtained for each challenge - hence higher scores are better than lower scores. In S G M the scores refer to how many bricks were used to successfully complete each challenge - hence lower scores are better. For S G M there is definite success or failure, and the brick-count is only recorded i f the challenge has 2 I t is possible to turn this off either at compile-time or run-time to play the game in one-player mode. 3 A s the record scores are stored in a file, the records date back to whenever the file on the machine was last moved or deleted. 39 successfully been completed. For M G M the area is always recorded if it is non-zero; i.e., there is at least one enclosed room. The second condition-specific difference is that players in E C M (enhanced communication mode) can choose an icon to represent themselves from this screen. A n addit ional button (labelled "Choose icon") is provided for this purpose. Players who do not select an icon before choosing a challenge are assigned default icons (chess pieces). C l i ck ing on the "Choose icon" button displays a simple a u x i l i a r y screen showing the icons to choose from. The icons are 32x32 pixel images which were downloaded from the W W W . Cl i ck ing on one of these images sets the player's icon and returns the player to the Challenge-Selection screen, wi th the "Choose icon" button no longer visible. The only variation is i f the icon has been picked already, in which case a message asks the player to choose again. 4.4.2 "Challenge-Info" (Screenshot 2) This screen presents the goal and available materials for the selected challenge. The center of the screen displays the challenge goal and states the max imum floor area per room constraint. For example, for Challenge 1 in S G M the screen displays the following: Challenge 1 Info. Your goal i s to b u i l d a house with area: 80 square units. Maximum f l o o r area per room: 100. Whereas for Challenge 1 in M G M the screen displays: Challenge 1 Info. Your goal i s to make the largest possible house. (Size i s in area.) Maximum f l o o r area per room: 100. A t the bot tom left the current number of bricks in the brick store is indicated. The sizes of the available frame pieces are listed at the bot tom right. M o s t of this information is also available in Top-View, wi th the exception of the list of frame pieces which is shown only in the dialog box when players set the frames. This screen serves the addit ional purpose of providing a signal that the challenge is about to begin, which is especially valuable for the player who did not choose the challenge. C l i ck ing anywhere on the Challenge-Info screen wi l l send the player into Top-View, which is the main building screen. Players can spend as long as they wish at the Challenge-Info 40 screen, irrespective of the partner's actions (e.g., Sonia can start building while Dave is s t i l l looking at this screen). It is also possible to view the Challenge-Info screen at any time by clicking on the question-mark but ton from the Top-View screen. 4.4.3 "Top-View" (Screenshot 3) Manipulation of walls The central area, marked wi th grids, is for laying out the walls of the house. B o t h players can be active in the region simultaneously, but a particular wall can only be manipulated by one player at a time. Dave's currently active wall is indicated by a yellow highlight. Sonia's currently active wall is coloured green. In E C M , players' icons are displayed on their active wall as a further indication of ownership. The following actions can be performed on walls in Top-View: • Walls can be created by cl icking the wall button on the right-hand button-bar (this button is labelled in Screenshot 3). Th is causes a wall 5 bricks long by 10 bricks high to appear in the lower left of the central area. U n t i l the player that created the wall selects another Top-View object, only s/he can manipulate the new wal l . Under the wall button on the button-bar are the window and door buttons, which are greyed-out and disabled in Top-View, producing an alert sound if clicked. • Wal ls can be selected by cl icking on them. If the clicked wall is currently selected by the partner, the game displays the message "Partner 's wal l" and plays an alert sound. Successful selection is indicated by a yellow highlight appearing around the wall , and the previously selected wall is de-selected. • A selected wall can be moved by cl icking and dragging wi th the mouse. Upon release, the walls snap-to-grid, al lowing ease of alignment wi th other walls for the user and simple, whole-number area calculations for the program. • A selected wall can be flipped 90 degrees by clicking on the flip button on the button-bar. The flip, resize and delete buttons appear in that order on the far right column of Screenshot 3. • A selected wall can be resized by clicking on the resize button on the button-bar. Th is 41 produces a dialog box displaying the current dimensions of the wal l , which can then be reset by the player to the desired size. The height of a wall can only be set in the last two challenges, which are concerned with volume. For the first three challenges, walls are fixed at a height of 10 bricks, and the height is not displayed in the resize dialog box. Resize requests that call for more bricks than are currently available in the brick store are disallowed and the player is notified by a message displayed to the screen. Resize requests that would leave windows or doors not completely contained by the wall are also disallowed. Otherwise the wall is redrawn wi th the desired dimensions. Height changes are not visible from Top-View, but can be seen when the player enters Side-View. • A selected wall can be deleted by clicking on the delete button on the button-bar. The wall 's bricks and any window or door frames used on the wall are returned to the store. Feedback and statistics • Shadow box: The top left box of the Top-View screen contains a shadow image of the top view of the working area, wi th the walls displayed in miniature. M i r r o r i n g the colours in the main building area, Dave's active wall is yellow and Sonia's green. Th i s box is also visible in Side-View, where it keeps the player aware of the current layout of the house. In both views it provides a means of selecting walls. • Statistics: Game statistics are provided in the lower half of the left hand side of the screen. The statistics include: the number of bricks that have been used as well as the total number allocated for the challenge, the names of both players, the view the partner is in , the area/volume target (for S G M ) , and the most recent calculation of area/volume. • Feedback box: Below the main building area is a black feedback box which displays information and error messages to the player. For example, if a resize request calls for more than the available number of bricks, the player is informed by the message "Not enough bricks for resize operat ion". Players are also informed i f their partner enters 3D-View or is wait ing to end the challenge. 42 • B u t t o n b a r : In addition to the statistics, which are constantly displayed, two buttons on the right-hand button-bar also provide information to the player. The " i " button gives information on the current selected object (e.g., for a wall it gives the length, height and orientation of the wall) . The large but ton marked "Area" 4 calculates the current area of the house. The algori thm to calculate floor-area works by start ing at a grid which is "outside" the house (for this purpose an unseen row and column of grids is added so that the program always has an outside grid to start at) . It then marks all grids it can "touch", including diagonally-touching grids, as indicated by the walls in Figure 4.2, which do not enclose an area. 5 After this the algori thm sweeps the space in search of an unmarked gr id . Upon finding one it uses the same process to mark all "touching" grids, then tallies up this group of grids to get the area for that " room". This sweep-and-tally process is continued unti l the sweep finds no unmarked grids. The algorithm terminates and feedback to the player is provided in the following manner. Each of the rooms is painted a different colour and the area value is displayed in this painted region. The total area of the house is displayed in the feedback box at the bot tom of the screen. 6 If the current arrangement of walls does not enclose a group of grids, no unmarked grids wi l l be found in the first sweep, and the value returned wil l be zero. A n y rooms wi th an area greater than 100 square units are painted black and the message " T O O B I G " is displayed in the room. The area of such rooms does not contribute to the to ta l area. Volume calculations take the height of the lowest contr ibut ing wall for each room and mult iply it by the floor-area. View Control F r o m Top-View players can move to one of two other views of the house. M o v i n g to another view does not affect the partner's view. Under the shadow-box are the two buttons that toggle the player between Top-View and Side-View. The current view is indicated by the depression and highlighting of the appropriate but ton. Players can move to the 3 D - V i e w by cl icking the house button on the right-hand but ton bar. The other buttons relating to 4For the last two challenges it is marked "Volume". 5Apologies for the pic, it was done in LaTeX :-) 6The total is displayed on both players' screens but the individual room feedback is provided only on the screen of the player who clicked the button. 43 Figure 4.2: Corner of a room that does not enclose an area the 3D-View (the eye and light buttons) are discussed in the 3D-View section. Game Control Players can also move to either the Challenge-Selection or Challenge-Info screen from Top-V i e w . The question-mark, as previously mentioned, returns the player to the Challenge-Info screen without ending the challenge. To exit the challenge, and hence move back to the Challenge-Selection screen, players click the red " Q " but ton. If Dave clicks this button while Sonia is s t i l l bui lding, Dave wil l be sent to a wait ing screen, from where he can either wait for Sonia to click " Q " , or choose to re-enter the challenge. If Sonia clicks the " Q " while Dave is at the wait ing screen, both players are returned to the Challenge-Selection screen. If they have successfully completed the challenge, the appropriate button wi l l be coloured yellow to indicate competi t ion and their score for the challenge wi l l be displayed. 4.4.4 "Side-View" (Screenshot 4) In Side-View players can add windows or doors to the existing walls of their house. A d d i n g windows and doors contributes to the successful completion of the challenge by freeing up bricks which players can then use for making more walls or enlarging existing ones. If the player already has a wall selected when s/he enters Side-View this wall is displayed from the side. N o other walls are visible. Below the wall (on the "grass") are data on the current surface area and bricks consumed by the active wal l . The player uses the door and window buttons on the right-hand button-bar to create the objects. The wall but ton is inactive in Side-View, and wil l produce an alert sound if clicked. Newly-created windows 44 and doors appear in a default location, as for walls, but are different in that they are ini t ia l ly incomplete, which is indicated by dotted lines. To complete construction of doors and windows players must choose their horizontal and vertical frame pieces from the l imited store provided at the beginning of each challenge. A door or window, which we wi l l refer to as an "object", does not become included in the active wall unti l both of the following conditions are met: • both horizontal and vertical sets of frames of the object have been set; • the object is entirely contained by the active wal l , and does not overlap any existing objects. The various possible states are indicated by the colour of the object and its outline. W h e n first created the whole outline of the object is dotted. Initially, since the object is active, the dotted outline is coloured yellow and black. If the object is not active (i.e., not the currently-selected object) the outline is coloured white and black. W h e n the player chooses a set of frames the appropriate two sides of the object 7 go from dotted to solid lines (yellow if active, black if not) . The top two buttons on the right-most column of the button-bar set the horizontal and vert ical frames respectively, and when clicked present a dialog box list ing the frames st i l l available and asking the player to type the length of the desired piece. U n t i l the object becomes officially part of the wall it is coloured pink, and in Top-View wi l l not be visible. Once the player has placed the object so that it is fully contained by the wall and not overlapping any existing objects, the image of the object changes from pink to a picture of a door or window. A t this point the server adds the object to the wal l , which results in the bricks covered by the object being returned (the player is notified by sound and visual feedback, and told exactly how many bricks have been returned), and the object appearing on the appropriate wall in Top-View. A window or door can also be removed from a wall , at which point the object wi l l again become pink, the released bricks taken back, and the object wi l l disappear from Top-V i e w . A n object cannot be removed, however, if there are not enough bricks available to fill the hole it would leave. P i n k objects remain in the current Side-View even when the player 7 That is, the top and bottom if the player has chosen the horizontal frames, or the left and right if the player has chosen the vertical frames. 45 switches between walls. In this way the player can choose to place an already-completed window on whichever wall s/he wishes. The player can switch between frame sizes at any time, or delete the whole object, which returns the object's frame pieces to the store. 4.4.5 "3D-View" (Screenshot 5) The 3D-View allows players to move around and inspect the house that they have con-structed. There are three buttons on the button bar in Top-View relevant for the 3 D - V i e w (shown in Screenshot 3): an eye but ton which allows the player to set the viewing location and direction; a light button by which players can place an addit ional point-light source in the scene; and the house button which sends the player into 3 D - V i e w . Once in 3 D - V i e w the player can navigate around using the arrow keys. Walls appear in red wi th holes where windows and doors have been placed. Players can move through objects and so can enter into the house and see what it looks like from the inside. In E C M , if Sonia is simultaneously in 3D-View and in Dave's field of view, Dave can see Sonia's avatar moving around and vice-versa. 4.5 The Client-Server Structure A s Builder is a networked multiplayer game, distributed programming is necessary, mean-ing more than one process runs simultaneously and processes must communicate with each other. Th is is done using a client-server model, which this section briefly describes. In a client-server model clients do not communicate with each other directly, but rather commu-nicate with a central server. Builder was originally implemented using a client-client model wi th clients communicat ing directly wi th each other, but was converted to the client-server model after the first pilot study (Section 6.1) to address synchronization and other stabil i ty problems. Builder runs wi th in the Island framework, wi th the servers and clients implemented as 04-+ classes. There is a base class Server and a base class Client, and IslandServer and IslandClient (hereafter IS and IC) and BuilderServer and BuilderClient (BS and B C ) are all children (at the same level) of the two base classes. A l l other activities wi th in 46 Island are implemented in the same way as Builder. IS and IC are always active, while each of the other server-client pairs is active only when the player is engaged in the specific activity. Whenever an IC or B C is created, it checks whether there is already a server running, and if not it creates one. Other machines are notified of where each of the servers are running so that they know where to send messages. These network messages are received ini t ia l ly by the IC on the machine running the server in question and passed on to B C if appropriate. The servers do not run as separate processes but are rather collections of functions that are called by the client on the same machine. IC contains the main program loop. Whi le Builder is running, most user events are handed off (by IC) to B C . Exceptions to this include events relating to outgoing wri t ten and spoken messages, which are handled by I C . In addition to user events, IC also handles all incoming messages arr iving over the network. Incoming wri t ten or spoken messages are dealt wi th directly by IC (displayed to the message window or played through the speaker). A l l Builder-related messages are sent on to B C . A s B C processes the user events it determines when calls to B S are required. A n y t h i n g that alters the collective game information or the user's status in the game requires a message to the server. For more information on messages passed between B C and B S see Append ix B . 47 Chapter 5 Tools for Assessing Outcomes The study forming the central part of this thesis compared the effect of various conditions on a set of outcomes. The independent variables of the study were: mode of communicat ion (written vs. enhanced), nature of task (specific vs. general), and gender(male vs. female), and wil l be discussed further in a subsequent design section (Section 7.1). Th i s section is concerned with the dependent variables, and the tools developed to measure them. The main outcomes of interest were: academic achievement, performance in the game, the nature of game-play, and attitudes toward the task and the playing partner. Fo rma l tools were developed to assess academic achievement and at t i tudinal outcomes. Performance in the game and nature of game-play were assessed v i a automatically-recorded game logs and anecdotal observations recorded by the researchers. 5.1 Academic Measures 5.1.1 Target Areas The target learning areas, as discussed under the section on Task above (Section 4.2), were: • the relationship between perimeter and area; • addit ion and subtraction of areas and volumes; • t i l ing of surfaces. 48 5.1.2 Tests Academic improvement in the three target areas was assessed using a pre-test and a post-test, which are presented in Appendix C. The tests each consisted of 10 items, with each of the post-test items being a variant of the corresponding item in the pre-test. The original versions of the tests were developed by the researcher based on inspection of standardized mathematical test materials [CT0BS88] and textbooks [Les86, e.g.] for the Grade 7 level. The tests were then revised based on pilot testing (see Pilot Study 1, Section 6.1) and consultation with researchers in mathematics education. The final structure adopted was as follows. • Three items asked for simple calculations concerning area, volume, perimeter and tiling. • Three items asked for calculations regarding particular scenarios from the activity -e.g., calculation of floor area given the length of walls, calculation of bricks saved by windows of a certain size, etc. • Four word problems required understanding of the target concepts but ih different contexts - e.g., how much area has a grass-cutter of width X cut after one cycle around a park of dimensions YxZ. The only difference between the pre-test and post-test were the numbers involved in the calculations, and the scenarios used in the word problems. The comparability of the two tests, and the assumption that the items on each test assess a unitary underlying construct, 1 are addressed at the beginning of the results section. 5.2 Socio-motivational Measures Appendix D presents the questionnaire that was completed by subjects directly after playing the game. Twenty attitudinal items were included to assess the sociomotivational outcomes 1It is possible, especially given the three categories of question discussed above, that different items draw on different knowledge or achievement domains. In this case it would be inappropriate to use only the whole test scores in data analysis, as we could be averaging over important differences that would be seen if the test items were broken into domain-specific groups. 49 of game-play. For each i tem, students indicated the degree to which they agreed or disagreed on a 5-point, Liker t scale ( Y E S , yes, maybe, no, N O ) . The dependent variables which these items were designed to assess were as follows: • attitude toward the game (e.g., "I enjoyed playing Builder", "I learned something by playing Builder"); • attitude toward partner (e.g., "If I play Builder again I would like to play wi th the same partner", " M y partner was friendly"); • desire to continue playing (motivation) (e.g., "I would like to play Builder again", "I would like to play Builder at home"); • perception of collaboration (e.g., "I would prefer to have my partner in the same room", "Communica t ing wi th my partner helped us to play the game"). In addition to the a t t i tudinal items, there were three background questions regarding home computer use. These questions were used to gather descriptive information on patterns of use amongst the sample pool , and to ascertain whether these patterns reflected the gender differences outlined in the li terature review. Items on an ini t ia l version of the questionnaire were modified on the basis of pilot studies and consultation with a psychology research group, the latter of which suggested adding related items for each of the target dependent variables to improve the psychometric robustness of the questionnaire. The research group also identified the need to obtain a pre-game measurement of partners ' l ik ing for each other, without which it would be dubious to make comparisons between conditions based only on their responses to the post-game questionnaire. Th is was addressed by introducing a one-item pre-game questionnaire which asked subjects to answer the question: "How much do you like playing with your allocated partner (<name-of-partner>)?" on a 5-point scale between "Not at a l l " and " A lo t" . 5.3 L o g Files Builder logs two types of information for each session of play. F i r s t , for al l completed challenges, a record of the challenge number and attained score is kept. A s mentioned in 50 the chapter on Builder, the challenge score indicates either the number of bricks used (for S G M ) or the area/volume attained (for M G M ) . Scores are only recorded i f challenges are successfully completed, defined either as achieving the target ( S G M ) or enclosing a non-zero area/volume ( M G M ) . The number of challenges completed and the challenge scores were used as measures of performance in the activity. Second, information about the messages passed between players is recorded, includ-ing a script of the entire writ ten dialog. We decided not to record speech messages because of the overhead of wr i t ing such large chunks of data to file during play. Instead, at the completion of play, a measurement of the total amount of speech data sent over the net-work is writ ten to the log file. These logs of writ ten and spoken communicat ion allow for comparison of how, and how much, subjects in different conditions communicated. 5 . 4 Observations Dur ing each session of game-play, researchers also completed observation forms (shown in Append ix E) to supplement the information in the log files. A s the forms indicate, one of the main issues here was the type of communication between players. Th is was part icularly important for players using spoken communicat ion, since the log files did not record spoken messages. The forms also allow for ad hoc observations which are often useful as suggestions for further research or game improvements. For example, if a player repeatedly at tempted to resize or move an object in a way that the interface does not support , this could be noted in the observations and considered as a modification to the interface. 51 Chapter 6 Pilot Studies Four pilot studies were conducted prior to the commencement of the final study. Such preliminary field testing was necessary given that both the software and the assessment tools were entirely new. The pilot studies also played an important role in the design of the interface and other aspects of the software. The dates and locations were as follows: • Pilot Study 1: - Date: M a r c h 11-13, 1997 - Location: Trafalgar Elementary, Vancouver - Subjects: 24 grade 6/7 students • Pilot Study 2: - Date: A p r i l 24, 1997 - Location: E - G E M S laboratory, U B C (students visi t ing from Island Pacific School, Bowen Island) - Subjects: 8 grade 7-9 students • Pilot Study 3: - Date: M a y 7, 1997 - Location: Trafalgar Elementary, Vancouver - Subjects: 14 grade 6/7 students 52 • Pilot Study 4 : - Date: M a y 20, 1997 - Location: Kerrisdale Elementary, Vancouver - Subjects: 12 grade 5 students For each of the pilot studies, the duration of one session of play was approximately 30 minutes. A s was the case for the final study, the number of subjects for these studies was l imited by the number of Power Macintoshes available for play. W i t h the exception of P i lo t Study 2, playing was l imited to the two machines brought into the school by the researchers, allowing only one pair of students to play at a t ime. 6.1 Description of Pilot Study 1 The ini t ia l pilot was the most extensive in terms of duration and size of subject pool, and had the most influence on design changes because it was the first opportuni ty to realistically evaluate the user interfaces and stabil i ty of the software. Therefore it wi l l be discussed in detail . 6.1.1 Goals The ini t ia l pilot study had four objectives. The primary objective was to evaluate the usability of the game wi th respect to the following questions. • Does the software behave reliably under "real" conditions; i.e., when played by two people of the target age-group for a reasonable length of time? • Is the game interface understandable to first-time users? • A r e the challenges within the game of an appropriate level of difficulty for the target age group, given the intended durat ion of play in the final study? • Is the game sufficiently s t imulat ing to engage players for the durat ion of play? A secondary objective was to evaluate the effectiveness of the proposed assessment methods. Th i s was part icularly important for the pre- and post-tests assessing academic outcomes, 53 where it was necessary to determine whether they were of an appropriate level of difficulty for students in the target age group, and whether the pre- and post-test were comparable. Our goal for the level of difficulty was for mean test results of approximately 50% on each test (assuming that there is no game-play or other form of task-related instruction between the two tests), thereby allowing a wide spread of marks and a sufficient margin to observe improvement on post-test scores. For the questionnaire, it was also necessary to ensure that all items were readily understood by the students. A th i rd objective was to evaluate the proposed set of independent variables for the final study. The main independent variable being considered in P i lo t 1 was mode of communicat ion (written vs. spoken). Accordingly, subjects were split into two groups: those wi th both wri t ten and speech communicat ion, and those wi th only wri t ten communicat ion. 1 Furthermore, as planned for the final study, subjects were tested in same-sex pairs so that there was an opportuni ty to observe gender differences in playing. F ina l ly , the pilot allowed the researchers to practice running the study. A study of this nature involves juggling a multi tude of practical constraints, and pilot studies are very useful in identifying problems in the procedure that might compromise the final study. 6.1.2 Procedure Pr ior to the first day of P i lo t 1, consent forms were distributed to students in order to obtain parental permission for their involvement in the study. Only those students who received parental permission and who themselves agreed to participate were included in the sample. O n the first morning of the study, two Power Macintoshes were set up in the l ibrary of the school so that there was no direct visual or aural contact possible between the players at the two computers. The researcher then gave a 10-minute orientation to the class, introducing the students to the task and interface of the activity. The orientation was followed by the pre-test (completed in the classroom), for which students were allowed 20 minutes. Fol lowing completion of the pre-test, the dyadic playing sessions began, wi th same-sex pairs being taken in turns to the l ibrary area where the computers were set up. Each session began wi th a researcher briefly explaining the interface to the student at x The virtual presence element had not been implemented at the time of Pilot 1. 54 each computer. Th i s consisted simply of point ing to some of the main buttons and briefly describing their function. Al though a help screen was provided, few players spent much time reading i t . The pair then played the game for approximately 30 minutes. W h e n their time was up, the students were seated together in another area of the l ibrary and asked to complete both the post-test and the a t t i tudinal questionnaire, for which they were allowed as much time as they needed. In addit ion to the 16 5-point scale atti tude items, 2 the questionnaire included the following open-ended items that allow students to identify problems with and propose enhancements to the game. • Please write anything you found frustrating about playing Builder . • D o you have any comments or suggestions on the communication? • D o you have any other comments or suggestions about Builder? 6.1.3 Results of Software Testing Performance of game The observations of the game's performance in P i lo t 1 revealed the presence of several bugs in the implementation. A t the time of P i lo t 1, Builder used a client-client distributed model where clients inform each other of their actions, rather than informing a central server. A l -though a simple form of locking objects and actions had been implemented, observations indicated that delays in the receipt of between-client messages resulted in inconsistent in -formation about the state of the game, and occasionally in one of the machines crashing. 3 Crashes impeded game-play considerably because players had to wait for the machines to reboot, which took several minutes. Changes made to improve performance Whi le these problems could have been fixed wi th in the client-client model by implementing a thorough verification system, we decided to reorganize the code to follow the conceptually simpler client-server model, in which the server enforces consistency of information. These 2 The 5-point items contained in the original questionnaire are similar to those of the final ques-tionnaire (presented in Appendix D, items 1-20). 3 Crashes were mostly the result of one program trying to act on messages from the other program that referenced an object about which the machines held inconsistent information. 55 changes took several weeks as it was also necessary to restructure the central Island code, and, because the changes were quite extensive, led to the need for further field testing. 6.1.4 Assessment of the Interface The following assessments regarding the design of the game are drawn from the observation forms and the students' feedback on the open-ended items in the questionnaire. M a n y of the points relate to "dressing up" the game's interface. The changes that were made are also described. M o s t of these were made to improve the ease-of-use and/or enjoyment of the game, and relate pr imari ly to providing adequate feedback to the users. Feedback • Players did not appear to be sufficiently aware of what their partner was doing. Th i s was partly related to the degree of communicat ion between partners which is discussed further below. A t the t ime of P i lo t 1 , the wall currently being used by the partner was indicated only by a blue highlight around the small image of the wall in the shadow box. 4 It appeared that most players did not pay attention to this. Th i s was evidenced, for example, by one wonderful response on the questionnaire: "The computer started moving and adding walls." Therefore the interface was modified so that the partner's wall was outlined by a blue highlight in Top-View as well as in the shadow box. Further pilots indicated that players were st i l l not noticing the blue highlight, so in the final version the partner's wall was given an entirely different colour, rather than just a highlight. • In addition to the problems of awareness of the partner's actions, there was evidence of a need for more feedback in general. Whi l e some illegal actions were flagged by an alert box, 5 for many important actions there was no feedback. Rather than adding alert boxes for al l these actions, we decided to add a feedback box to the bot tom of the main building area, which would display messages regarding important actions to 4The shadow box is the miniature copy of the top view, as described in Section 4.4.3. 5 A n alert box, typical in Macintosh and Windows applications, pops up within an application like a dialog box to present important information, and must be clicked by the user before work within the application can continue. 56 the player, and remove the interruption of play caused by alert boxes. Messages to be displayed in the new feedback box included confirmations of the player's actions (e.g. "Moved wal l" ) , notification of the partner's actions (e.g. "Partner is wait ing to end the challenge"), error messages if the player's action is illegal (e.g. "Partner 's wal l " when the player tries to select the partner's current wall) , and game information (e.g. "Area : 90", after the player clicks on the area but ton) . • Feedback for area or volume attained was also improved as a result of P i lo t 1. A t the time of the pilot the feedback consisted simply of displaying the total area or volume of the house as a number on the screen. We decided to enhance feedback in this area by calculating the area/volume of each room separately, and then painting each intact room a different colour and displaying the area/volume in the middle of the room itself (while displaying the total in the feedback box). Furthermore, to draw attention to the problem of exceeding the room-size l imi t , we decided to paint oversized rooms black and mark them with the message "Too big!" . Enjoyment • Based on suggestions from several subjects on the questionnaire, and also for the sake of addit ional feedback, it was decided to add sound effects to Builder. Sounds added include: generic clicking sounds for buttons, specific sounds for flipping, moving and resizing objects, and various different alert sounds. • T w o of the sound effects added addressed specific feedback issues. F i r s t , researchers identified the need for feedback regarding bricks being returned after the placement of doors or windows. Th i s is an important concept in the game because of the connection between the amount of bricks released and the lengths of the frames which the'player has selected. There should also be a sense of reward attached to the procurement of more bricks to build walls w i th . Therefore, to draw greater attention to the bricks returned, we decided to add a sound effect emulating a slot machine and to display the exact number of bricks returned in the feedback box. Second, an alert sound was added to draw attention to incoming messages, which players often failed to notice. • A m o n g the responses were several specific enhancement suggestions for the 3-D com-57 ponent of the game, which appeared to generate a substantial amount of interest. M o s t common among these was the request for animation, which had in fact been part ial ly implemented at the time of P i lo t 1, but was not ready for inclusion in the version of Builder used in the study. A l o n g with the facility to move around the 3-D space we added moving avatars to represent players when both are in the 3D-View simultaneously. The effect of this addit ion was included in the design of the final study (in the mode of communicat ion comparison). Ease-of-use • One respondent requested the addition of a button to send messages rather than having to use the menu bar, which was implemented in the revised version. • The interface for moving walls in Builder is drag-and-drop. In the in i t ia l version, the first click on a wall initiated the drag-and-drop process. M a n y players showed some frustration when they clicked on a wall wi th the intention of selecting it (e.g. for resizing or flipping) and the wall then jumped to another gr id , because they were moving the mouse around while cl icking. To remove this frustration, and also to provide a convenient way to implement locking, the interface was changed so that an ini t ia l click on a wall indicates only selection. The wall must then be clicked again to initiate the drag-and-drop. 6 • The controls for switching between Top-View and Side-View were not prominent enough. Initially implemented just as words on the screen that players had to click, they were updated to look like buttons. A l l buttons in the interface were modified so that they appeared raised out of the screen, and depressed into the screen when clicked. Inactive buttons were either hidden or darkened. • Several players counted grids on the screen to t ry to calculate the interior lengths of walls, which appeared inconvenient. One respondent also complained that the grids were too small for counting. O n the basis of this, we added an information but ton 6 T h e r e is o f t e n a t r a d e - o f f , h o w e v e r , w i t h s u c h d e c i s i o n s . I n t h e u p d a t e d v e r s i o n o f t h e g a m e p l a y e r s w e r e f r u s t r a t e d w h e n t h e y c l i c k e d o n a w a l l t o i n i t i a t e t h e d r a g - a n d - d r o p a n d t h e w a l l d i d n o t m o v e . It m i g h t b e p o s s i b l e t o r e f i n e t h e i n t e r f a c e f u r t h e r b y m e a s u r i n g h o w l o n g t h e m o u s e b u t t o n is h e l d d o w n a n d h e n c e d i s t i n g u i s h i n g b e t w e e n c l i c k s i n t e n d e d f o r s e l e c t i o n a n d g r a b b i n g . 58 (marked "i") to provide dimensions of objects when clicked, though many subsequent players nevertheless continued to count grids. Al though this might change i f players had had a longer time to familiarize themselves with the interface, counting grids is a useful act ivi ty in terms of reinforcing the players' understanding of length and area. • The need to type sizes into dialog boxes for resizing objects was observed to be un-intuit ive for players. W h e n presented wi th a list of sizes to choose from and asked to type one of them (which was the case for choosing frame pieces) players often tried to click on items in the list rather than typing. The frustration seen wi th the dialog-box method of resizing raised the design question: what is the best interface for resizing? To address this question an alternative resize interface was implemented, which is described below in connection to P i lo t 3. Other areas addressed • Modif icat ions were made to the area and volume goals for some of the challenges to adjust the level of difficulty. A more important change which was incorporated later was to require that the area/volume goals be met exactly. A t the time of P i lo t 1 the goal was to at least reach the stated area/volume target. It was thought that making the goal more rigid would draw greater attention to the numbers involved and also make the challenges more difficult to achieve by trial-and-error. These conjectures were strongly supported by observations in the final study, which indicated that many players did not start to think about the lengths of their walls unt i l they had had the experience of making the house both too big and too small . • The statistic of the overall surface area of the house was removed because it was rarely looked at, and when it was, seemed to be confusing rather than helpful. • Fol lowing observations that players spent negligible t ime in the help screen, it was removed and replaced with the Challenge-Info screen (as described in Section 4.4.2). 6.1.5 Findings Regarding Communication and Gender Statist ical analyses were not performed on the results of the pilot, since the interruptions caused by the instabil i ty of the software may have had a disproportionate effect on differ-59 ent groups. There were, however, several interesting communication- and gender-related game-play observations. To begin wi th , there was evidence of a wide variety of differ-ent collaborative strategies. The most highly-structured approaches were characterized by partners tell ing each other what to do, informing each other when they had finished a task, requesting confirmation of what the other was currently doing, giving reasons for their ac-tions (e.g. "I 'm t ry ing to save bricks") and even ini t ia t ing turn-taking systems. A t the other end of the spectrum were pairs that used the communication mainly for fun or for insults. In the latter case, partners would sometimes not communicate at all until the partner had done something annoying. There was a final category of players who barely communicated at a l l , and this did not appear to be related to whether they had speech communicat ion available to them or not. Some of the observed communicat ion differences appeared to interact wi th gender. For example, it was noted that some players wi th access to both forms of communicat ion st i l l showed considerable use of the slower wri t ten medium, and this was more often the case for girls than boys (one gir l even used the spoken communication to tell her partner to write to her more). M o s t boys with speech tended not to use writ ten messages much. Another notable gender difference was that some girls seemed more focussed on the communicat ion than the task, which was almost never true for the boys. 6.1.6 Difficulty and Engagement The degree of difficulty appeared to be appropriate for the pilot sample. M o s t players were able to successfully complete at least one of the challenges, suggesting that the act ivi ty was not prohibit ively difficult. Some players completed two or more of the challenges, but none completed more than three, indicating that the game was not too easy. A s mentioned above, the difficulty of some challenges was slightly modified by changing the area/volume goal or the number of available bricks. The findings regarding degree of engagement in the activity were encouraging. In contrast to the researchers, players seemed relatively unconcerned by any problems encoun-tered, and many expressed a desire to play again. M o s t of the comments on the questionnaire expressed a positive at t i tude toward the game. Several players said the game was fun to 60 play, and about the same number again specifically mentioned that communicat ing was fun (referring both to wri t ten and spoken communicat ion, but especially the latter) . 6.1.7 Findings Regarding the Test Materials The pre-test produced a satisfactory range of results, wi th a mean of just less than 50% (about 2.2 out of a possible 5). The teacher of the class confirmed that the results reliably predicted known math level. Post-test performance, however, was lower than expected, and, surprisingly, lower than pre-test scores, wi th an average of around 1 out of 5. The low average was an obvious concern, though we did not interpret this as imply ing that playing the game was detrimental to math performance. Rather, we concluded that items on the post-test may have simply been more difficult. Accordingly, pre- and post-tests were restructured so as to be more symmetr ical . The pilot was helpful in this restructuring since it gave a good indication of the difficulty of each i tem. The low post-test scores may also be attr ibutable to the administrat ion of the post-test, which is discussed in the following section. Given that the results of the pilot study were open to the interpretation that playing the game had a negative effect on test performance, it was decided to add a control group for the final study. Con t ro l subjects would sit both pre- and post-tests, and the difference between their scores would provide a baseline against which to compare the test results of the experimental groups. There were several minor changes made to the questionnaire after P i lo t 1. It was found that one of the items was confusing because it was expressed in the negative and therefore hard to answer on the yes-no scale. Th i s i tem was reworded. The open-ended section was removed for the final study because it would increase the time required for each subject, and because the questions regarding problems of and improvements to the game were specifically relevant dur ing the development of Builder. 6.1.8 Procedure-related Findings The main procedural concern that emerged during P i lo t 1 was in regard to the adminis-trat ion of the post-test and questionnaire. There was some evidence of partners making identical responses to the questionnaire, and it also appeared as though relatively l i t t le effort 61 was put into the post-test, compared to the pre-test. For the pre-test, the whole class was present together wi th the teacher supervising, as for any normal classroom test. Therefore there was some pressure to perform well on the test. Furthermore, since the teacher main-tained silence for the allocated time l imi t , students were encouraged to spend more time thinking about the questions. For the post-test, however, students were seated together and left unattended, allowing them to discuss responses and also removing the aspect of test pressure. Students also knew they could leave whenever they were done, which may have provided the temptat ion to skip over t ime-consuming questions which the classroom setting would have prompted them to at tempt. To address this concern, two procedural changes were introduced. The first was to delay the administrat ion of the post-test unti l after all participants had played Builder, and thus allow the test to be taken as a class in the normal classroom setting. This had the disadvantage that the game would no longer be fresh in the subjects' minds, but had the advantage that the test would be taken more seriously and performance would therefore be comparable wi th that on the pre-test. The second change was to separate the partners while wri t ing the questionnaire (which would s t i l l be done directly after the game - since it was not mentally taxing like the post-test, there would be less reason for students to skip items). It was hoped that this modification would encourage students to provide considered responses based on their own experience. The other major change to the procedure was in regard to the specific pre-game orientation. F i r s t , so that the orientation would be standard across al l participants, it was formalized and always given by the same researcher wi th both members of the dyad in front of the computer (which had the addit ional advantage of freeing one of the researchers to oversee the last pair s t i l l working on the questionnaire). Dur ing the formalization of this orientation, several important features were added (see Section 7.4). One addit ion that emerged specifically from Pi lo t 1 was the decision to explici t ly encourage players to com-municate wi th each other during game-play. A lack of effective communicat ion between partners was frequently observed during the pilot study, though there were certainly excep-tions to this. The extent to which players communicated was entirely up to them, wi th the intention being that if they felt a need to communicate they would do so. This approach 6 2 was reconsidered in light of the observations, as well as i l luminat ing student responses such as: "Make players tell each other what they're doing more often". It was also thought that explicit ly encouraging players to communicate was in accordance wi th the afore-mentioned Cooperative Learning technique of t raining learners in collaborative techniques. In addition to the human communicat ion problems, there were also technical problems such as fuzzy and choppy messages. These were addressed by including, in the description of how to com-municate, reminders to hold the Cont ro l button right unti l the end of the spoken message and to not talk directly into the microphone. 6.2 Other Pilot Studies Due to the extensive restructuring of the Island and Builder code discussed above, further field testing was essential to ensure that the software was sufficiently reliable for use in the final study. The other pilot studies, however, wi l l not be discussed in detail , since the problems identified and the resulting changes were considerably smaller, both in number and importance, than those from Pi lo t 1. Pi lots 2 and 3 revealed the existence of problems with the modified software, including a serious memory leak which occasionally led to crashes. B y Pi lo t 4 the memory leak had been located, and the number of problems had been reduced to the extent that only a single bug was identified during the six sessions of play. Between P i lo t 1 and P i lo t 2 the pre- and post-tests were extensively modified, both approximately doubling in length. Th is necessitated further testing to again at tempt to balance the two tests for level of difficulty. Hence the academic tests were administered to students in both P i lo t 2 and 3, and there was some resultant shuffling of questions. 6.2.1 The "Unintuitive" Interface (Pilot 3) A s mentioned above with regard to interface problems identified in P i lo t 1, there was some concern over how best to allow players to set the size of walls, windows and doors in Builder. Observations indicated that typing a number into a dialog box to set the size of an object was "unintui t ive", where we define "intui t ive" as being the way that naive users naturally at tempt to perform an operation. A s an experiment in interface design, we 63 decided to implement an alternative method of setting the size of an object. The alternative interface, which we wi l l refer to as drag, does away with the dialog box, and instead displays a small green dot at the bottom-right corner of the object, which users can drag to reset the object's size. A s the dot is dragged (i.e., while the mouse but ton is st i l l depressed), feedback is provided dynamical ly in two forms. F i r s t , a "shadow" rectangle appears, whose top-left corner is set at the top-left corner of the object, but whose bottom-right corner follows the moving cursor. 7 Second, the dimensions of the object at the current position of the dot are displayed to the user v i a the feedback box. In Top-View, for example, if the dot is dragged such that the shadow rectangle is ten units long, the feedback box would say "Current length: 10". (Note that in Top-View the only possible size modification is to the length of a wall , so when players drag the green dot the shadow rectangle changes only in length, not in width , since the width of a wall is set at one unit.) In Side-View the situation is more complex, since feedback is required on the current width and height of the window or door. Furthermore, the program dynamical ly calculates the nearest available horizontal and vertical frame lengths corresponding to the current cursor posit ion, and displays these two numbers in the feedback box. Upon release of the mouse but ton, providing the new size is legal, walls are enlarged or reduced to the nearest grid-line, and windows and doors are set to the nearest available frame lengths. The drag interface would be judged a "better" or more "intui t ive" interface by most H C I cri teria. It involves direct manipulat ion, giving the il lusion of active stretching or shrinking of an object. It also provides interactive feedback, wi th the shadow providing a W Y S I W Y G - s t y l e preview of what the modified object wi l l look like. In the dialog interface, on the other hand, naive users may have no idea of what the result of their action might be, and may therefore be tentative about typing in a new number. However, returning to the discussion of H C I issues in the literature review, it is not clear whether an intuit ive interface is appropriate wi th in a game or educational application. W i t h i n the Builder context, where the aim is for learners to experiment wi th mathematical concepts, it is important that attention is paid to the object sizes. A n intuit ive and easy-to-use interface like drag has two potential disadvantages. F i r s t , because users do not have to think of a number to 7This "shadow" rectangle is familiar to anyone who has dragged the bottom-right corner of a window in most current Macintosh, Windows, and even UNIX-based X-Windows applications. 64 type into the dialog box, they may be less aware of the size they have set the object to. Second, because it is faster and easier than dialog, users are more likely to use trial-and-error, continually modifying objects without reflecting on what they are doing. A s a result of the preceding points, when players in S G M , for example, reach their area/volume goal, they may not be aware of what size the walls have been set to, which makes it less likely that they wi l l make any discoveries about interior area in relation to side-length. The drag interface was pilot-tested during P i lo t 3, where half the pairs used drag and the other half used dialog. A t this point in the development of Builder, the interface issue was being considered as an addit ional independent variable to manipulate in the study. The hypotheses for the different interface styles were: • drag wi l l be considered easier and more enjoyable than dialog • dialog wi l l result in better performance on the post-test Unfortunately, it was not possible to look at pre-post comparisons for P i lo t 3, due to the fact that the tests were st i l l being calibrated and because some of the participants that played Builder during P i lo t 3 had also played in P i lo t 1. Interviews with the players who had used both styles, however, confirmed that drag was preferred as an interface style. Al though indications suggested that the interface comparison would make an interesting variable for the final study, it was decided not to include it because it does not bear directly on collaboration in the game. Therefore P i lo t 3 is best viewed as the prel iminary investigation of a possible future study. Based on the potential advantages of the "unintui t ive" interface discussed above, dialog was chosen as the interface for the final study. 65 Chapter 7 Study Design and Methodology 7.1 Design The experiment was a 2x2x2 factorial design. The independent variables were as follows: • gender (SEX) - male-male pairs vs. female-female pairs • mode of communication ( C O M M ) - basic communication mode ( B C M ) vs. en-hanced communicat ion mode ( E C M ) • nature of task ( G O A L ) - specific goal mode ( S G M ) vs. maximize goal mode ( M G M ) . The dependent variables were: • academic gain - post-test score minus pre-test score • game performance - number of challenges completed and challenge scores • sociomotivational effects - atti tudes toward the activity, atti tudes toward partner, persistence of interest in game, and perceived collaboration. A no-activity control group was used to provide a baseline for pre- and post-test compar-isons, so that any differences seen in the experimental conditions could be judged relative to differences seen in the control group. It was hypothesized that the game group (composed of al l students who played the game, regardless of G O A L or C O M M condition) would show greater academic gain than the control group. A l l comparisons other than this in i t ia l game 66 vs. control test were between the eight cells of the 2x2x2 design. Based on theories and findings from Cooperative Learning, it was hypothesized that the E C M group would show higher achievement, both in terms of academic improvement and game performance, than the B C M group. For the nature of task hypothesis, the si tuation was less clear. Based on findings that ill-structured tasks lead to more effective Cooperat ive Learning [Coh94], it might be expected that M G M dyads would show greater learning improvement than S G M dyads. However, a specific goal may make players focus more on the numbers involved (especially relevant for mathematical learning), and may also be more motivat ing due to the direct feedback of knowing whether or not the goal has been attained. Hence no clear hypothesis was made on the effect of nature of task. 1 For the socio-motivational out-comes, it was expected that E C M would lead to more positive attitudes across each of the four categories identified above. No specific hypotheses were made for gender, since the intent was to discover differences that should be addressed in C S C L design, rather than to support an explicit theory. O n the basis of the computer game research discussed in the literature review, however, we might expect males to perform better on the task. Given the goal-oriented nature of the game, we might also expect that males would be more positive than females towards Builder. O n the other hand, the fact that the game centers around communicat ing and working together might lead us to expect a more positive response from females. 7.2 Participants The participants of the study were 134 students from two elementary schools: Queen E l i z a -beth Elementary in Vancouver and Diefenbaker Elementary in Richmond , and had received parental permission for their part icipation in the study. A l l participants were from grades 6 and 7 (10-12 years old). There were 100 students who played the game, 48 girls and 52 boys. The other 34 students were in the control group and therefore completed only the pre- and post-tests. To minimize the confound of sampling from two different populations it was ensured that half of the subjects in each of the cells in the study design came from 1 Regarding game performance, it was not possible to consider differences for the two GOAL conditions because of the different nature of the challenges undertaken in either condition. 67 each school. Pr io r to the commencement of the study, permission forms which explained the purposes of the study were sent home wi th the students to obtain parental consent for their part icipat ion. 7.3 Materials • Hardware: two Power Macintosh computers connected v ia AppleTa lk . • Software: the game Builder. • Tests: wri t ten pre-tests and post-tests containing mathematical questions related to the concepts in Builder, as described in Chapter 5. • Questionnaire: the writ ten form described in Chapter 5, consisting of 20 5-point Liker t scale items assessing socio-motivational outcomes of the study, and three questions on home-computer use. 7.4 Procedure Following is the sequence of steps followed in the study, as administered at each of the schools. • Written mathematical pre-test Several days before the first dyad from each class played Builder, the first of the two academic tests was administered by the teacher during normal class-time. Students were allowed 30 minutes to complete the test. • General orientation Between the pre-test and the start of game-play, a 5-10 minute informal presentation by the researcher was given in front of the class to put the research and game in context. It was hoped that introducing some of the main features of the act ivi ty (building a house using walls, windows and doors; the various 2-D and 3-D views) would help players during the specific orientation and game-play phases. Th i s also served to introduce the researchers to the students in the hope that students would feel more comfortable when they came in pairs to play the game. 6 8 Specific orientation A 5-minute explanation was given by the researcher with each pair in front of the screen of one of the computers immediately before they commenced playing. The function of each of the buttons in the interface was explained and briefly demonstrated. Players were told how to communicate (different for B C M and E C M groups), and the scoring was explained (different for S G M and M G M groups). They were encouraged to complete as many of the challenges as possible, but warned that they would not be able to complete all five challenges during the allotted t ime. Addi t ional ly , in an attempt to enhance the perception of positive goal dependence and individual accountability, dyads were told: "Do you remember the special math test that you did a few days ago...? There wi l l be another test later on. For both of these tests everyone gets their own individual score. B u t in the game [indicate screen], your score wi l l be as a pair. Every th ing is done as a pair - you share bricks, you are working on the same house at the same t ime. Because of this it is good to communicate as much as possible so that your partner knows what you are doing." Partner pre-question After the specific orientation, one student was taken to the room wi th the other computer. Each player was then asked to rate on a 5-point scale how much they liked playing wi th their partner on an everyday basis. Since partners were assigned by teachers either randomly or according to other class act ivi ty constraints, this was to test for the possible confound of some cells ending up wi th more partners who happened to be good friends than other cells. Game play Dyads played the game for 30 minutes, wi th one student and one researcher at each computer. Dur ing game-play researchers silently made wri t ten observations on the nature of play and communicat ion. Assistance wi th the interface was given if players asked a specific question or if they were having technical difficulties (for example, if a player was t ry ing repeatedly to click on a button when s/he first needed to complete a dialog box, the researcher would explain what had to be done). 6 9 Questionnaire A t the end of the time l imi t players were asked to stop and were taken to a separate table to complete the a t t i tudinal questionnaire. If possible, the two participants were seated at different tables or in different rooms. O n the occasions when this was not possible, players were asked to complete the questionnaires silently. A duration of 5-10 minutes was allowed for the questionnaire. Written post-test This was administered on the day after the last students from the class had played the game (for some students this meant up to five days after the playing session). The administrat ion was as described for the pre-test. 70 Chapte r 8 Results The discussion of the results is organized as follows: Dependent Variables Results discussed Achievement (i) pre- and post-test comparison of participants who played the game wi th the control group (n=134) (ii) comparison across levels of C O M M , G O A L and S E X on pre- and post-test and performance in the game (n—100) Sociomot iva t iona l at t i tudes questionnaire data Game-p lay log files; observation forms For simplicity, abbreviations wi l l be used for the independent variables and their levels, as in the previous chapter. The three independent variables of the design - gender, mode of communicat ion and nature of task - wi l l be designated by the terms S E X , C O M M and G O A L respectively. The abbreviations E C M (enhanced communication mode), B C M (basic communication mode), S G M (specific goal mode) and M G M (maximize goal mode) wi l l be used to designate the C O M M and G O A L modes. Unit of analysis In studies involving dyads, it is often unclear whether results should be analysed on an individual or dyad basis. Analys is conducted using the individual as the unit of analysis 71 are preferable in that they allow for greater statistical power due the increase in sample size (i.e., the number of da ta points is doubled by considering each individual separately). However, such individual da ta may violate analytical and statistical assumptions regarding the complete independence of observations. Even though separate scores may be available for each member of the dyad, scores of a given dyad may rise or fall together, making most analyses invalid. In the present study, data obtained regarding performance in the game was, by necessity, analysed using the dyad as the unit of analysis, since players worked together on a single game and received a single score. However, for questionnaire and pre-and post-test data, obtained for each individual , decisions regarding the appropriate unit of analysis were made on the basis of procedures developed in [DG95]. Specifically, the degree of relationship between scores obtained for dyad members was evaluated using Pear-son Produc t Moment Correlat ions. Fol lowing [DG95], if there was a significant correlation (pjO.05, 1-tailed) observed between partner scores on any particular measure, the dyad was used as the unit of analysis, wi th a single score derived from the average of the two players' scores. If the correlation was found to be non-significant, the individual served as the unit of analysis, maximizing statistical power. 8.1 Achievement Outcomes There are two types of achievement outcomes discussed in this section: learning outcomes and performance outcomes. The former refers to the data obtained through the pre- and post-tests of task-related mathematical skills, while the latter concerns performance in the Builder activity, in terms of number of challenges completed and challenge scores. For the pre- and post-tests, in addition to the data from participants who played the game, there were the results of the control group. Therefore the first part of the analysis of academic (pre-post) da ta was a comparison between the control group and all those who played the game. Th i s was followed by comparisons on the same data across the eight groups of the S E X x C O M M x G O A L design, all of whom played the game. 72 8.1.1 Results on Academic Tests Reliability of tests Prel iminary analyses were conducted to evaluate the comparabil i ty and reliability of the tests. F i r s t , it was necessary to ensure that the pre- and post-test measured the same underlying mathematical constructs. A s there was a control group who did only the math-ematical tests, we were able to address this concern by looking at the correlation between their pre- and post-test scores. 1 If there is not a strong relationship between the control group's scores on the two tests, it is likely that the tests are measuring different skills and are therefore not comparable. The result of the Pearson Product moment correlation was sufficiently high to support the pre-post comparison (r(29)=0.74, p<.01, 2-tailed). Another concern was whether it was valid to compare total scores on the tests, rather than clusters of related items. Th i s is important because if the items do not "hang together" it is inap-propriate to use overall test scores as the dependent variable, because we may be averaging over significant differences among sub-groups of items. Internal consistency analyses were performed on the 10 items of each of the tests with the following results: pre-test alpha coejfecient = 0.812 (n=134); post-test alpha coefficient = 0.727 (n=131). 2 A n alpha of 0.6 or above is considered acceptably high for research purposes, hence it is appropriate to use total scores as the dependent variable. "Improvement" scores Learning results were assessed according to the difference between the pre- and post-test total scores, i.e., P O S T - P R E . This difference score wi l l be referred to as the improve-ment score. T w o sets of improvement score da ta were investigated wi th separate analysis of variance ( A N O V A ) tests. F i r s t , scores of those who played the game were compared wi th the control group. Second, each of the eight cells that made up the 2 ( S E X ) x 2 ( G O A L ) x 2 ( C O M M ) design were compared with each other, to ascertain the effect of the different conditions of game play. 1Some practice effect may be expected, but this does not impact on the correlation. 2There were some missing post-test scores due to absenteeism on the day of the post-test. 73 Dependence of observations between partners To determine the unit of analysis for the improvement scores, Pearson Produc t moment correlations were computed for improvement scores for partner 1 and partner 2. A s the relationship was not significant (r(43)=0.22, ns, 1-tailed), suggesting a fair degree of inde-pendence in scores obtained across partners, the improvement scores were analysed on an individual basis. Game group vs. control group A n ini t ia l G A M E (Play, Control) x S E X ( M , F ) A N O V A was performed to assess whether playing the game, irrespective of communicat ion or task mode, led to greater academic improvement than no instruction. Sex was included in the analysis to avoid averaging over an unseen gender difference. Results indicated a significant main effect for G A M E {F(l,126)=8.36, p<0.01), wi th the game group (M=1.22, SD=3.56, N=100) showing greater improvement than the control group (M=-1.23, 3 SD=3.31, N=31). There was no hypoth-esis that gender would have an effect, nor was any effect found. S G M M G M B C M E C M B C M E C M Male M 1.93 2.36 0.0 1.18 SD 3.79 4.62 3.20 4.13 N 14 11 14 14 Female M 1.71 2.43 -0.6 0.23 SD 2.52 3.03 3.19 3.33 N 12 14 10 11 Table 8.1: Academic improvement across three independent variables Within-game comparisons Table 8.1 shows the improvement score means and standard deviations for each of the eight 3 The negative result for the control group indicates that, for this sample at least, the post-test was more difficult than the pre-test. Had the tests been of exactly the same degree of difficulty (as intended), the control group's mean improvement would have been zero, or positive due to practice effect. 74 cells in the S E X ( M , F ) x G O A L ( S G M , M G M ) x C O M M ( B C M , E C M ) A N O V A (now ex-cluding the control group). Inspection of the means across each of the independent variables indicated higher scores for E C M (M=1.58, SD=3.79) than B C M (M=0.84, SD=S.S1), and for males (M=1.32, SD=3.92) than females (M=1.09, SD=3.15), but these main effects were not significant. On ly the G O A L main effect was significant (F(l,92)=8.95, p<0.01), with S G M (M=2.11, SD=3.45) scoring higher than M G M [M=0.28, SD=3.4T). There were no significant interactions. 8.1.2 Game Performance Builder logs information about how dyads perform on the game by recording a score for each challenge completed. The simplest measure of game performance is the number of challenges the dyad successfully completed. Th i s measure alone may not be a satisfactory indicator of game success, however, because successfully completed challenges differ in terms of how opt imal the solution is. In fact, we might postulate that players -who did fewer challenges achieved better solutions precisely because they spent relatively-more t ime on each challenge. Scores were assigned for challenges in the following way. For M G M the score is the area or volume attained, wi th the goal being to maximize the score. R a w scores can be converted to percentages, based on the formula: RAW SCORE / MAXIMUM POSSIBLE SCORE FOR CHALLENGE * 100 For S G M , a successully-completed challenge has a fixed area/volume, so the score is in terms of the amount of bricks used to achieve the fixed goal, wi th the goal being to achieve a low score. These raw scores can also be converted to a percentage using the formula: (INITIAL BRICK COUNT - RAW SCORE) / (INITIAL BRICK COUNT - MINIMUM POSSIBLE SCORE) * 100 The two scoring methods were different in terms of both the aspect of the game they related to and how much importance was placed on them. In S G M , minimiz ing the number of bricks used is a secondary goal, whereas in M G M maximiz ing area/volume is the pr imary goal. Therefore it was not appropriate to compare across G O A L modes, so results were analysed 7 5 separately for S G M and M G M . Players in M G M in fact completed fewer challenges overall than players in S G M (2.04 vs. 2.38). B C M E C M M a l e M 3.29 2.67 SD 0.76 0.52 N (pairs) 7 6 Female M 1.83 1.71 SD 0.75 1.25 N (pairs) 6 7 Table 8.2: Number of challenges completed across S E X and C O M M for S G M players B C M E C M M a l e M 2.29 2.29 SD 0.76 0.76 N (pairs) 7 7 Female M 1.5 2.0 SD 0.84 0.00 N (pairs) 6 6 Table 8.3: Number of challenges completed across S E X and C O M M for M G M players A series of two A N O V A s (2x2) were performed, one for S G M and one for M G M means, as presented in Table 8.2 and Table 8.3, to compare the performance across S E X and C O M M variables. Results for the S G M group (Table 8.2) indicated a significant main effect for S E X (F (1,22)=12.07, p<0.01), wi th males (M=3.00, SD=0.71) completing more challenges than females (M=1.77, SD=1.01). There were no significant effects for the M G M group, but the results indicated a similar trend (F (1,22)=3.9 4, p=0.06) wi th males (M=2.29, SD=0.73) completing more challenges than females (M=1.75, SD=0.62) The challenge scores were also examined separately for the two different goal groups. The data analysed was the best score that a dyad had attained across all challenges played. Choosing a dyad's worst or average score would be problematic because the pair may have run out of time at a point when they had attained an unrepresentatively low score for them. Aga in two 2x2 A N O V A s were performed to compare across S E X and C O M M . None 76 of the results were significant at the 0.05 level, though for the S G M condition there was a trend towards males (M=81.85, SD=25.19, N=13) scoring higher than females (M=58.56, SD=39.35, N=13) [F (1,22)=2.88, p=0.10). 8.1.3 Relationship Between Game Performance and Academic Gain No. of chals (N) Best score Improvement Pre-test 0 ( 3 ) 0.00 1.83 4.00 1 ( 4 ) 96.60 3.75 7.12 2 (17) 58.04 1.85 6.65 3 (20) 82.69 1.82 8.50 4 ( 6 ) 96.27 2.83 8.08 Table 8.4: Challenge score, improvement and pre-test as a function of number of challenges completed for S G M ••; No. of chals (N) Best score Improvement Pre-test 0 ( 1 ) 0.00 -2.00 8.50 1 (6) 93.11 -1.83 9.50 2 (32) 88.67 0.10 9.50 3 (12) 82.30 1.67 8.71 Table 8.5: Challenge score, improvement and pre-test as a function of number of challenges completed for M G M Improvement and number of challenges completed Tables 8.4 and 8.5 show how three other measures vary according to the number of chal-lenges completed. The pr imary question addressed was whether academic improvement was a function of performance, wi th performance being measured in terms of number of challenges. Aga in , due to the fact that the two G O A L conditions were not comparable for performance measures, results were analysed separately for S G M and M G M players. A s seen in the tables, results suggest that there is litt le relation between number of challenges completed and improvement for players in S G M , but that for M G M the improvement ap-pears to increase along wi th number of challenges completed. Pearson Produc t moment correlations were performed to test these relationships, and it was found that there was no 77 relationship for S G M (r(48)=-0.12, ns, 2-tailed), but that there was a significant positive correlation for M G M {r(48)=0.31, p<0.05, 2-tailed). Challenge scores and pre-test totals The two other scores given in Tables 8.4 and 8.5 are the best challenge score and the pre-test to ta l . We were interested, first, in whether there was a relation between challenge scores and improvement, which correlations indicated was not the case for either G O A L mode. The second question regarding challenge scores was whether they were related to the other per-formance measure of number of challenges completed. A s Table 8.4 indicates, a significant correlation was observed between challenge scores and number of challenges completed for S G M players (r(24)—0.56, p<0.01, 2-tailed4), indicating that those who completed more challenges also did well wi th in at least one of the challenges they completed. For M G M players there was a similar positive but non-significant correlation. The fact that both of the correlations are positive discounts the possibility that doing more challenges may have led to a less opt imal performance wi th in challenge. Regarding the pre-test data, we want to know if prior abil i ty in the task-related area predicted performance in Builder. A s suggested by the da ta presented in Tables 8.4 and 8.5, there was no correlation between pre-test total and number of challenges for either of the G O A L groups. There were, however, significant correlations between pre-test scores and challenge scores both for S G M {r(49)=0.34, p<0.05) and M G M players (r(49)=0.36, JXOM). Pre-test and improvement A n important complexity wi th the improvement scores was also discovered in that there was a strong negative correlation between pre-test and improvement scores {r(98)=-0.62, p<0.01). Th i s may explain why achieving a good within-challenge score does not lead to a high improvement score. T h a t is, those who went well did so part ly because they had a good mastery of the domain, but their mastery prevented them from improving as much as those wi th less mastery in the domain . Since the highest scores on the pre-test approached 4 T h e low number of observations is due to the fact that this correlation was conducted using the dyad as the unit of analysis, since both the measures are for game performance. 78 100%, the lack of observed improvement in proficient students may also be explained by ceiling effects. 8.2 Sociomotivational Outcomes Before looking at the questionnaire data, we analysed the results of the pre-game question, which asked players to rate on a 5-point scale how much they liked (playing with) their allocated partner. A S E X ( M , F ) x G O A L ( S G M , M G M ) x C O M M ( B C M , E C M ) A N O V A revealed no significant main effects or interactions at the .05 level. Th i s result alleviates the concern that some cells in the design may have been composed of closer friends than other cells, which could otherwise confound the results on atti tude to partner. The investigation of the questionnaire data began with a factor analysis on the 20 Likert-style att i tude items contained therein. 5 The purpose of a factor analysis is to increase the robustness of the dependent measures by suggesting possible groupings (factors) of items according to their statist ical relationships across observations. For the factors to be useful in analysis they must also be conceptually related. It is expected that this wi l l be the case given that the items were designed to assess a part icular set of constructs. The Pr inc ipa l Components Analys is produced six factors wi th eigenvalues over 1.0, a typical selection cri teria for factors worthy of further analysis. Three of these factors were excluded on the basis of investigation of the Scree P lo t , which tends towards a straight line after the th i rd factor, suggesting that only the top three factors accounted for sufficient variance to be worth considering. Factors 1, 2 and 3, respectively, accounted for 27.1%, 11.8% and 9.4% of the variance, making a to ta l of 48.3% accounted for. Below are presented the factor loadings of each item for the three factors, as well as the mean i tem scores. Recal l ing that a "yes" is 4.0 on the scale, and a " Y E S " is 5.0, inspection of the means of Table 8.6 indicates players reported generally positive attitudes. The highest rated items were: Q I ("I enjoyed playing Builder", M=4-55), Q8 ("I would like to play Builder again", M=4-55) and Q17 ("I wish I could have played Builder for longer", M=4-43). A l though positive attitudes are good, the high means and relatively low standard deviations raise 5The final three questions of the questionnaire were omitted as they did not relate to the game, but to the respondent's everyday patterns of computer use. 79 Factor 1 Factor 2 Factor 3 Mean SD QUEST1 [enjoy game] .67525 .47009 .00525 4 55 0 57 QUEST10 [home game] .58433 .58240 -.02327 4 15 0 98 QUEST12 [again partn] .65602 -.25319 .44500 3 69 1 11 QUEST13 [other partn] .42513 -.09363 .48746 3 78 1 16 QUEST14 [ l i k e comm] .61700 -.22693 -.27850 3 92 0 81 QUEST16 [same room] .37022 -.20292 -.46613 3 69 1 35 QUEST17 [play longer] .45604 .45428 -.28802 4 43 0 81 QUEST18 [easy comm] .58103 -.37145 -.52007 4 02 0 89 QUEST2 [ l i k e comp] .26929 .38001 .14002 4 39 0 62 QUEST19 [prefer alone] .55895 -.24421 -.03429 4 29 0 93 QUEST20 [comm helped] .63982 -.27900 .13648 4 46 0 84 QUEST3 [ f r i end partn] .40344 -.16561 .47171 4 36 1 00 QUEST4 [school game] .39073 .50203 .04354 4 20 0 97 QUEST5 [ d i f f i c comm] .45664 -.34204 -.44847 3 78 1 20 QUEST7 [partn game] .64526 -.13592 .14240 4 43 0 81 QUEST8 [again game] .64868 .61452 -.16597 4 55 0 71 QUEST9 [help partn] .49936 -.32521 .43879 4 02 1 .09 QUEST6 [easy game] .15958 -.00837 -.02034 3 54 0 .87 QUEST15 [more i n s t rn ] .53028 -.32543 ' -.24217 3 92 1 .13 QUEST11 [ learn game] .54092 .06815 .20106 3 81 0 .95 Table 8.6: Factor loadings and means of questionnaire items 80 the concern that scores wi l l not discriminate between conditions well because there is not a large enough range of results. It could be hypothesized that the high scores were part ly a result of respondents t ry ing to please researchers. Inspection of the loadings onto Factor 1 indicates that over half of the items are above the typical cri teria of 0.4, possibly due to the uniformness of answers discussed above. It would be difficult to conceptually interpret such a large group of items. Factors 2 and 3, on the other hand, provide sets of appropriately related items. The items loading onto Factor 2 were: Q I ("I enjoyed playing Builder1''), Q10 ("I would like to play Builder at home"), Q17 ("I wish I could have played Builder for longer"), Q4 ("Computer games like Builder should be used more in school"), and Q8 ("I would like to play Builder again"). These can be sensibly grouped under the heading persistence of interest in activity, one of the target areas of the questionnaire. The items loading onto Factor 3 were: Q12 ("If I play Builder again I would like to play with the same partner"), Q13 ("I would enjoy playing other games wi th the same partner"), Q3 ( " M y partner was friendly") and Q9 ( " M y partner was helpful"). Therefore this factor can be considered as measuring affect towards partner. G iven that factors 2 and 3 address two of the four areas that the questionnaire aimed to assess, Factor 1 was considered in relation to the other two areas: attitude towards the game and perception of collaboration. Q14 ("I liked communicat ing with my partner using the computer") , Q18 ("It was easy communicat ing wi th my partner using the computer") , Q19 ("I would prefer to play Builder alone" [reverse scored]), Q20 ("Communicat ing wi th my partner helped us play the game") and Q7 ("I like playing computer games wi th a partner"), were all amongst the highest of the loadings on Factor 1. Accord ing to purely statist ical cri teria, the other items that loaded highly onto Factor 1 should be included. However, so that the group of items was interpretable, al l those items that cross-loaded onto Factor 2 or Factor 3 were excluded, as were those which did not relate to atti tude towards communicat ion, based on conceptual grounds. Given these mixed cri teria for inclusion of Factor 1 items, it was decided to consider separately one item relating to the perception of collaborat ion. A t the time of developing the questionnaire it was felt that responses to Q16 ("I would prefer to have my partner in the same room") might be part icularly interesting, part ly because the item addresses the central C S C L / H C I question of s imulat ing real collaboration 81 in a vi r tual setting, but also because the wording makes the item less open to the problem of respondents t ry ing to please researchers (at least compared to items like Q14, "I liked communicat ing using the computer") . The fact that the mean was relatively low for the question also suggested that it may be somewhat more useful as a distinguisher. Therefore Q16 was included separately in the analysis. For the items returned for Factors 1, 2 and 3, a total score was computed as the aver-age score received across items included in each factor. Higher scores always indicate a more positive att i tude. A series of four S E X ( M , F ) x G O A L ( S G M , M G M ) x C O M M ( B C M , E C M ) A N O V A s were performed for Q16 and each of the three factors. Pr io r to this, the relation-ship between partners ' scores on each of the measures was considered to decide whether the analysis should be conducted at the individual or dyad level. Perception of collaboration (Factor 1) Results of the correlational analyses revealed a significant relationship (r(48)=0.34, p<0.05) between partners on Factor 1, so the dyad was used as the unit of analysis. There were no significant main effects in the A N O V A , but there was an interaction between S E X and • C O M M (F(l,42)=4.70, p<0.05). A s suggested by the means of Table 8.7, p'ost-hoc analyses (Tukey) revealed that females were significantly more positive when in E C M than in B C M (F(l,49)=4-869, p<0.05), whereas the two conditions d id not differ significantly for males. B C M E C M M a l e M 4.33 4.15 SD 0.57 0.46 N (pairs) 13 12 Female M 4.08 4.5 SD 0.62 0.39 N (pairs) 12 13 Table 8.7: Means for Factor 1 (Perception of Collaborat ion) showing S E X * C O M M interac-tion Persistence of interest in game (Factor 2) Factor 2 scores did not show a significant correlation between partners (r(48)—0.22, ns), so the unit of analysis was the individual player. Results of the A N O V A revealed a significant 82 main effect for S E X (F(l,91)=4.00, p<0.05), wi th males {M=4-43, SD=0.54) scoring higher than females(M=^J7, SD=0.87), and a significant G O A L * S E X interaction (F(l,91)=5.22, p<0.05). Post-hoc analyses (Tukey) revealed that males in S G M showed significantly greater persistence of interest than females in S G M (F(l,46)=7.43, p<0.01), but there was no par-allel gender difference in M G M (see Table 8.8). S G M M G M M a l e M 4.64 4.27 SD 0.30 0.63 N (pairs) 23 26 Female M 4.05 4.31 SD 0.95 0.66 N (pairs) 26 24 Table 8.8: Means for Factor 2 showing S E X * G O A L interaction Attitude to partner (Factor 3) The correlation between partners' scores on Factor 3 indicated that the dyad should be the unit of analysis (r(48)=0.41, p<0.01). Results of the A N O V A again revealed a significant main effect for S E X (F(l,42)=4.78, p<0.05), wi th females (M=4-05, SD=0.46, N=25) being more positive towards their partners than males (M=3.75, SD=0.97, N=25). There was also a significant G O A L * C O M M interaction (F(l,42)=5.78, p<0.05) for which none of the post-hoc tests were significant at the 0.05 level. A notable trend which approached significance was the difference between the two G O A L modes wi th C O M M held constant at B C M (F(l,23)=3.46, p=0.07), indicat ing that given basic means of communicat ion, M G M players were less positive about their partners than S G M players (see Table 8.9). Same room (Question 16) Individual scores were the unit of analysis for Q16 as there was no significant correlation between partners (r(48)=0.08, ns). The A N O V A revealed a significant main effect for C O M M (F(l,92)=4.19, p<0.05), wi th those in E C M (M=3.96, SD=1.19, N=50) scoring higher than those in B C M (M=3.38, SD=1.48, N=50). The intent of this question was to 83 S G M M G M B C M M 4.18 3.63 SD 0.46 0.90 N (pairs) 12 13 E C M M 3.85 4.14 SD 0.58 0.60 N (pairs) 12 13 Table 8.9: Means for Factor 3 showing G O A L * C O M M interaction ascertain how frustrated players became by not being able to communicate face-to-face. It was reverse-scored because a high desire to have the partner in the same room indicates a negative atti tude towards the communicat ion. Hence the results indicate that those in E C M felt less frustrated in their attempts to communicate than those in B C M . 8.3 Game-play One of the main points of interest in relation to game-play was the communicat ion between players. Th is section presents data on the volume of communicat ion between players of different conditions, as well as some anecdotal examples of interesting features of commu-nication and other aspects of game-play. Number of wri t ten messages sent Males: 27.34 Females: 28.44 SGM: 28.45 MGM: 27.31 BCM: 35.6 ECM: 20.02 Table 8.10: Wri t t en message count across S E X , G O A L and C O M M A m o u n t of sound da ta sent (in Kilobytes) Males: 175.5 Females: 183.6 SGM: 157.6 MGM: 200.8 Table 8.11: Volume of spoken communicat ion across S E X and G O A L 84 Tables 8.10 and 8.11 present data from the log files regarding number of wri t ten messages and volume of spoken data sent between partners during game-play. Table 8.10 includes the means for the two C O M M groups, showing that B C M players wrote more mes-sages to each other than E C M players. Given that there is less need to write messages if players have speech, this difference is to be expected, and C O M M was not included in the A N O V A s . A second expectation, given the findings in the computer game literature that fe-males are more interested in games that involve relationships and communicat ion, was that females would communicate more than males, which does not appear to be supported by the means. The other hypothesis wi th regard to communicat ion was that a less-structured task ( M G M ) might lead to more communicat ion. The means for speech da ta show some support for this hypothesis, with M G M players sending an average of 2 0 0 K B per session compared to S G M ' s 1 5 8 K B . S E X ( M , F ) x G O A L ( S G M . M G M ) A N O V A s , however, revealed no significant main effects or interactions in either the wri t ten or the spoken data . One of the more interesting points of the communicat ion data analysis is that males and females : differed so li t t le on these measures. Th i s was part icularly surprising in the light of obser- ' vations that suggested females were more interested in the communicat ion than males. A ' J • tentative interpretation of the lack of difference is that the challenge of the game itself led to males communicat ing more, even though they did not appear to be as interested in the communicat ion as in completing the challenges. Observation Forms D a t a from the observation forms is important because it provides information not obtainable from the wri t ten tests, questionnaires or the log files. For example, there was no indicat ion in the log files of what players in E C M talked about. There was also no way to ascertain how comfortable players were with the interface. A l though data from the observation forms is subjective and variable in level of detail , it is helpful when used in light of the more formal results - part icularly if observations can be used to suggest explanations of the other results. The observations recorded included the following types of information: • whether players had trouble wi th the interface; • how well players worked together; 85 • how focussed players were on the task; and • whether players had a good sense of the concept. Several players were observed to be either part icularly adept or inept wi th respect to the interface of Builder. There was no noticeable tendency, however, for any part icular group to fall into either of these categories. In particular, there was no evidence from the observations that girls were any worse at adapting to the interface than boys, which might be expected based on the reported gender differences in computer proficiency discussed in the literature review. Concerning the observed degree of collaboration, more male pairs were noted to have collaborated ineffectively than female pairs, while there was no difference in which pairs were noted to have collaborated well. Comments from the observation forms regarding male pairs collaborat ing poorly included: "didn' t seem to be working together", "poor communicat ion wi th partner", etc. M o r e male pairs were noted to have shown a strong grasp of the concept or to be part icularly focussed on achieving the goal. It was also noted that more players in M G M had trouble wi th the concepts than players in S G M . 1 ' 8 . 4 Reported Home Computer Use The final three items on the questionnaire were open-ended questions regarding the part ici-pants' home computer use. There were two motivations for asking these questions. The first was to gather descriptive data about computer use amongst school children of the target age group, and the second was to compare use across gender. The other, manipulated in-dependent variables of the design are not relevant for these questions, since home computer use is outside of the context of the study. Results for the three questions were as follows. • N u m b e r of computers at home: The overall mean for this i tem was 1.65 com-puters (N=99, SD=0.93). On ly three respondents reported having no computers at home. 6 The most common responses were one computer (49 respondents) and two 6It should be noted that, in addition to the three respondents who reported zero computers, there were three missing values, which may have been intended to indicate zero. 86 computers (30). A slightly higher number of computers was reported by females (M=1.72, SD=0.94, N=48) compared to males (M=1.59, SD=0.92, N=51), though this was not significant according to a t-test (t(97)=-0.70, ns). Hours per day spent on computer (at home): The average amount of t ime spent using a computer per day was 1.13 hours (N=101, SD=:0.92). The most com-mon responses were one hour (30), half an hour (15), two hours (14) and 1.5 hours (12). There was a non-significant trend for males (M=1.28, SD=1.07, N=52) to spend longer on the computer than females (M=0.97, SD=0.70, N=49) (t(99)=1.69, p=0.09). What respondents used the.computer for: Th i s i tem produced an incredible range of responses, including over 50 different games. Other than games, respondents reported using computers for typing, painting, homework, wr i t ing letters and email , using the internet, and using some other form of reference material , mostly electronic encyclopedias. Reports of " typing" , where the exact purpose of the typing was not specified, were counted in the same category as doing school homework or, projects. Table 8.12 indicates the number of males and female respondents who reported com-puter use wi th in each of the usage categories. The results for non-game uses are presented first. To summarize the game responses we decided to present results only for the games mentioned by more than two females or more than two males. Aside from the games presented in the table, a casual glance at the names of other games listed by males and females supports the general dist inct ion. M o s t of the games listed by male respondents were either sports-related (e.g., Lakers vs. Celt ics , Michea l Jordan In F l igh t , N B A ) or were among the dizzying array of fantasy/battle-oriented games available on the market (e.g., T i m e Commando , Dune 2, Star Wars , W i n g Commander , Crusader, A l p h a Bat t le , Mech Warr ior , Quake, Hellbender, etc.). The games listed by girls tended to be adventure-oriented or based on popular television or movie themes (e.g., Carmen Sandiego, A l l a d i n , L i o n K i n g , P u m b a and T i m o n ' s Jungle Games, Treasure Cove, Simpsons, D i n o Park , Dick Tracy, Theme P a r k ) . There was a good deal of gender overlap on certain types of games, most notably card games and S imCi ty , but the expected imbalance wi th regard to violent action games is clearly 87 Females Males Homework 26 28 Letters 2 0 Stories 2 1 E m a i l 4 1 Encyclopedia 5 6 Internet (general) 17 13 Internet (chat) 3 1 Pain t ing 6 2 C a r d games 12 6 Tetris 6 1 P inba l l 5 3 JezzBa l l 4 3 Y u k o n / A m a z o n Tra i l 3 0 C C Red Ale r t 1 15 S i m C i t y 4 8 D o o m 1 7 N H L 97 1 7 Warcraft 0 6 Duke-Nukem 1 6 C a r racing 2 5 G o l f 1 3 table b.1'2: Uses ot computer across gender demonstrated, part icularly wi th respect to C o m m a n d and Conquer Red A le r t , D o o m and Duke-Nukem. 7 7Referring back to the discussion in the literature review, Duke-Nukem is a prime example of a male-biased game, particularly in terms of its incidental objectification of women. 88 Chapter 9 Discussion 9.1 Summary of Findings 9.1.1 Achievement '.. ' To begin wi th the academic outcomes, the results revealed two significant effects. F i r s t , in the control-group comparison, it was found that pairs playing the remote C S C L act ivi ty Builder showed greater improvement in the target mathematical areas than students who received no instruction. Second, it was found that the manipulat ion of the goal specification wi th in the act ivi ty affected the degree of improvement. Specifically, players who were given a clear, numeric area or volume goal showed greater academic improvement than players who were given the more open-ended goal of maximizing the area or volume. Results for the second manipulated variable, mode of communicat ion, failed to show that enhancing com-munication wi th spoken messages and v i r tua l presence led to greater achievement. There were no significant gender differences on academic improvement, but it was found that males completed more challenges and scored higher in the act ivi ty than females. There was a positive relationship between number of challenges completed and academic improvement, but only for players in maximize goal mode ( M G M ) . 89 9.1.2 Attitudes There were several communicat ion- and gender-related differences found in the sociomoti-vational outcomes. Males showed a greater persistence of interest in the game than females, suggesting that the act ivi ty was more motivat ing for males. Th is gender difference in at-ti tude to the game, however, was not uniform across modes 'of task, as the analysis of the interaction wi th the task mode revealed that males showed significantly greater persis-tence of interest than females only when in specific goal mode ( S G M ) . In terms of perceived collaboration, results indicated that females were more positive about the communicat ion when they had speech and vi r tua l presence ( E C M ) , while for males this did not effect their att i tude. Furthermore, responses to the specific i tem "I would prefer to have my partner in the same room" showed the expected result that players wi th enhanced communication ( E C M ) found the remote collaboration less frustrating than players wi th basic communica-tion ( B C M ) . Final ly , in terms of the social measure of interpersonal att i tude, it was found that females were more positive about their partners than males. Given that there was no parallel gender difference on the pre-game rat ing of att i tude toward partner, the results imply that the game had a positive effect on interpersonal attitudes (as expected from the Cooperat ive Learning literature), but only amongst females. In terms of the manipulated variables, S G M generated a more positive att i tude to the partner than M G M when players were in basic communication mode ( B C M ) . Anecdota l observations supported the performance findings in that more males were observed to be strongly focussed on the task than females. There was also anecdotal evidence of some females being more interested in communicat ing than in the task, compared to males, and there were more observations of males communicat ing ineffectively than females. Players in M G M were observed to have more trouble with the game than players in S G M , which may help explain the achievement findings. 90 9.2 Relating Findings to the Literature 9.2.1 Remote Collaborative Learning? W h e n introducing the motivations for the study, two categories of interest were identified, the first of which was the question of whether the positive results of Cooperat ive Learn-ing can be facilitated in a remote computer-supported setting. A s previously stated, the design of the current study did not at tempt to answer this question in terms of whether re-mote C S C L is better than individual C A L , co-present C S C L , or non-computer Cooperat ive Learning. W h i l e these questions could foster interesting follow-up research, the position advocated in the current study is that such comparisons may be ungrounded in terms of the educational goals of the different methods, and may also be difficult to investigate ex-perimentally. Nevertheless, on the basis of the control group comparison and some of the descriptive data presented in the results, several observations can be made regarding this broad question. F i r s t , there is evidence that the Builder act ivi ty led to significant learning gains in the target mathematical areas. W h y did Builder produce this improvement? One explanation is that players were able to rehearse existing knowledge by applying it to the problems presented in the activity. Given the collaborative nature of the game, it may also be that players improved because they had to verbalize their problem-solving strategies. Such verbalization, and consequent rehearsal and reappraisal of learning constructs, has been found to be the pr imary agent of academic improvement in C L . Based on the current evidence, it is not clear whether the task itself or the collaboration played the greater role in improvement. Looked at from the learners' perspective, however, the collaboration appeared to play a central role. Th is was evident in that the th i rd highest-ranked item in the questionnaire (with a mean of 4.46) was "Communica t ing wi th my partner helped us play the .game". Second, the results of the a t t i tudinal questionnaire suggest that Builderh&s a positive effect in non-academic areas. Those who played the game appeared to find it substantially mot ivat ing and engaging; as indicated by the two highest-ranked items on the questionnaire, which were "I enjoyed playing Bui lder" and "I would like to play Bui lder again" (each wi th 91 a mean score of 4.55 on the 5-point scale). The positive mood evoked by the game may have transferred to the isometric act ivi ty of the post-test, providing an alternative explanation for the improvement seen in the game-group. The magnitude of this general positive atti tude toward the game is difficult to ascertain, since we are not comparing those who played the game with those who performed some other activity. There is also the possibility that other aspects of the study could have contributed to the positive responses; one example being that students may have enjoyed the variation from normal classwork, and that given the short length of play there was not long enough for the ini t ia l novelty of the game to wear off. Further studies within slightly different contexts, such as having students play Builder in their lunch break everyday for a week, would help confirm the degree of generated engagement. Nevertheless, the positive results discovered were what we expected, given that the act ivi ty was designed to be game-like (i.e., more like play than work) , and they were sup-ported by anecdotal observations, both in the final study and the pilots. It was noted that communicat ion was a substantial component in students' appreciation for the game, wi th those who had only B C M often expressing disappointment when they later found out that other players could talk to each other. W h i l e it was found that almost all players enjoyed the communicat ion, there were more noticeable individual differences in how absorbed dif-ferent players became in the act ivi ty itself. Several players - especially, but not only, boys - worked almost feverishly on the challenges during their session, and expressed a desire to play Builder again during break time. There were a few players, however, who seemed either confused by or relatively uninterested in the act ivi ty - this was more often the case for players in M G M . In terms of the varying attitudes toward Builder, the results of the cur-rent study are i l luminat ing mainly in regard to gender, as is discussed further below. These addit ional anecdotal observations indicate that it may be worthwhile to further investigate different types of learners to decide who "profits" most (both in terms of engagement and achievement) from a specific C S C L tool . A non-tr ivial challenge that must be addressed by researchers in the field is that games and collaboration should involve and stimulate interest in learners who are not already proficient in or motivated by the target learning area. Such research requires extensive profiling of the learner, beyond simple distinctions 92 such as gender or academic level. In summary, the academic control-group comparison and descriptive da ta on general attitude to the task indicate that Builder may be quite effective in the role for educational games proposed by [KP95] - that of s t imulat ing interest wi th in and support ing a broader instructional environment. The findings are also an encouraging step towards incorporat-ing computer-supported collaboration into Distance Educa t ion . Future study should assess what factors mediate engagement across different types of learner, so that alternative in-terface or task elements can be considered to include a wider range of learners. 9.2.2 The Role of the Task The most important variable manipulated wi th in the act ivi ty in terms of academic improve-ment was the nature of the task. Given the apparent conflict between the current results and the success of i l l-structured tasks in Cooperat ive Learning [Coh94, J M J + 8 1 ] , we need to consider why the clearly-structured goal had more effect on learning. A n ini t ia l consid-eration in response to this conflict is that the S G M task, though having a well-defined goal, was st i l l i l l-structured to the extent that there are many possible solutions to a challenge and many decisions that players may wish to discuss. The remainder of this section suggests other explanations for the success of S G M . Differences between the two task modes that may be responsible for the results include: the fact that S G M has a clearer goal; the direct feedback in S G M ; and the relative difficulty of performing opt imal ly in M G M . Hav ing a clear goal has been found to be the most important deciding factor in the populari ty of games [Mal82], hence it may have been more engaging for learners. Immediate and simple computer feedback has also been found to be more motivat ing than less-tangible, self-regulated feedback in co-present C S C L [NC93]. M o r e specific features of the current context may also explain the results. If M G M is a harder activity, some players may have failed to grasp the concepts, or at least had trouble doing so during the short playing t ime. There was support for this in anecdotal observations made by the researchers that more players in M G M appeared to be struggling or confused by the game. F r o m observations of others who have played the game outside of the formal study, it would appear that for older players M G M is more challenging and interesting than 93 S G M . The direct feedback of S G M may also be less important for adults ([NC93]'s study was also with elementary-level students). The relevance of age is supported by comparison wi th [BTR + 95] 's mult i- input C S C L study of high-school and college students using a simple act ivi ty with a clear goal. Col laborat ion had no significant learning effect, and many players in the college student sample said that they would prefer to play alone. The type of act ivi ty used by [BTR +95] may work better with a population of younger learners, while tasks such as M G M in Bui lder may work better wi th older learners. Future research may benefit from addressing such developmental issues. The task results may also be domain-dependent. In S G M there is a greater need for players to be focussed on the numbers, and this emphasis on detail may be part icularly beneficial in mathematical learning. There is a need for further research to ascertain the effect of ill-structured goals in other domains (such as the languages and social sciences). To conclude, the results obtained indicate that wi th in a short time period, the use of a C S C L act ivi ty with a specific goal can have a postitive effect on task-related learning in mathematics for young learners. 9.2.3 Supporting Interaction O f equal interest to the significant effect found for the goal manipulat ion itself is the fact that it was more influential than the mode of communicat ion. It was expected, based on the elemental role of learner interaction in Cooperat ive Learning, that varying communicat ion would have a greater impact than varying the specificity of the task. The straightforward interpretation of the lack of effect is that learners found they could communicate as well wi th wr i t ing as they could wi th speaking. To further i l luminate the role of communicat ion in the remote C S C L setting, a follow-up study comparing spoken and writ ten communicat ion wi th no verbal communicat ion is necessary. In the latter condit ion, players would be forced to rely purely on aspects of the visual interface in order to collaborate. A n alternative explanation for the equivalence of E C M and B C M in the academic results is that students did not feel comfortable using the speech because it was embarrass-ing. Th is reason was volunteered by some students, either spontaneously or when asked why they used speech sparingly. Furthermore, it may be that learning improvements emerged 94 more out of the task itself than the communicat ion between players, which could be ad-dressed by comparing the current outcomes with those from a single-player version of the game. Al though the different modes of communicat ion did not affect academic improve-ment, they had significant effects on the sociomotivational outcomes, a result also expected from the C L literature. Responses to the "same room" question indicated that E C M made it significantly easier for players to collaborate remotely. Females in E C M were also more positive about the collaboration in general than those in B C M . Enhancements to the com-munication were therefore important in terms of perceived collaboration. The interaction of the effects of task and communicat ion in the atti tude data is suggestive of the role different modes of communicat ion play in different types of C S C L activities. It was found that players in M G M were less positive about their partners if they had only basic communicat ion, suggesting there may be more of a need for enhanced communicat ion wi th in less-structured activities. W i t h i n a certain range of domains (e.g. mathematics) and learning activities (e.g. well-structured), wri t ten communication may be as effective as multiple channels of communicat ion. O n one hand we can see these results as positive indications for distance education projects that use simple forms of communicat ion, such as email, bulletin boards or chat facilities. These styles might be quite sufficient for collaborative learning wi th in certain domains and types of task. O n the other hand, the results suggest the need for further work in enhancing communicat ion to support less-structured tasks. Furthermore, the at t i tudinal results suggest that to create a subjective environment of collaboration and positive regard for co-learners, enhancing communicat ion wi th spoken communicat ion and /or v i r tual presence may be benificial. 9.2.4 Gender Differences W h i l e there were no significant gender differences in academic improvement, it was found that males completed more challenges in Builder, and had different attitudes towards the game, their partners, and collaboration in the game. The fact that males showed a greater persistence of interest in the game than females, part icularly when in S G M , supports find-ings from previous studies indicating that goal-focussed games wi th record scores are more 95 interesting to boys than girls [ O K d V 9 6 ] . The alternative hypothesis, that the prominent role of communication might make the game more interesting to girls, was not supported by the results. Other att i tude measures discussed above also showed gender differences. F i rs t , females were more positive about the communication when they had speech and vi r tual presence, while for males this d id not affect their att i tude. Second, in terms of the social measure of interpersonal affect, it was found that the game had a more positive influence on females than males. A general way to summarize these findings would be to say that males responded well to having a specific goal, while females responded well to being able to speak to and/or see an image of their partner. Th i s was supported by the anecdotal observations of males being more strongly focussed on the task and females being more interested in communicat ing. Findings such as these gender differences can be used as guidelines in the design of learning activities like Builder to ensure the inclusion of elements that are effective for different types of learners. 9.3 Future Research The current results suggest guidelines for future research to further elucidate the important elements in a remote C S C L setting. One suggestion is to investigate whether the positive effect of a specific goal and the lack of effect of enhanced communicat ion is replicable in different domains, wi th learners of different ages, and over longer periods of instruct ion. T w o specific questions that have emerged in relation to learning are: • Is there a set of learners, a domain, and/or a durat ion over which less-structured goals show greater academic gain in remote C S C L ? • Does enhancing communicat ion wi th in the less-structured setting influence the size of the gain? W i t h respect to the gender results, the most important consideration is whether there is a real benefit in different styles of communicat ion and task for different types of learners. One interesting approach to this would be to create an entirely user-configurable learning environment, such that users can choose both how structured their goal is and what type of 96 communicat ion channels they use. Such an open-ended investigation would help define the set of elements wi th in remote C S C L that are valuable across the range of possible learners. Interpreting the results in the context of related work suggests a wide range of direc-tions that could be investigated in further studies wi th the Builder activity. For example, we could study Builder when played wi th in the broader, M U D - l i k e setting of Island. Th i s would involve players making more decisions about how to spend their t ime, which might require more or different types of interaction. W i t h i n a large game of Island the notion of group decisions or politics could be introduced. The more decisions that have to be made, the more opportunity there is for divergent ideas and conflict, factors claimed to be powerful agents in cognitive change by Nastasi and Clements. Modes of interaction could be manipulated within such a setting, as a supplement to the study of the lower-level question regarding media of communicat ion. Elements of the interaction could also be used as dependent measures, and again the observational assessments used by Nastasi and Clements [NC93] could provide important information supplemental to the learners' questionnaire responses. P lac ing Builder wi th in Island also provides a different type of motivat ion to perform well in the activity. A player's house would be displayed to all the other players in the broader game, and might also have to withstand weather or other hazards. It may be that some learners are more motivated by such concerns. Th is enriching of the fantasy aspect of the game is in accordance with the work of Malone . Further studies could alternatively concentrate on the aspects of the interface that support learner interaction. The investigation of vi r tual presence, for example, could be extended to allow a l imited set of symbolic gestures, which could be compared wi th the provision of a more open-ended tool for gesturing (such as manipula t ing simulated 3-D hands). How users choose to represent themselves in a fantasy-type v i r tua l environment is also an interesting research question, and one that is t imely in the context of the emerging cyberspace technology. In the current study some gender differences were observed in type of icon chosen, but this could be further explored by allowing a much broader range of choices, the adoption of names, or even allowing players to create their own avatars. The anonymity and possibility for identity creation may lead to interesting types of collaboration 97 not found within co-present models. Appendix A Implementation Details CodeWarr ior (versions 9 and 10) was used as the compiler and debugger. ResEdi t was used for handling resources (pictures, sounds, etc.). The free-ware l ibrary Spr i teWorld was used for many of the graphical objects. Th is was originally wri t ten by Tony Myles , but has been updated by K a r l Bunker and Vern Jensen (official internet site: http://members.aol.com/ Spr i t eWld2 /index.html). Bre t t A l l e n , an E - G E M S programmer, wrote an add-on to this l ibrary called Spr i teCan, which was also used in Builder. Thanks are also due to Bre t t for help with trouble-shooting, part icularly wi th Sprite World-related issues. Other credits are due to the following E - G E M S programmers. Greg Smolyn wrote the Client and Server classes for the overall game Island (this code was adapted from the previous Island driver wri t ten by Hardeep Sidhu and myself). Greg also wrote the 3D renderer used in Builder. O m a r Odeh wrote the original network code and also the functions dealing with the wri t ten messages. Ben Wal ton wrote the code for handling the log files and the high score files. Ryan Fugger provided the sound effects for Bui lder and also wrote the functions dealing wi th spoken messages. Graphics for the start-up screens for Bui lder were provided by F i o n a Wong . Note that the communicat ion and task modes (i.e., whether players are in E C M or B C M , and S G M or M G M respectively) are set at compile-time by flags in the source code, meaning they cannot be changed during run-time. 99 Appendix B Client-Server Messages Messages are sent between the BuilderServer and the BuilderClient using 'the generic Island message package, which includes a specific sub-package called "Bui ldNetMessage" . The header file Bu i ldNe t .h declares a Bui ldNetMessage as follows: struct BuildNetMessage{ netEvtType what; short f i e l d l ; short f i e l d 2 ; short f i e l d 3 ; Point where; Rect i n f ; char special[1]; }; The message type, which is the first thing looked at by BuilderServer or BuilderClient upon receipt of a message, is given in the "what" field, which is of type "ne tEv tType" . The enum statement defining n e t E v t T y p e is: enum netEvtType{ kChooseRequ, kNewWRequ, kLockRequ, kChangeWallRequ, kChooseConf, kNewWConf, kLockConf, kChangeWallConf, kRendlnRequ, kAreaRequ, kChangeSideRequ, kEndRequ, 100 kRendlnConf, kAreaConf, kChangeSideConf, kEndConf, kVPRequ, kPerspRequ, kJoinRequ, kRendMoveRequ, kVPConf, kPerspConf, kJoinConf, kRendMoveConf, kDoneJoin, kBr ickConf , kFrameln i t , kFrameTake, kFrameReplace }; A s wi l l be noted, most message types end in either "Requ" or "Conf" . The prior indicates a request from the client, while the latter is a confirmation from the server (which is often sent to both clients). The meaning of each type is briefly stated below. In the cases where there is both a request and a confirmation of the same type, only the request is defined. • kChooseRequ- request to set the challenge (from "Challenge-Selection" screen). • kNewWRequ - request to make a new wal l . • kLockRequ - request to select a wal l . • kChangeWallRequ - request to move, flip, resize or delete a wal l . • kRendlnRequ - notification of entry to 3D-View (other client has to be informed). • kAreaRequ - request for current area/volume calculat ion. • • kChangeSideRequ - request to move, delete or set frame of a window or door. • kEndRequ - request to quit a challenge. • kVPRequ - request to set icon (from "Challenge-Selection" screen). • kPerspRequ - notification of switch between Top- and Side-View. • kJoinRequ - new client's request to join Builder. • kRendMoveRequ - notification of movement in 3D-View. • kDoneJoin - server tells new client that ini t ia l iz ing information is complete. • kBrickConf - server updates client on current brick count. • kFramelni t - server gives client list of frames allocated at the start of a challenge. • kFrameTake - server informs client of a frame removed from the store. • kFrameReplace - server informs client of a frame returned to the store. 101 Appendix C Academic Tests .1 Pre-test attached .. .2 Post-test attached .. 102 Appendix D Questionnaire For each item 1-20 players were given a set of 5 words to choose from: N O , No , M a y b e , Yes, Y E S . 1. I enjoyed playing Bui lder . 2. I like using computers. 3. M y partner was friendly. 4. Computer games like Bui lder should be used more in school. 5. It was difficult to communicate with my partner. 6. I thought the game was easy to play. 7. I like playing computer games wi th a partner. 8. I would like to play Bui lder again. 9. M y partner was helpful. 10. I would like to play Bui lder at home. 11. I learned something by playing Bui lder . 12. If I play Bui lder again I would like to play with the same partner. 13. I would enjoy playing other games wi th the same partner. 14. I liked communicat ing wi th my partner using the computer. 15. I needed more instructions to play Bui lder . 16. I would prefer to have my partner in the same room. 103 17. I wish I could have played Bui lder for longer. 18. It was easy communicat ing with my partner using the computer. 19. I would prefer to play Bui lder alone. 20. Communica t ing wi th my partner helped us to play the game. 21. How many computers do you have at your house ? 22. A b o u t how many hours a day do you think you spend on a computer (don't include school) ? 23. Please make a list of the main things you do on computer. If you play games, please write the names of the games you play. 104 Appendix E Observation Form .. attached .. 105 Appendix F Builder Screenshots .. attached .. 106 Bibliography [ARS97] John R . Anderson, Lynne M . Reder, and Herbert A . S imon. Appl ica t ions and misapplications of cognitive psychology to mathematics education, (in press), 1997. [Bat92] Trent Batson . F ind ing value in C S C L . SIGCUE Outlook, 21(3):26-28, 1992. [Bro93] Herb Brody. Video games that teach? Technology Review, November / December:51-57, 1993. [BTR+95] Lauren J . Bricker , Steve L . Tanimoto, A . I. Rothenberg, D . C . Hu tama , and T . H . Wong . Mul t iup layer activities that develop mathematical coordinat ion. In CSCL '95: First International Conference on Computer Support for Col-laborative Learning, pages 32-39, 1995. [Bul94] W . M . Bulkeley. Gender affects how user sees the computer. Wall Street Jour-nal, M a r c h 1 6 : B 1 - B 4 , 1994. [Car87] John M . C a r r o l l . The adventure of getting to know a computer. In W i l l i a m S. B u x t o n Ronald M . Baecker, editor, Readings in Human-Computer Interac-tion: A Multidiscliplinary Approach, pages 639-648. M o r g a n Kaufmann , San Mateo , C A , 1987. [Cle81] Frank J . Clement . Affective considerations in computer-based education. Ed-ucational Technology, Apr i l :28 -32 , 1981. [Coh94] El izabeth G . Cohen. Restructur ing the classroom: Condi t ions for productive small groups. Review of Educational Research, 6 4 ( l ) : l - 3 5 , 1994. 107 [CT0BS88] Mul t i l eve l Ed i t ion Canadian Tests of Basic Skills. Nelson Canada , Englewood Cliffs, N J , 1988. [Dal90] D a v i d W . Da l ton . The effects of cooperative learning strategies on achievement and attitudes during interactive video. Journal of Computer-Based Instruc-tion, 17(1):8-16, 1990. [DG95] D.Gr i f f in and R . Gonzalez. Correlat ional analysis of dyad-level data in the exchangable case. Psychological Bulletin, 118(3):430-439, 1995. [DHH89] D a v i d W . Da l ton , Michae l J . Hannafin, and Simon Hooper. Effects of indi-vidual and cooperative computer-assisted instruction on student performance and attitudes. Educational Technology Research and Development, 37(2): 15 -24, 1989. [Gol92] Shelley V . G o l d m a n . Computer resources for supporting student conversations about science concepts. SIGCUE Outlook, 21(3) :4-7, 1992. [Gol96] M u r r a y W . Goldberg . C A L O S : A n experiment wi th computer-aided learn-ing for operating systems. In ACM's 27th SIGCSE Technical Symposium on Computer Science Education, pages 175-179, Phi ladelphia , P A , 1996. [GRWB92] S. Greenberg, M . Roseman, D . Webster, and R . Bohnet . H u m a n and tech-nical factors of distr ibuted group drawing tools. Interacting with Computers, 4(3):364-392, 1992. [HDH87] Michea l H . Hannafin, D a v i d W . Da l ton , and Simon Hooper . Computers in education: Ten myths and ten needs. Educational Technology, October:8-14, 1987. [HH91] Simon Hooper and Michae l J . Hannafin. The effects of group composit ion on achievement, interaction, and learning efficiency during computer-based cooperative instruct ion. Educational Technology Research and Development, 39(3):27-40, 1991. 108 [ H L K M 9 2 ] R . Her tz-Lazarowitz , V . B . K i r k u s , and N . M i l l e r . Implications of current research on cooperative interaction for classroom applicat ion. In R . Hertz-Lazarowitz and N . Mi l l e r , editors, Interaction in cooperative groups: the the-oretical anatomy of group learning. Cambridge Universi ty Press, New York , N Y , 1992. [HSGB82] Jan Hawkins , Ka ren Sheingold, M a r y l Gearhart , and C h a n a Berger. M i c r o -computers in schools: Impact on the social life of elementary classrooms. Jour-nal of Applied Developmental Psychology, 3(4):361-373, 1982. [HZD93] Shelley Hymel , Beverly Zinc , and Elise Di tner . Cooperat ion versus competi t ion in the classroom. Exceptionality Education Canada, 3(1-2):103-128, 1993. [ IBGK95] K o r i M . Inkpen, Kel logg S. B o o t h , Steven D . Gr ibble , and M a r i a M . Klawe . Give and take: Chi ldren collaborating on one computer. In J . M . Bowers and S .D. Benford, editors, CHI '95: Human Factors in Computing Systems, pages 258-259, Denver, C O , 1995. A C M . Conference Companion . [IBK95] K o r i M . Inkpen, Kel logg S. B o o t h , and M a r i a M . Klawe . P lay ing together beats playing apart, especially for girls. In CSCL '95: First International Conference on Computer Support for Collaborative Learning, pages 177-181, 1995. [IK92] Hiroshi Ishii and M i n o r u Kobayashi . Clearboard: A seamless medium for shared drawing and conversation wi th eye contact. In CHI '92: Human Factors in Computing Systems, pages 525-532, 1992. [IM91] H . Ishii and N . M i y a k e . Toward an open shared workspace: computer and video fusion approach of team workstat ion. Communications of the ACM, 34(12):36-50, 1991. [IUK+94a] K o r i Inkpen, Rena Upi t i s , M a r i a Klawe , Joan Lawry, A n n Anderson, M u t i n d i Ndunda , K a m r a n Sedighian, Steve Leroux, and D a v i d Hsu . "We Have Never-Forgetful Flowers in O u r Garden:" G i r l s ' Responses to Electronic Games. Journal of Computers in Math and Science Teaching, 13(4):383-403, 1994. 109 [ I U K + 9 4 b ] K o r i M . Inkpen, Rena Upi t i s , M a r i a Klawe, Joan Lawry, A n n Anderson, M u t i n d i Ndunda , K a m r a n Sedighian, Steve Leroux, and D a v i d Hsu . "We have never-forgetful flowers in our garden:" G i r l s ' responses to electronic games. Journal of Computers in Math and Science Teaching, 13(4):383-403, 1994. [JJ74] Roger T . Johnson and D a v i d W . Johnson. Instructional goal structure: C o -operative, competitive, or individual is t ic . Review of Educational Research, 44(2):213-240, 1974. [JJ83] Roger T . Johnson and D a v i d W . Johnson. Social interdependence and per-ceived academic and personal support in the classroom. Journal of Social Psychology, 120:77-82, 1983. [JJ86] D a v i d W . Johnson and Roger T . Johnson. Computer-assisted cooperative learning. Educational Technology, 26:12-18, 1986. [JJS86] Roger T . Johnson, D a v i d W . Johnson, and M a r y Be th Stanne. Compar ison of computer-assisted cooperative, competit ive, and individual is t ic learning. American Educational Research Journal, 23(3):382-392, 1986. [JJSR85] Roger T . Johnson, D a v i d W . Johnson, L i n d a E . Scott, and Beverly A . Rarao-lae. Effects of single-sex and mixed-sex cooperative interaction on science achievement and attitudes and cross-handicap and cross-sex relationships. Journal of Research in Science Teaching, 22(3):207-220, 1985. [JMJ+81] D a v i d W . Johnson, Geoffrey M a r u y a n a , Roger Johnson, Deborah Nelson, and L i n d a Skon. Effects of cooperative, competit ive, and individual is t ic goal struc-tures on achievement: A meta-analysis. Psychology Bulletin, 89( l ) :47-62, 1981. [ K B W 8 3 ] James A . K u l i k , Robert L . Bangert , and George W . W i l l i a m s . Effects of computer-based teaching on secondary school students. Journal of Educational Psychology, 75( l ) :19-26, 1983. 110 [KP95] M a r i a Klawe and Eileen Phi l l ips . Electronic games engage children as re-searchers. In CSCL '95, B loomington , I N , October 1995. [KR93] K . K e l l y and H . Rheingold. The dragon ate my homework. Wired, Ju ly :69-73, 1993. [Les86] (Ma in author)Jack W . Lesage. Journeys in Math 7. G i n n and Company, Canada , 1986. [Lev92] James A . Lev in . Educat ion on the networks of today and tomorrow: The integration of computer-supported cooperative learning and work. SIGCUE Outlook, 21(3):8-9, 1992. [Mal82] Thomas W . Malone . Heuristics for designing enjoyable user interfaces: lessons from computer games. In Human Factors in Computer Systems, pages 63-68, Gai thersburg, M A , 1982. [Mal86] M a r y M a l e . Cooperative learning for effective mainstreaming. The Computing Teacher, 14( l ) :35-37, 1986. [McG96] Joanna McGrenere . Design: Educat ional electronic multi-player games a l i t-erature review. Technical Report 96-12, Department of Computer Science, Univers i ty of Br i t i sh Columbia , Vancouver, Br i t i sh Columbia , V 6 T 1Z4, 1996. [MF91] C . Mornings tar and R . F . Farmer. The lessons of Lucasfilm's habitat . In M . Benedikt , editor, Cyberspace: First Steps, pages 273-302. M I T Press, 1991. [Mic77] James W . Micheals . Classroom reward structures and academic performance. Review of Educational Research, 47( l ) :87-98 , 1977. [NC93] Bonnie K . Nastasi and Douglas H . Clements . Mot iva t iona l and social outcomes of cooperative computer education environments. Journal of Computing in Childhood Education, 4 ( l ) :15-43 , 1993. [OKdV96] R . Oosterholt , M . Kusano, and G . de Vries. Interaction design and human factors support in the development of a personal communicator for children. I l l In CHI '96: Human Factors in Computing Systems, pages 450-457, Vancouver, B C , 1996. [OL93] Sierra On-Line . The Even More Incredible Machine. Computer Software, 1993. [Pap93] Seymour Papert . The Children's Machine: Rethinking School in the Age of the Computer. BasicBooks, New York , N Y , 1993. [Per94] J . Pereira. Video games help boys get a head start. Wall Street Journal, M a r c h 1 6 : B 1 - B 4 , 1994. [Pro92] Eugene F . Provenzo. The video generation. The American Schoolboard Jour-nal, March :29-32 , 1992. [Rep93] Jud i Repman . Collaborat ive, computer-based learning: Cogni t ive and affective outcomes. Journal of Educational Computing Research, 9(2):149-163, 1993. [Res92] Mi t che l Resnick. Col laborat ion in simulated worlds: Learning through and about collaboration. SIGCUE Outlook, 21(3):36-38, 1992. [Rie92] Margare t R ie l . Cooperative learning through telecommunications. SIGCUE Outlook, 21(3):14-17, 1992. [ R M W W 9 2 ] J . M . Randel , B . A . Mor r i s , C D . Wetzel , and B . V . Whi t eh i l l . The effectiveness of games for educational purposes: A review of recent research. Simulation and Gaming, 23(3):261-276, 1992. [Rob94] Jenifer Walsh Robertson. 1 Usabi l i ty and children's software: A user-centered design methodology. Journal of Computing in Childhood Education, 5(3/4):257-271, 1994. [Ros92] Jeremy Roschelle. W h a t should collaborative technology be? A perspective from dewey and situated learning. SIGCUE Outlook, 21(3):39-42, 1992. [SBM+89] M . Scardamalia, C . Bereiter, R . S. M c L e a n , J . Swallow, and E . Woodruff. Compute r supported intentional learning environments. Journal of Educa-tional Computing Research, 5:51-68, 1989. 112 [Sed97] K a m r a n Sedighian. Interface style, flow, and reflective cognition: Issues in designing interactive mult imedia mathematics learning environments for chil-dren. Doc tora l dissertation, Dept of Computer Science, Universi ty of Br i t i sh Co lumbia , 1997. [Sha80] Shlomo Sharan. Cooperative learning in small groups: Recent methods and effects on achievement, attitudes, and ethnic relations. Review of Educational Research, 50(2):241-271, 1980. [SK96a] K a m r a n Sedighian and M a r i a Klawe . Super tangrams: A child-centered approach to designing a computer supported mathematics learning environ-ment. In International Conference on Learning Sciences '96, pages 490-495, Evanston, IL , 1996. [SK96b] K a m r a n Sedighian and M a r i a M . Klawe . A n interface strategy for promoting reflective cognition in children. In CHI '96: ACM Conference on Human Factors in Computing Systems, pages 177-178, Vancouver, B C , 1996. [Sla80] Robert E . Slavin. Cooperative learning. Review of Educational Research, 50(2):315-342, 1980. [Sla90] Robert E . Slavin. Cooperative Learning: Theory, Research, and Practice. P ren-tice H a l l , Englewood Cliffs, N J , . 1990. [SM94] K a r l E . Steiner and Thomas G . Moher . A comparison of verbal interaction in literal and vi r tua l shared learning environments. In CHI '94: Human Factors in Computing Systems, pages 97-98, Bos ton , M A , 1994. Conference Compan-ion. [Sol91] El l io t Soloway. How the nintendo generation learns. Communications of the ACM, 34(9):23-26, 1991. [Sol93] El l io t Soloway. Interactive learning environments: Where they've come from & where they're going. Tutor ia l Notes, Hyper text '93, Seattle, W A , 1993. 113 [ST79] Robert E . Slavin and Al l en M . Tanner. Effects of cooperative reward structures and individual accountabili ty on product ivi ty and learning. The Journal of Educational Research, 72(5):294-298, 1979. [SU97] S. Saxton and R . Upi t i s . Players, resistors and creators: Adolescents and computer games. Technical report, 1997. [SW87] S .B . Silvern and P . A . Wi l l i amson . The effects of video game play on young children, agression, fantasy and pro-social behaviour. Journal of Applied De-velopmental Psychology, 8:453-462, 1987. [TM94] P . Thomas and R . Macredie . Games and the design of human-computer in -terfaces. ETTI, 32(2): 134-142, 1994. [Uni97] Simon Fraser University. V i r t u a l - U research project. http://virtual-u.cs.sfu.ca/vuweb/, 1997. [Web82a] N . M . Webb. Peer interaction and learning in cooperative small groups. Jour-nal of Educational Psychology, 74(5):642-655, 1982. [Web82b] Noreen M . Webb. Group composit ion, group interaction, and achievement in cooperative small groups. Journal of Educational Psychology, 74(4):475-484, 1982. [Web82c] Noreen M . Webb. Student interaction and learning in small groups. Review of Educational Research, 52(3):421-445, 1982. [Web84] Noreen M . Webb. Microcomputer learning in small groups: Cogni t ive require-ments and group processes. Journal of Educational Psychology, 76(6):1076-1088, 1984. [WM94] J . H . Wiebe and N . J . M a r t i n . The impact of a computer-based adventure game on achievement and atti tude in geography. Journal of Computing in Childhood Education, 5(1):61-71, 1994. 1 1 4 Pre-test Name: Area: Perimeter: 2. If you have 20 square tiles with sides of length 10 cm, what area can you cover with them ? (It may help to draw a picture.) 3. What is the surface area and volume of this cube: .. Area covered: Surface area: Volume:_ 4. The picture shows the top view of a room with walls 1 unit thick. You know the outer sizes of the walls (as shown). Calculate the area of the floor inside the room. Floor area: 10 5. The picture shows a brick wall which is 8 units high and 10 units long. If the bricks are of size l x l unit, it will take 80 bricks to build the wall. If I make a hole in the wall for a window I will be able to save on bricks. Suggest possible (there may be more than one answer) dimensions for the window if I want to save 16 bricks. Length: Height: If I was framing the window with pieces of wood, what is the total length of wood that I need to make this window? Total length of wood: 6. Imagine wood is very expensive. Could you change the dimensions of your window from question 5 to make it take less wood but still save 16 bricks (your answer may be the same as 5) ? New length: New height: 7. Again imagine a wall made of lx l bricks. This time the wall is 14 bricks long and 10 bricks high. If I had one window of size 3x6 bricks and one window of size 2x4 bricks, how many bricks would I need to make the wall ? Number of bricks needed for wall: 8. A grasscutter is cutting a field which is 20 m x 20 m. It cuts a strip which is 1 m wide. After mowing once around the whole edge of the field, what is the area of grass cut already & the area that still needs to be cut. Area cut already: Area still to be cut:. 9. You have some spare pieces of wood laying around your house, so you decide to make a box to hold your cassette tapes. The wood is 1cm thick. You cut 2 pieces 50 cm long, and 2 pieces 20 cm long. Then you make the box as shown in the picture (top view). If your cassettes are 20 cm long and 2 cm thick, how many of them can you fit in the box? 10. A l l the shapes shown have the same perimeter. Without measuring them, which one do you think has the greatest area (circle your choice). Post-test Name: 1. Calculate the area & perimeter of the following shapes: Area: Perimeter: 2. You have tiles which are 20cm x 50cm. How many of them do you need to cover the rectangle shown ? 1rn A ^ — 3 m — - > Number of tiles needed: 3. What is the surface area and volume of this box: A 8 V Surface area:. Volume: • 6 - > 4. The picture shows the top view of 2 rooms with walls 1 unit thick. Some of the outer sizes are shown. If the walls are 1 unit thick, calculate the combined floor area of the 2 rooms. Floor area: 5. The picture shows a brick wall which is 10 units high and 10 units long. If the bricks are of size 1 x 1 unit, it will take 100 bricks to build the wall. If I make a hole in the wall for a window I will be able to save on bricks. Suggest possible (there may be more than one answer) dimensions for the window if I want to save 24 bricks. Length:. Height: If I was framing the window with pieces of wood, what is the total length of wood that I need to make this window? Total length of wood:. 6. Imagine wood is very expensive. If you have a total length of 20 units of wood, what size would you make the window to save the most bricks? New length: New height:. 7. Again imagine a wall made of l x l bricks. This time the wall is 12 bricks long and 10 bricks high. If I had two windows each of 4x4 units, how many bricks would I need to make the wall ? Number of bricks needed for wall: 8. Imagine you have a bathroom floor that you want to cover with tiles. The tiles are 1 unit square. The bathroom floor has dimensions 12 units x 10 units. How much area (in units squared) do you still need to cover if you go all around the edges of the room once? (It may help to draw a picture.) Area still to cover: -4 100cm 9. Look at the picture of the shelf. Imagine you have a lot of books that are all 5cm thick. You have to decide how many of the books you can fit on the shelf. The thickness of the boards is 3 cm, so you will have to think about the inside length of the shelf. (All the books will be upright. You don't have to worry about the height of the books.) Number of books that will fit: 10. A l l the shapes shown have the same perimeter. Without measuring them, which one do you think has the greatest area (circle your choice) ? BUILDER OBSERVATION F O R M Pair observed: Observer: Date: Time: Condition: Challenges played: Comments on communication: player 1: questions directives information off-task player 2: questions directives information off-task Comments on game play: General comments about Game (bugs, improvements, etc): sonia: Are you ready to start? dave: Yep! Let's do challenge 1. Send Message Clear Message r £ File N e t w o r k H e l p 12:40 A M < g ] Your goal is to build a house with area: 80 square units. Maximum Floor Area per Room: 1 0 0 Click anywhere to enter sonia: Are you ready to start? dave: Yep! Let's do challenge 1. Send Message Clear Message ffc File Network TOP SIDE B R I C K S U S E D : 2 5 5 / 5 8 6 mom GO mmiM* o sonia: Are you ready to start? dave: Yep! Let's do challenge 1. i -P f 1 * • . | 1 a I Resize wall length: Cancel Done • • • nipped uiaii 8:22 P M 0 4^  Area Send Message Clear Message « K File Network 8:23 PM H ^ File Network 8:25 PM i l ^ sonia: Are you ready to start? dave:Yep! Let's do challenge 1. Send Message Clear Message 

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.831.1-0051306/manifest

Comment

Related Items