Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Designing educational games and advanced learning technologies : an identification of emotions for modeling… Gutica, Mirela 2014

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2014_spring_gutica_mirela.pdf [ 3.66MB ]
Metadata
JSON: 24-1.0167291.json
JSON-LD: 24-1.0167291-ld.json
RDF/XML (Pretty): 24-1.0167291-rdf.xml
RDF/JSON: 24-1.0167291-rdf.json
Turtle: 24-1.0167291-turtle.txt
N-Triples: 24-1.0167291-rdf-ntriples.txt
Original Record: 24-1.0167291-source.json
Full Text
24-1.0167291-fulltext.txt
Citation
24-1.0167291.ris

Full Text

 Designing Educational Games and Advanced Learning Technologies: An Identification of Emotions for Modeling Pedagogical and Adaptive Emotional Agents   by  Mirela Gutica   Master of Electrical Engineering, University of Craiova, Romania, 1983 Master of Arts, University of British Columbia, 2008     A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF   DOCTOR OF PHILOSOPHY  in  The Faculty of Graduate and Postdoctoral Studies  (Curriculum Studies)  THE UNIVERSITY OF BRITISH COLUMBIA  (Vancouver)   April 2014      Mirela Gutica, 2014  ii  Abstract  Emotional, cognitive, and motivational processes are dynamic and influence each other during learning. The goal of this dissertation is to gain a better understanding of emotion interaction in order to design advanced learning technologies (ALTs) and intelligent tutoring systems (ITSs) that adapt to emotional needs. In order for ITSs to recognize and respond to affective states, the system needs to have knowledge of learners’ behaviors and states. Based on emotion frameworks in affective computing and education, this study responds to this need by providing an in-depth analysis of students’ affective states during learning with an educational mathematics game for grades five through seven (Heroes of Math Island) specifically designed for this research study and based on principles of instructional and game design.  The mixed methodology research design had two components: (1) a quasi-experimental study and (2) affect analysis. The quasi-experimental study included pretest, intervention (gameplay), and posttest, followed by a post-questionnaire and interview. Affect analysis involved the process of identifying what emotions should be observed, and video annotations by trained judges. The study contributes to related research by: (1) reviewing sets of emotions important for learning derived from literature and pilot studies; (2) analyzing inter-judge agreement both aggregated and over individual students to gain a better understanding of how individual differences in expression affect emotion recognition; (3) examining in detail what and how many emotions actually occur or are expressed in the standard 20-second interval; (4) designing a standard method including a protocol and an instrument for trained judges;  iii  and (5) offering an in-depth exploration of the students’ subjective reactions with respect to gameplay and the mathematics content. This study analyzes and proposes an original set of emotions derived from literature and observations during gameplay. The most relevant emotions identified were boredom, confidence, confusion/hesitancy, delight/pleasure, disappointment/displeasure, engaged concentration, and frustration. Further research on this set is recommended for design of ALTs or ITSs that motivate students and respond to their cognitive and emotional needs. The methodological protocol developed to label and analyze emotions should be evaluated and tested in future studies.    iv  Preface  This dissertation is original research by the author, Mirela Gutica. Data collection was approved by the University of British Columbia’s Research Ethics Board (certificate #H06-80670 and #H13-00245). A version of affect analysis found in Section 3.3.2.2 (methodology) and Section 4.3 (data analysis) has been published (Gutica & Conati, 2013). I was the lead investigator for this research and worked in collaboration with the research Supervisor and under Supervision of the committee on all major areas of concept formation, data collection and analysis, and manuscript composition.     v  Table of Contents  Abstract..................................................................................................................................... ii Preface ..................................................................................................................................... iv Table of Contents ......................................................................................................................v List of Tables ......................................................................................................................... viii List of Figures ......................................................................................................................... ix Acknowledgments ................................................................................................................... xi Dedication ............................................................................................................................. xiii Chapter One: Introduction ...................................................................................................... 1 1.1. Statement of the Problem ....................................................................................... 4 1.2. Research Questions ................................................................................................. 7 1.3. Purpose of the Study ............................................................................................... 8 1.4. Theoretical Framework ........................................................................................ 10 1.4.1. Emotion Theory .................................................................................................. 11 1.4.2. Emotion and Learning ........................................................................................ 12 1.4.3. Affective Computing .......................................................................................... 15 1.4.4. Methodology ....................................................................................................... 16 1.5. Terminology .......................................................................................................... 17 1.6. Limitations of the Study ....................................................................................... 19 1.7. Dissertation Overview and Its Structure ............................................................ 20 Chapter Two: Literature Review ........................................................................................... 22 2.1. Introduction........................................................................................................... 22 2.2. What is Emotion? ................................................................................................. 24 2.3. Theories of Emotion ............................................................................................. 25 2.3.1. Feeling-Centered, Traditional and Contemporary Cognitivism ......................... 25 2.3.2. Current Findings in Neuroscience and Psychology ............................................ 30 2.4. Emotion and Cognition ........................................................................................ 37 2.5. Emotion Classifications ........................................................................................ 43 2.5.1. Dimensional Models of Emotion: Core Affect ................................................... 50 2.5.2. Emotion as a Consequence of Reward or Punishment ....................................... 52 2.5.3. Emotion Synthesis: OCC Cognitive Theory....................................................... 53 2.6. Affective Computing ............................................................................................. 55 2.7. Cognition, Emotion and Learning....................................................................... 57 2.8. Emotion Models with Respect to Learning ........................................................ 60  vi  2.8.1. Positive Emotions ............................................................................................... 61 2.8.2. Emotion in Instructional Design – FEASP (Fear, Envy, Anger, Sympathy, Pleasure) ......................................................................................................................... 64 2.8.3. Emotion, Identity and Learning .......................................................................... 68 2.8.4. Analytical Model of Emotions and Learning ..................................................... 70 2.9. Intelligent Tutoring Systems ................................................................................ 73 2.9.1. Intelligent Tutoring Systems and Emotions ....................................................... 74 2.10. Conclusion ............................................................................................................. 78 Chapter Three: Methodology ................................................................................................ 82 3.1. Introduction........................................................................................................... 82 3.2. Section One: Heroes of Math Island ................................................................... 82 3.2.1 Game Description .................................................................................................. 82 3.2.1.1. Learning Outcomes ..................................................................................... 85 3.2.1.2. Emotional Character ................................................................................... 87 3.2.2. Game Mechanics ................................................................................................ 88 3.3. Section Two: Study Design .................................................................................. 93 3.3.1. Participants and Recruitment .............................................................................. 94 3.3.2. Procedure ............................................................................................................ 94 3.3.2.1. First Part: Quasi-Experimental Study ......................................................... 95 3.3.2.2. Second Part: Affect Analysis .................................................................... 100 3.3.2.2.1. Stage One: Emotion Labeling Process ................................................. 100 3.3.2.2.2. Stage Two: Video Annotations by Trained Judges .............................. 108 3.4. Conclusion ........................................................................................................... 110 Chapter Four: Data Analysis .............................................................................................. 111 4.1. Introduction......................................................................................................... 111 4.2. Demographics ...................................................................................................... 111 4.3. Affective States Analysis .................................................................................... 112 4.3.1. Agreement over One Emotion per Interval ...................................................... 114 4.3.2. Agreement over Multiple Emotions per Interval .............................................. 118 4.4. Descriptive and Causal-Comparative Analysis ................................................ 122 4.4.1. Attitude Before Play ......................................................................................... 123 4.4.2. Affective Domain ............................................................................................. 125 4.4.2.1. Affective Domain: Attitude Towards the Game ....................................... 127 4.4.2.1.1. Game Mechanics .................................................................................. 128 4.4.2.1.2. Gender Issues ....................................................................................... 130 4.4.2.1.3. Would You Play the Game Again? ...................................................... 131 4.4.2.2. Affective Domain: Attitude Towards Mathematics ................................. 134 4.4.2.3. Emotional Character: The Monkey. Is She Annoying? ............................ 136 4.4.3. Cognitive Domain ............................................................................................. 146 4.4.3.1. Cognitive Domain: Attitude Towards the Game ...................................... 147 4.4.3.2. Cognitive Domain: Attitude Towards Mathematics ................................. 150  vii  4.4.4. Learning ............................................................................................................ 153 4.4.4.1. Achievements: Pretest and Posttest .......................................................... 153 4.4.4.2. Students’ Perceptions of Learning ............................................................ 156 4.4.4.2.1. Before Game Play ................................................................................ 157 4.4.4.2.2. After Game Play ................................................................................... 157 4.5. Triangulation: Judges’ Perception of Process and Student Interaction ........ 165 4.6. Conclusion ........................................................................................................... 169 Chapter Five: Conclusion ................................................................................................... 172 5.1. Summary of the Study ........................................................................................ 174 5.2. Summary of Contributions ................................................................................ 176 5.3. Conclusions .......................................................................................................... 177 5.3.1. Identifying and Detecting Emotions ................................................................. 177 5.3.2. Attitudes Towards the Game ............................................................................ 181 5.3.3. Learning ............................................................................................................ 182 5.4. Recommendations for Future Research ........................................................... 183 5.4.1. Recommendations for Design Issues and Heuristics Emerging from Results . 184 5.4.2. Recommendations for Further Investigation of the Set of Emotions Proposed 185 Bibliography ......................................................................................................................... 188 Appendix 1: Recruitment ..................................................................................................... 195 Appendix 2: Study Protocol ................................................................................................. 197 Appendix 3: Tutorial ............................................................................................................ 198 Appendix 4: Pretest .............................................................................................................. 199 Appendix 5: Posttest............................................................................................................. 204 Appendix 6: Post Questionnaire.......................................................................................... 209 Appendix 7: Interview Guide Students ............................................................................... 213 Appendix 8: Observer Guide ............................................................................................... 214 Appendix 9: Protocol for Observing Affective States ......................................................... 215 Appendix 10: Instrument ..................................................................................................... 218 Appendix 11: Interview Guide Judges ................................................................................ 219 Appendix 12: Sample of Raw Data Analysis: Aggregated Analysis .................................. 221 Appendix 13: Sample of Raw Data Analysis: Students 2, 6, 9, 13 and 21......................... 223 Appendix 14: Sample of Raw Data Analysis: Number of Emotions per Interval ............. 233 Appendix 15: T-Test and Descriptive Statistics .................................................................. 235    viii  List of Tables  TABLE 1. SPEARMAN’S COEFFICIENTS....................................................................................... 98 TABLE 2. QUESTIONNAIRE BLUEPRINT ...................................................................................... 98 TABLE 3. CONFUSION MATRIX: AGGREGATED DATA ............................................................... 116 TABLE 4. CONFUSION MATRIX: NUMBER OF EMOTIONS PER INTERVAL .................................... 119 TABLE 5. ATTITUDE BEFORE PLAY .......................................................................................... 123 TABLE 6. AFFECTIVE DOMAIN: STUDENTS’ RESPONSES ........................................................... 126 TABLE 7. AFFECTIVE DOMAIN: QUESTIONNAIRE BLUEPRINT ................................................... 127 TABLE 8. AFFECTIVE DOMAIN: ATTITUDE TOWARDS THE GAME .............................................. 127 TABLE 9. AFFECTIVE DOMAIN: ATTITUDE TOWARDS MATHEMATICS ........................................ 134 TABLE 10. AFFECTIVE DOMAIN: ATTITUDE TOWARDS THE EMOTIONAL CHARACTER ................ 136 TABLE 11. COGNITIVE DOMAIN: QUESTIONNAIRE BLUEPRINT ................................................. 147 TABLE 12. COGNITIVE DOMAIN: ATTITUDE TOWARDS THE GAME ............................................ 147 TABLE 13. COGNITIVE DOMAIN: ATTITUDE TOWARDS MATHEMATICS ...................................... 150 TABLE 14. PRE- AND POSTTEST RESULTS ................................................................................ 155 TABLE 15. LEARNING: QUESTIONNAIRE BLUEPRINT ................................................................ 156 TABLE 16. LEARNING: ATTITUDE TOWARDS THE GAME AND MATHEMATICS ............................. 156      ix  List of Figures  FIGURE 1. SELECTION OF LISTS OF BASIC EMOTIONS. ADAPTED FROM ORTONY AND TURNER (1990, P. 316). ................................................................................................................. 46 FIGURE 2. FRAMEWORK OF EMOTION. ADAPTED FROM  RUSSELL AND FELDMAN BARRETT (1999, P. 808). ................................................................................................................. 51 FIGURE 3. FRAMEWORK OF EMOTION BASED ON REINFORCEMENT. ADAPTED FROM ROLLS (2000, P. 179). ................................................................................................................. 53 FIGURE 4. GLOBAL STRUCTURE OF EMOTION TYPES. ADAPTED FROM ORTONY, CLORE AND COLLINS (1988, P. 19). ..................................................................................................... 55 FIGURE 5. GENERAL INSTRUCTIONAL STRATEGIES. ADAPTED FROM ASTLEITNER (2000, P. 191). ........................................................................................................................................ 67 FIGURE 6. MODEL OF EMOTION AND LEARNING. ADAPTED FROM INGLETON (2000, P. 88). ..... 69 FIGURE 7. ANALYTICAL MODEL OF EMOTIONS AND LEARNING.  ADAPTED FROM KORT AND REILLY (2002, P. 10). ....................................................................................................... 71 FIGURE 8. HEROES OF MATH ISLAND GAME AND SCENE. ........................................................... 84 FIGURE 9. THE MONKEY’S EMOTIONS (FROM THE LEFT): FRUSTRATED, SAD, NEUTRAL, HAPPY AND CONFIDENT. .............................................................................................................. 88 FIGURE 10. DIVISIBILITY ACTIVITY: A HINT IS DISPLAYED. ..................................................... 89 FIGURE 11. THE DIVISIBILITY ACTIVITY. .................................................................................. 90 FIGURE 12. THE PRIME NUMBERS ACTIVITY. ............................................................................ 91 FIGURE 13. THE DECOMPOSITION ACTIVITY. ............................................................................ 92 FIGURE 14. STUDY DESIGN. ..................................................................................................... 95 FIGURE 15. THE EXPERIMENT SETTINGS................................................................................... 96 FIGURE 16. JUDGES REPORT ON EMOTIONS: RAW AGREEMENT. .............................................. 113 FIGURE 17. COHEN’S KAPPA FOR INDIVIDUAL STUDENTS. ..................................................... 115 FIGURE 18. EMOTIONS REPORTED PER INTERVAL. .................................................................. 118 FIGURE 19. NUMBER OF EMOTIONS THAT MATCHED PER INTERVAL. ...................................... 120 FIGURE 20. AGREEMENT FOR 2 OR 3 EMOTIONS PER INTERVAL FOR INDIVIDUAL STUDENTS. . 121 FIGURE 21. PERCENTAGE OF TOTAL INSTANCES WHEN JUDGES AGREED ON EMOTION TYPE AND NUMBER FOR 1, 2, 3, OR 4 EMOTIONS. ............................................................................ 121 FIGURE 22. ABSOLUTE AGREEMENT FOR 1, 2, OR 3 EMOTIONS FOR INDIVIDUAL STUDENTS. .. 122 FIGURE 23. ATTITUDE BEFORE PLAY: I LIKE VIDEO / MATH GAMES. ....................................... 124 FIGURE 24. ATTITUDE BEFORE PLAY: I LIKE MATH. ............................................................... 125 FIGURE 25. I WANTED TO HAVE FUN. ..................................................................................... 128 FIGURE 26. I HAD FUN. ........................................................................................................... 129 FIGURE 27. SUMMATIVE GAME MECHANICS. .......................................................................... 130 FIGURE 28. THIS GAME IS BORING / I WOULD LIKE TO PLAY AGAIN. ....................................... 132 FIGURE 29. THIS GAME IS INTERESTING. ................................................................................ 133 FIGURE 30. I FELT UPSET WHEN I DID NOT ANSWER CORRECT. ............................................... 134 FIGURE 31. I HAD FUN WITH THE PRE- AND POSTTEST. ........................................................... 136 FIGURE 32. THE MONKEY WAS ANNOYING. ............................................................................ 137 FIGURE 33. THE MONKEY REACTED CORRECTLY TO MY EMOTIONS. ...................................... 140 FIGURE 34. I WAS PROUD TO SEE THE MONKEY HAPPY. .......................................................... 142 FIGURE 35. I FELT SHAME WHEN I MADE MISTAKES. .............................................................. 143  x  FIGURE 36. I HAD FUN WHEN THE MONKEY WAS HAPPY / SAD. ............................................... 144 FIGURE 37. LEVEL OF DIFFICULTY. ........................................................................................ 148 FIGURE 38. I WOULD LIKE TO SEE MORE QUESTS. ................................................................... 149 FIGURE 39. I BECAME CURIOUS ABOUT MATH. ....................................................................... 151 FIGURE 40. I DO NOT WANT TO THINK ABOUT MATH. ............................................................. 153 FIGURE 41. REPORTED PREVIOUS LEVEL OF KNOWLEDGE. ..................................................... 154 FIGURE 42. I WANTED TO LEARN MATH. ................................................................................ 157 FIGURE 43. I LEARNED MATH WHEN I PLAYED THE GAME. ..................................................... 158 FIGURE 44. I LEARNED MATH WHEN I HELPED MINERS / FINISHED THE QUEST. ....................... 159 FIGURE 45. I LEARNED MATH FROM MISTAKES / HINTS / HARDER QUESTIONS. ....................... 160 FIGURE 46. LEARNING: DETAILED HISTOGRAM. .................................................................... 161 FIGURE 47. LEARNING: SUMMATIVE HISTOGRAM. ................................................................. 162    xi  Acknowledgments  I wish to thank to the faculty, staff and my fellow students at the Faculty of Education at the University of British Columbia (UBC), and my colleagues and students from the British Columbia Institute of Technology (BCIT) who inspired and encouraged me to research this topic.  My thesis committee included Dr. Stephen Petrina (supervisor), Dr. Cristina Conati and Dr. Don Krug.  Dr. Conati was very helpful in concept formation and manuscript edits especially for affect analysis. Dr. Krug was also very helpful in concept formation and manuscript edits especially with respect to methodology.   I would like to express the deepest appreciation to my supervisor Dr. Petrina for his remarkable mentoring, encouragement and support with all aspects of my work. Without his guidance this thesis wouldn’t be possible. I owe particular thanks to my committee members Dr. Conati and Dr. Krug for their tremendous help and guidance in important aspects of my thesis. I am extremely grateful for the support received from BCIT, the institution where I am a faculty member.  The game Heroes of Math Island was designed by me in collaboration with Dr. Petrina and Dr. Conati and the graduate team of researchers assembled in the How We Learn research lab including: Franc Feng (Faculty), James Gauthier (PhD), PJ Rusnak (PhD), Yifei Wang (PhD), Liang You (MEd), Fareed Teja (MEd), and Jennifer Zhao (PhD). Components of the game development have been funded through my supervisor’s SSHRC SRGs and BCIT funds. I was awarded a two year (2012-2013) research grant from BCIT through the  xii  VP Research Seed Fund; this fund allowed me to fulfil the requirements of this study and involve BCIT students in research. The following BCIT students from the School of Computing and Academic Studies, Computer Systems Technology program were involved in the design and implementation of the game and/or acted as observers and judges: Ranyl Bantog and Brian Hume from the Bachelor of Technology program, Gaming Option; Alex McGilvray, Darry Danzig, Aubrey Fowler and Quan Wu from the Diploma Program; and the Technical Programming Option graduation class of 2011. The game wouldn’t have been possible without the talent and dedication of the above students. Tyler Schroeder, was the graphic animator, for this version of this game. Alayne Armstrong (UBC), Jennifer Zhao (UBC) and Youdan Zhang (BCIT) provided important feedback for the game design. Alayne Armstrong offered her expertise in mathematical content. Special thanks are owed to my husband Marian, sons Alex and Filip, my mother, and all my family and friends who supported and encouraged me to continue this work.  To each of the above, I extend my deepest appreciation.    xiii  Dedication               To my family       1  Chapter One: Introduction  New technologies and applications, such as educational software in the form of games and tutors, are reconfiguring relationships between teachers and learners. From elementary schools to post-secondary institutions, the educational process includes technologies for enhancing teaching and learning by improving the presentation of course material, facilitating practice in terms of scope (e.g., interdisciplinary), simulating real-life situations, flipping classrooms, and generally engaging student interaction. Computer-assisted instruction, test-reconstruction software, concordance software, situated learning, and multimedia simulation software were explored and introduced since the 1970s; changes have been complex and contradictory. Significant issues have been raised: the degree of control of the learner, effectiveness of educational software, situated learning versus the traditional information-processing model, e-learning, and virtual reality versus reality.  This dissertation research involved a detailed analysis of students’ interaction with an educational game for students in grades five through seven (Heroes of Math Island) especially designed for this research study and based on principles of instructional and game design. This educational game described in details in section 3.2, qualifies as an advanced learning technology (ALT) device. According to Aleven, Beal and Graesser’s (2013) ALTs are:  [1] created by designers who have a substantial theoretical and empirical understanding of learners, learning, and the targeted subject matter… [2] provide a high degree of interactivity, reflecting a view of learning as a complex, constructive activity on the part of learners that can be enhanced with detailed, adaptive  2  guidance… [3] capable of assessing learners while they use the system along different psychological dimensions, such as mastery of the targeted domain knowledge, application of learning strategies, and experiences of affective states. (pp. 929-930)  ALTs encompass a range, including intelligent tutoring systems, interactive scaffolds, game interfaces, and applications responsive to emotion.  Affective computing is an interdisciplinary field spanning computer sciences (human-computer interaction, artificial intelligence, algorithms, and computational complexity), psychology, and cognitive science concerned with the theory, design and construction of machines or software applications that can detect, respond to, and simulate human affective states. The term affective computing was coined by Rosalind Picard, founder and director of the Affective Computing Research Group at the Massachusetts Institute of Technology (MIT) Media Laboratory in her Affective Computing book published in 1997. Picard argued that the only way to create intelligent machines is to make them sense and respond to emotion. The related concept of emotional design was introduced by (Norman, 2004) and refers to the ability to design objects that are pleasing and therefore, become more effective because users connect with them emotionally. In this dissertation the terms emotion and affect are used interchangeably, albeit with specific intentions when necessary.  Theorists agree that interaction between humans and computers is more effective if machines are able to detect and respond to emotions (Chalfoun & Frasson, 2012; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Marsella, Gratch, & Petta, 2010; Scherer, 2010; Um, Song, & Plass, 2007; Baker, D’Mello, Rodrigo, & Graesser, 2010; Rodrigo, et al., 2012).  Currently, the field of affective computing is evolving rapidly with laboratories and  3  research teams interested in computational modeling of emotion and design and implementation of intelligent agents capable of expressing and responding to emotion. There is an increased realization that emotions simulated by ALTs and specific technology, including robots, can be successful only if they are credible and close to natural emotions (Scherer, 2010). Marsella, Gratch and Petta (2010) stated:  Modern research in the psychology, cognitive science and neuroscience of emotion has led to a revolution in our thinking about the relation of emotion to cognition and social behavior, and as a consequence is also transforming the science of computation. (p. 5) During the past two decades, attention has been given to technologies that could provide customized and individualized or personalized instruction.  Intelligent Tutoring Systems (ITSs) are integral to ALTs and allow learners opportunities to interact within meaning-rich contexts through which they construct and acquire competences, provide customized instruction or feedback to students, and express “(a) knowledge of the domain (expert model), (b) knowledge of the learner (student model), and (c) knowledge of teaching strategies (tutor)” (Shute & Psotka, 1996, p. 578). Aleven, Beal and Graesser (2013) define ITSs as “systems that provide detailed guidance (e.g. through hints, feedback, after-action review, or individualized problem selection) as learners work through complex problem scenarios and hone their understanding and problem-solving skills” (p. 930). According to VanLehn (2006), ITSs are characterized as having two loops: an outer loop that executes once for each task (a task usually consists of solving a multi-step problem) and an inner loop that executes once for each step taken by the student in solving a task. The inner loop is aimed to adapt to the student’s needs by giving feedback on each step  4  and assessing the student’s evolving competence, which is used by the outer loop to select the most appropriate next task. Software educational games intend to expose players to experiences that improve motivation and the level of engagement in learning processes by teaching and evaluating the concepts in game-like activities. Many ITSs are designed into ALT-driven educational games. Increasingly for ALTs, ITSs and educational games, researchers are addressing the role of emotion generally in interaction with the machine and more precisely in the context of learning (Baker, D’Mello, Rodrigo, & Graesser, 2010; Conati, 2002; Conati & Maclaren, 2009; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Jraidi, Chalfoun, & Frasson, 2012; McQuiggan, Robison, & Lester, 2010). For educational software and ALT’s, the “question is not whether intelligent machines can have any emotions, but whether machines can be intelligent without emotions” (Minsky, 1986, p. 163). 1.1. Statement of the Problem Understanding emotional reactions and responses with respect to ALTs is extremely timely and relevant for teaching and learning in digital or virtual (e.g., gaming) environments taking in consideration that technology is now ubiquitous in schools and in students’ lives. In this dissertation research, by providing a detailed analysis, I focused on identifying what emotions are relevant to learning in the context of interaction with the educational mathematics game Heroes of Math Island. As Petrina (2007) argued, teachers should recognize the importance of technology in the classroom as well as the feelings and values of students: “Our task [as instructional designers and teachers] is to validate, direct, and transform the emotion in our students’ experiences” (p. 59).  5  Contemporary discourses in emotion theory place emotion in a pivotal position in education (Arnone, Small, Chauncey, & McKenna, 2011; Astleitner, 2000; Boler, 1997, 1999; Hascher, 2004, 2010; Ingleton, 2000; Petrina, 2007; Picard, 2003; Picard, Kort & Reilly, 2003; Scherer, 2001a; Weiss, 2000). However, even if the importance of emotion was recognized in “encouraging and inhibiting effective learning and approaches to study… educational research and models of learning have shed little light on the interrelationships between emotions and learning” (Ingleton, 2000, p. 86), and despite the “high relevance of emotions in basic research and in daily school life, for decades, the focus in instructional design and related research was on considering learner’s above all cognitive and to some degree motivational processes” (Astleitner, 2000, p. 170). According to Hascher (2010) more should be done since “there is a huge need for further research because we know so little about learning and emotion” (p. 24). Understanding emotional interaction is also crucial for designing ALTs and ITSs that recognize and respond to emotions in a proper and natural way. Past research related to ITSs addressed cognitive tutors and, privileged cognitive over affective needs by observing learning as information processing and “marginalizing affect” (Woolf et al., 2009, p. 129). As affect has begun to play an increasingly important role in ITSs, research has focused not only on the cognitive aspects of interaction, but also on affect recognition and response. There is increasing evidence that, in order to design an intelligent and responsive tutor, the learner’s emotions should be properly identified (Baker, D'Mello, Rodrigo, & Graesser, 2010; Chaouachi & Frasson, 2012; Conati, 2002; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Jraidi, Chalfoun, & Frasson, 2012) . There has been extensive work on identifying and detecting emotions elicited by educational software (Baker, D’Mello,  6  Rodrigo, & Graesser, 2010; Conati & Maclaren, 2009; Chaouachi & Frasson, 2012; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Jraidi, Chaouachi, & Frasson, 2013; McQuiggan, Robison, & Lester, 2010; Rodrigo, et al., 2012).  This study adds to related work and contributes to affective computing, ALTs  and ITSs by: (1) reviewing sets of emotions important for learning derived from literature and pilot studies and proposing an original set of emotions observed during gameplay; (2) analyzing inter-rater or inter-judge agreement aggregated and at the individual student level, to gain a better understanding of how individual differences in expression affect emotion recognition; (3) examining in detail what and how many emotions actually occur or are expressed in the standard 20-second interval used in literature (Baker, D’Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Rodrigo, et al., 2008; Rodrigo, et al., 2012) and the challenge or ease in which judges identify them; and (4) designing a standard method including a protocol and an instrument intended for trained judges to label and analyze emotions over 20-second intervals.  Besides Baker, D’Mello, Rodrigo and Graesser`s (2010) guide for emotion observation, in related literature no formalized method, instruments or recommendations were available for studies that use judges to identify and label emotions. Studies that addressed emotional aspects of interaction with an ITS, generally took into consideration a set of emotions based on pre-existing frameworks: e.g., Ekman’s Facial Action Coding System (1999), the OCC theory of emotion proposed by Ortony, Clore, & Collins (1987), and the framework proposed by D’Mello and Graesser (2006). These sets of emotions do not include some emotions that in the educational literature are considered important for learning, for example, confidence, curiosity, pride and shame. Ingleton (2000) argues that  7  confidence creates a “disposition to learn,” as opposed to negative emotions such as anxiety, grief and dejection, which can prevent learning and lead to inactivity and isolation (p. 88). Pride and shame play a role in identity and self-esteem, which influence the formation of confidence. Curiosity is another key emotion for learning because it “can be a powerful motivator of behavior, initiating actions directed at exploring one’s environment to resolve uncertainty and make the novel known” (Arnone, Small, Chauncey, & McKenna, 2011, p. 181).  In order to design effective ITSs and, more generally ALTs, it is important to understand the students’ subjective perceptions of learning gains and enjoyment. More research is needed to identify what students expect from educational software and ALTs, to what extent they would choose these over another instrument for learning, or if they would return and use or engage with it again.  This study contributes to the field of Instructional Design (ID) by (1) exploring to what extent learning happened in the context of the Heroes of Math Island game, (2) exploring the students’ subjective reactions with respect to gameplay and mathematics content, and (3) investigating the students’ levels of interest and achievement in mathematics. This study also contributes to methodology by designing a protocol used to label and analyze emotions. 1.2. Research Questions The main research questions of this study are: 1. What affective states are important with respect to student’s interaction with an educational game?    8  a. What affective states are elicited during the Heroes of Math Island gameplay? Additionally, this study responded to the following two questions: 2. What are the students’ subjective reactions with respect to Heroes of Math Island and to the underlying mathematics content? 3. What are students’ levels of interest and achievement in the mathematics content after gameplay?  The themes explored in these questions are: empirical identification of emotion (main theme; question 1), cognitive gains and interest in mathematics by playing the game (question 2), and subjective reactions of students with respect to the game and learning mathematics (question 3). These themes will be used in study design and data analysis and discussed in the following chapters.  1.3. Purpose of the Study A high-level goal of this dissertation is to gain a better understanding of emotion interaction in order to design ALTs and ITSs that improve the students’ motivation by adapting to their emotional needs. In order for ITSs to recognize and respond to the students’ affective states, the system needs to have knowledge of learner’s behaviors and states. However, there is a gap between the students’ behaviors observable or registered by an ITS and the states and behaviors that need to be modeled (Bondareva, et al., 2013). As Jraidi, Chaouachi and Frasson, (2012) stated, strategies emplyed in tutors to enhance the learners’ mental states “can be in some cases excessive, inappropriate, or intrusive to the dynamics of the learning session. They can also be approximate or target basically superficial aspects of  9  the interaction” (p. 2). The purpose of this study is to contribute to filling this gap by providing an in-depth understanding and analysis of the students’ affective states during learning with an educational game (Heroes of Math Island). A detailed description of this game is provided in Chapter Three.   Instead of using a pre-existent set of emotions, this study analyses and proposes an original set of emotions derived from emotion theory, educational literature and affective computing, and from pilot studies and observations during gameplay.  Related research adopted a variety of approaches for emotion identification, including video annotation (D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006), quantitative field observation (Baker, D'Mello, Rodrigo, & Graesser, 2010; Rodrigo, et al., 2008; Rodrigo, et al., 2012), self-reports (Conati & Maclaren, 2009; McQuiggan, Robison, & Lester, 2010; Chaouachi & Frasson, 2012), and sensors (Conati, 2002; Chalfoun & Frasson, 2012; Jraidi, Chalfoun, & Frasson, 2012; Jraidi, Chaouachi, & Frasson, 2013). In terms of how long the interval of observation should be, many studies used a 20-second interval (Baker, D'Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Rodrigo, et al., 2008; Rodrigo, et al., 2012). In this study I designed a standard protocol that relies on trained judges to report emotions over 20-second intervals. Even if an emotion coding manual proposed by Baker, D’Mello, Rodrigo and Graesser (2010) was used by judges in related research (Baker, D'Mello, Rodrigo, & Graesser, 2010; Rodrigo, et al., 2008; Rodrigo, et al., 2012), there is no indication that this process was formalized. Therefore, this dissertation research analyzed the observation process for a deeper understanding of the methodology and to gain a deeper understanding of judges’ confidence. Finally, this study proposes a set of emotions, a standard method, (including a protocol and  10  an instrument to be used for the emotion labeling process) and recommendations for future studies.  Studies by Rodrigo et al. (2008, 2012), Conati and Maclaren (2009) and Woolf et al. (2009) used animated agents. Rodrigo et al. used animated agents with emotional expressions to provide adaptive scaffolding (2008) and to reduce the incentive to “game the system” or guess the correct answer (2012). Similar to Rodrigo et al., the Heroes of Math Island game has an emotional agent (the monkey character) that uses emotional expressions to react to students’ performance. This study analyses the students’ reactions towards the emotional agent in order to determine to what extent the agent improves the student’s experience and motivation for playing and doing well in the game. One recognizable goal of an educational game ought to be advanced learning. This study analyses the students’ interaction with the game with respect to emotions and in correlation with achievements (test scores) and the subjective perceptions of students towards learning gains. Besides emotions, this study analyses the students’ reactions towards gameplay and the mathematics content from three perspectives: the attitude towards (1) game, (2) mathematics content and (3) emotional agent of the game. Ultimately, the results and recommendation resulting from this study provide critical information that can be used in design methodologies of ALTs, ITSs and educational games. Its timeliness can be seen in the September 9, 2013 issue of the Journal of Educational Psychology, focusing on ALTs and ITSs.  1.4. Theoretical Framework Three theoretical frameworks are most relevant to this dissertation: emotion theory, emotion and learning, and affective computing.  11  1.4.1. Emotion Theory With respect to emotion theory, three key paradigms evolved over time: a feeling-centered paradigm where emotions are considered bodily sensations, a biological paradigm in which emotions are explained as biological reactions needed for survival, and a cognitive paradigm that involves thought and cognitive processes in emotion definition. In responding the research questions I took a cognitivist position, also considering that emotion and cognition are interrelated and cannot be separated. Cognitive-appraisal theories of emotion, first formulated by Arnold (1960) and Lazarus (1982, 1984, 1991) and extended by Scherer (2001b, 2009), state that emotions are triggered by events that elicit thinking and, in the end, arousal of emotion. According to Scherer (2009, p. 3460) these theories “assume an emotion architecture that is based on an individual's subjective evaluation or appraisal of the significance of events for their well-being and goal achievement, postulating a specific set of appraisal criteria”. To what extent does a specific emotion impact learning? Memorization, judgment and choice are key aspects of learning processes and are also heavily influenced by emotion. For example, we know that emotion is beneficial because it drives attention, which in turn drives memorization, but we do not fully understand our emotional system, and we don’t know exactly how to regulate it in learning activities. Positive emotions like confidence facilitate learning opposed to negative emotions, which can prevent learning (Ingleton, 2000). Many contemporary scientists and theorists define emotion in the context of cognition and motivation (LeDoux J. E., 1995; 1996; Ortony & Turner, 1990; Plutchik, 1984; Rolls, 1995). LeDoux (1995) is Socratic or Platonic when noting that cognition “historically has been thought of as part of a trilogy of mind that also includes emotion and will (motivation) rather than as an all-encompassing description of mind” (p. 226), and Ortony and Turner  12  (1990) argued that emotion has cognitive representation and is “causally related” to motivation (p. 318). Theories of emotion employ very different mechanisms for classifying emotion: several taxonomies of emotions have been established: e.g., Ekman’s Facial Action Coding System (1999), Ortony, Clore, & Collins’ OCC theory of emotion (1987), Russell’s (2003) core affect model, etc. However, what emotions are to be included in a framework, or what classifies as an emotion, is debatable; for example, surprise, included in many taxonomies, is not considered an emotion by some authors (Lazarus, 1982).  1.4.2. Emotion and Learning Learning is an active process that involves acquiring new knowledge and skills. As a result, there are changes in attitude, behavior, performance, mental skills, and relationship with the environment and other members of the community. As a cognitive process, learning involves emotion, memorization, motivation, participation and socialization. In a cognitivist paradigm, the center of learning is the brain: the brain and nervous system are responsible for construction and representation of knowledge, adaptation to the environment, dynamically processing, retrieving and storing information regarding past experiences, and making connections. Cognitive processes are complemented by motivational systems responsible for identifying needs and setting goals, and emotional systems enabling relationships with the world, perception of surroundings, responses to stimuli, and communication. Amygdalae, areas of the brain that process emotions, are also concerned with learning (LeDoux, 1995). However, learning was generally associated with cognition and motivation, and to a lesser extent emotion. “For decades, learning was mainly analyzed in terms of cognitive or motivational aspects. As a consequence, learning theories ignored affective processes for a  13  long period of time. In order to gain a deeper insight into the complex area of learning they focused on cognition only” (Hascher, 2010, p. 13). Moreover, emotional aspects of learning were once considered detrimental (Astleitner, 2000; Boler, 1997).  Current research in neuroscience and education indicates a strong relationship among cognition, emotion, and learning; however emotion is no longer considered a hindrance to learning. Research in cognitive science and neuroscience demonstrate that emotion and learning are connected to an extent that “learning doesn’t take place when there’s no emotional arousal” (Weiss, 2000, p. 46). Emotion is strongly correlated with motivation: LeDoux (1996) stated that “once emotions occur they become powerful motivators of future behaviors. They chart the course of moment-to-moment action as well as set the sails toward long-term achievements” (p. 20).  Emotion in learning processes is a complex issue that can be addressed from different perspectives: affective states of students and teachers, emotions that enhance or hinder learning, emotions directed towards the object or means of learning (e.g., technology, tools, books), deconstruction of socially accepted concepts of “good emotions” opposed to “bad emotions”, positive versus negative emotions, or critique of normative emotional responses expected in institutions like schools, universities and business organizations. However, this study is focused on identifying what emotions should be considered when designing educational software and how emotional cues and affordances can be incorporated into ALSs and ITSs for better learning. The relationship between emotion and learning is well documented. Astleitner (2000) empirically validated his FEASP theoretical instructional design approach employing five emotions (Fear, Envy, Anger, Sympathy, Pleasure) and demonstrated the existence of a  14  significant correlation between sympathy and pleasure-related instructional strategies and corresponding emotions. Ingleton (2000) analyzed interactions that give rise to emotions of pride or shame. “Accompanying these emotions are positions of solidarity or distance, attendant with confidence or fear. Together these are the basis of decision making, conscious or unconscious, about immediate or future action” (p. 88). The “mood-congruence-hypothesis” proposed by Bower (1981) is based on the idea of cognitive networks and predicts that mood congruence affects cognitive processes: information with a positive content (e.g., feedback from a successful exam) is easily recalled in a positive mood similarly, negative information is recalled in a negative mood. Hence, a positive environment is an “optimal precondition for holistic and creative thinking as it does not force the learner to cope with the situation but enables open-mindedness” (Hascher, 2010, p. 15). However, this is a simplistic approach and the “valence of a mood or an emotion (being positive or negative) is only one aspect of its quality” (Hascher, 2010, p. 16).  Overall, educational research is lacking empirical and theoretical studies of emotion and learning (Astleitner, 2000; Ingleton, 2000; Hascher, 2010). Criticizing this situation, Hascher (2010) argued: “Taken together, there are a handful of limited but very interesting theories but we need more empirical evidence about them, we need to investigate the effects of different emotion qualities, and we need to figure out the range of their validity” (p. 16).      15  1.4.3. Affective Computing With respect to affective computing, studies of emotion in the context of interaction with ALSs and ITSs have been employed by research groups conducted by Baker, Conati, D’Mello, Frasson, Graesser, Lester and Rodrigo.  Conati (2002) and Conati and Maclaren (2009) have proposed a framework that models the set of emotions taken from the OCC theory of emotion: admiration, joy, regret, reproach, pride and shame (Ortony, Clore, & Collins, 1988) . In the design of their Emotional Machine, Trabelsi and Frasson (2010) also used the OCC theory of emotion to predict user’s emotion. Other work (Baker, D’Mello, Rodrigo, & Graesser, 2010; Rodrigo, et al., 2008; Rodrigo, et al., 2012), has relied on a framework proposed by Graesser and D’Mello (D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006) which includes boredom, confusion, delight, flow (also named engaged concentration), frustration, and surprise as emotions considered relevant for learning. Flow was defined by Csikszentmihalyi (1990) as a positive mental state of feeling energized, focused and fully immersed in an activity.  D’Mello, Taylor and Graesser (2007) argued that even if some of these states (i.e., confusion of flow) might be viewed as “purely cognitive in nature, our position is that they should be classified as affective states (or emotions) because these states are accompanied by significant changes in physiological arousal compared with a “neutral” state of no apparent emotion” (D’Mello, Taylor, & Graesser, 2007). Using this framework, Graesser et al. (2006) found that emotion classification performed by trained judges was more reliable than classification performed by peer judges. The same framework has been used by various researchers to measure the likelihood of transition from one affective state to another. D’Mello, Tyler and Graesser (2007) focused on students interacting with a dialogue-based  16  ITS. McQuiggan, Robison and Lester (2008) investigated the likelihood of affective transitions in a narrative-centered educational game (Crystal Island), showing that engaged concentration was the predominant state for learning. Baker, D’Mello, Rodrigo and Graesser (2010) studied the incidence, persistence, and impact of these emotions with three different learning environments. They found that boredom was the most persistent state in each environment; however, engaged concentration was the most frequent state, followed by confusion. Rodrigo et al. (2008) found that two learning environments, Ecolab and M-Ecolab, were not able to disrupt the persistence of boredom and frustration. Chaouachi and Frasson (2012) found that the learners’ mental workload and engagement are related to emotions valence (positive/negative) and activation (high/low) with a significant main effect of valence on workload and engagement. 1.4.4. Methodology With respect to methodology, this exploratory study used an empirical identification of emotion with a design based research (DBR) paradigm by combining exploration with design (Design-Based Research Collective, 2003) and a triangulation mixed methods design (Gay, Mills, & Airasian, 2006) employing (1) a quasi-experimental study (pretest, intervention, posttest, followed by post–questionnaire and interview) and (2) affect analysis (emotion labeling process and video annotations) performed by trained judges. After performing video annotation, judges were formally interviewed. Characteristic of my approach is the fact that, even if qualitative and quantitative data from student participants were collected simultaneously, the quantitative data were analyzed first. The qualitative information was used to better interpret the quantitative information, give a deeper understanding, and elaborate on the quantitative results.   17  Three stages of data analysis were employed in the data analysis process of this study: affect analysis (quantitative approach), descriptive and causal-comparative analysis (quantitative and qualitative approach), and triangulation. Triangulation was used for cross-checking data from different sources and between data collected from students and from interviews with judges. Data analysis included descriptive statistics, t-test, and Cohen’s Kappa analysis (using SPSS software) categorization, content analysis, and matrix analysis. A more detailed theoretical framework and methodology is included in Chapter Two, Literature Review and Chapter Three, Methodology. 1.5. Terminology Intelligent Tutoring Systems- ITSs are “systems that provide detailed guidance (e.g. through hints, feedback, after-action review, or individualized problem selection) as learners work through complex problem scenarios and hone their understanding and problem-solving skills” (Aleven, Graesser & Beal, 2013, p. 930). In order for a tutoring system to qualify as an ITS, it should provide customized instruction and have a domain model, a student model, and a tutoring model (an instructional strategy). A more detailed discussion regarding ITSs follows in section 0.   Affect / Affective State / Emotion- The Latin etymology of the word “emotion” is composed of the words “e(x)” meaning “out” and “movere” meaning movement. Originally, the word was associated with physical movement, agitation, and later to mental disturbance. The English word was inherited from the French emouvoir. Originally, the meaning of the word “emotion” was associated with moving, a physical state. “Motivation” also originated  18  from “movere;” therefore from a linguistic point of view, emotion and motivation are closely related. For this dissertation research, emotions are complex phenomena and are “typically about some personally meaningful circumstance” (Fredrickson, 2001, p. 219). The research adhered to cognitive-appraisal theories of emotion, which agree that emotions are responses of a person’s assessment of a situation or an event. The appraisal process can be conscious or unconscious and triggers physiological, cognitive and motivational responses. Emotions are episodes characterized by synchronization of many different cognitive and physiological components that occur for a short time period when the subject responds to an event (Scherer, 2001b), opposed to moods that are more persistent. I adhere to the argument that mental states like confusion and engaged concentration (or flow) are also emotional states because of their valence (positive or negative) and the changes in subject’s physiological arousal (D’Mello, Taylor, & Graesser, 2007). In many instances, this dissertation uses the terms “affect”, “affective state” and “emotion” interchangeably. For example, emotions such as boredom, confusion, frustration, pleasure, etc. are often described as both “emotions” and “affective states” (Baker, D'Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Rodrigo, et al., 2008; Rodrigo, et al., 2012). “Affect” and “affective states” are not the same thing, however in the same literature it is common to refer to the response to emotions as “response to affect” or to adaptation to emotion as “adaptation to affect”.   Positive and Negative Emotions- Emotions are classified based on the result of their evaluation to have two valences: positive and negative. It is general knowledge that positive  19  emotions are related to well-being and satisfaction and negative emotions to aversion, dissatisfaction or danger. Ortony and Turner (1990) noted: “As far as the concept of emotion in general is concerned, the defining feature that we consider most reasonable and least contentious is that the appraisal underlying the emotion be valenced, either positively or negatively” (p. 323). “Although positive emotions can occur in adverse circumstances, the typical context of positive emotions is not a life-threatening situation” (Fredrickson, 2001, p. 221). By contrast, negative emotions carry adaptive benefits and generally represent “the sort of actions that likely worked best to save human ancestors' lives and limbs in similar situations” (Fredrickson, 2001, p. 220).   For this dissertation, emotions were classified as positive and negative based on their valence from a neutral state as perceived by a person and generally culturally accepted. Accordingly, boredom, confusion, disappointment, displeasure, frustration and shame are negative emotions and curiosity, confidence, delight, excitement, pleasure, and pride, are positive emotions. Surprise is neutral since it doesn’t have a valence. A much more detailed discussion of emotion follows in Chapter Two.  1.6. Limitations of the Study This study has the following limitations:  1. One limitation of this study is the external validity with respect to the number of participants. Though there was not a problem with the number of data points that were used for emotion analysis, data were collected and analyzed from 15 students; the emotions identified or observed need to be tested on a larger sample.   20  2. The study took place in a laboratory and observers were present with participants. A more extensive study will be conducted at a later date with a larger number of participants and in a more naturalistic school setting.  3. The educational game used in this study incorporated a simplified ALT; however findings from this study can be used for design of ALTs and ITSs and for the redesign of the Heroes of Math Island game as an ALT/ ITS employing better customized instruction, a student model, and a more effective affective agent.  1.7. Dissertation Overview and Its Structure  This thesis is organized into five chapters. The current chapter introduced the reader to the background of this study, presenting the statement of the problem, the research questions, purpose and a brief description of the theoretical framework.  Chapter Two is a critical review of literature related to epistemological aspects of emotion, philosophical perspectives of emotion and cognition, emotion frameworks, emotion and learning, emotion identification in affective computing, and ITSs.  Chapter Three presents a description of the Heroes of Math Island game used as a testbed for this study, the study design, participants, materials and procedures, and the methodology. The organization of Chapter Three follows the organization of the study in two parts and stages:  First Part:  Quasi-Experimental Study  Second Part:  Affect Analysis: Emotion Labeling Process and Video Annotations Stage One:  Emotion Labeling Process Stage Two:  Video Annotations by Trained Judges  21  Chapter Four presents the data analysis: affective states analysis (quantitative), descriptive and causal-comparative analysis (quantitative and qualitative) towards (1) affective domain, (2) cognitive domain and (3) learning with respect to attitudes towards (a) the game, (b) mathematics content and (c) affective agent of the game.  Chapter Five includes conclusions, practical implications, recommendations and directions for future research.  22  Chapter Two: Literature Review   2.1. Introduction How does learning occur? What transformations occur in a learner such that he or she becomes knowledgeable in a topic not-known before? What triggers these transformations? What enables learning? To what extent can internal or external stimuli facilitate a faster, easier, or more efficient learning process? In my teaching practice I often witness a change in my students during their time in class: most become not only more knowledgeable, but more mature, confident, and articulate. This transformation is accompanied by situations when students are curious, frustrated, intrigued, confused, happy, disappointed, bored, or proud.  Learning was always associated with cognition, but throughout the history of Western philosophy, cognition and emotion were separated, and emotion was even considered detrimental to learning. “Emotions and the Educative Process,” commissioned and reprinted ten times during 1938 and 1961 by the American Council of Education, advised disciplining emotions and recognized that labile (defined as over-emotional) students are a challenge to the educational system (Boler, 1997). More recently and on the same note, Astleitner (2000) argued that instructional designers neglected emotion because “they expect them to disturb the achievement of important cognitive learning objectives” (p. 171) and Um, Song and Plass (2007) mentioned studies that concluded that even positive emotions can have a negative impact on performance. At the beginning of the twentieth century, John Franklin Bobbitt, pioneer of curriculum development, applied scientific management to the field of education and  23  proposed a scientific approach to curriculum making. His work resonated with the rise of science and technology and with the belief that science can solve all problems. Using the same scientific tradition, social scientists started to analyze and measure emotions in education by the end of World War II (Boler, 1997). Boler (1997) analyzed emotions and power relationships in educational and cultural contexts and argued that educational systems are built on rationalist, Aristotelian models that are designed “to discipline young people’s social and moral values and behaviors” concluding that “this moral conduct is inextricably tied to emotional control” (Boler, 1999, p. 30). The Aristotelian tradition mentioned by Boler (1997) is consistent with Tyler’s rationale and scientific discourse. In the same way that Bobbitt applied scientific management to education, the Aristotelian logical model was used to explain and discipline emotions. Boler (1997) quoted the famous Aristotelian maxim: “Anyone can become angry – that is easy. But to be angry with the right person, to the right degree, at the right time, for the right purpose, and in the right way – this is not easy” (p. 209).  Contemporary learning theories generally consider emotion important to the learning process and agree that emotion and learning are interrelated (Arnone, Small, Chauncey, & McKenna, 2011; Astleitner, 2000; Boler, 1997; Hascher, 2004, 2010; Ingleton, 2000; Petrina, 2007; Picard and Klein, 2002; Picard, Kort and Reilly, 2003; Scherer, 2001a; Weiss, 2000). Petrina (2007) argued that learning theories “deal with specific notions of feelings, knowledge, and skills by addressing the problem of how we learn” (p. 154). However, research with respect to emotion and learning is scarce and fragmented and more theoretical and empirical studies are needed (Astleitner, 2000; Hascher, 2010; Ingleton, 2000).    24  2.2. What is Emotion? This dissertation research addressed emotion in a context of interaction with ALTs and ITSs.  Important discussions around definitions of emotion were carried in the last two centuries: what is emotion and how is emotion elicited? A central debate referred to precedence: is emotion determined by physiological changes or cognitive processes, or is the emotion involved in generating them? Significant questions should be debated and attempted to be answered before discussing emotion in the context of ALTs and ITSs: is emotion intentional, how are emotions classified, what is the relationship between cognition and emotion, is emotion rational / irrational, conscious / unconscious, and how emotions are classified and modeled?  Emotion definition and classification is controversial and challenging (Deigh, 1994; Gratch & Marsella, 2004; Lazarus, 1984; LeDoux, 1995; Scherer, 2010). Different meanings and frameworks of interpretation and methodologies for study are employed in the study of emotion: biological, computational, physiological, psychological, psychomotor, sociological, or political. Emotion and cognition were discussed with respect to primacy: in the past cognition was considered more important and emotion was associated with bodily sensations [e.g., Locke’s (1695) feeling-centered conception of emotion] or excitations from the soul (e.g., Descartes, 1649). However, contemporary philosophy and science agree that emotion and cognition are interdependent (Deigh, 1994; Gratch & Marsella, 2004; Lazarus, 1984; LeDoux, 1995; Picard, 1997; Norman, 2004; Scherer, 2001b).    25  2.3. Theories of Emotion Emotions had several, sometimes conflicting, views as special states of consciousness, unconscious reactions, bodily sensations, instincts, attitudes, or mental states.  2.3.1. Feeling-Centered, Traditional and Contemporary Cognitivism Three important paradigms evolved over time in emotion theory: a feeling-centered paradigm where emotions are considered bodily sensations, a biological paradigm in which emotions are explained as biological reactions needed for survival, and a cognitive paradigm that involves thought and cognitive processes in emotion definition. It is important to note that in some cognitivist theories emotions are equated with cognitive processes (e.g., Rolls, 1995, 2000), and in many current theories biological and cognitivist aspects are combined (e.g., Lazarus, 1984; LeDoux 1995, 1996).  A feeling-centered paradigm of emotion theory was prevalent over time since ancient Greece; however important early philosophes used a cognitivist approach when theorized emotion as a state of the mind: in his Republic, Plato defined emotion as a basic component of the human mind together with reasoning and desiring, and Aristotle defined emotion as a unified body-mind activity. Early modern philosophy (17th and 18th centuries) was preoccupied with emotion definition and interpretation. In his Passions of the Soul (Les passions de l'âme) Descartes (1649), a cognitivist and a pioneer of modern philosophy, argued that there is a separation between body and soul (mind) and defined “passions” (emotions) as perceptions of the soul (mind) detaching them from bodily sensations. Descartes was very influential for other philosophers like Locke (1695), also a cognitivist who viewed emotions as internal sensations produced by thoughts, separated from bodily sensations, and Hume (1739), a feeling-centered theorist, who considered that emotions (he  26  named them passions, sentiments, or tastes) are impressions, generated independent of reasoning.  In the feeling-centered paradigm, emotions are defined as turbulent states and bodily sensations to an extent that they “have no intrinsic relation to cognition” as in Hume’s theory (Deigh, 1994). Feeling-centered theories were scrutinized and critiqued by cognitivists, as well as Descartes’ theory of body/mind separation, which was later challenged, especially by modern neuroscientists (e.g., Damasio, 1994).  In the nineteenth century, Darwinian evolutionary theory emerged and emotion was regarded as adaptation whose purpose is to solve ecological and survival problems facing organisms. Following, a feeling-centered paradigm in the late nineteenth and early twentieth century in studies by James (1884), McDougall (1908), Lange and James (1922) and Cannon (1927), emotions were identified with instincts, and neuro-physiological and perceptual processes of internal (thoughts) or external objects (Deigh, 1994; LaRock & Kafetsios, 2004). Around the same time, two psychologists, William James and Carl Lange, proposed a theory of emotion (known as the James-Lange theory) employing that by manipulating the bodily state, a desired emotion is induced (Lange & James, 1922).  Our natural way of thinking about these standard emotions is that the mental perception of some fact excites the mental affection called the emotion, and that this latter state of mind gives rise to the bodily expression. My thesis on the contrary is that the bodily changes follow directly the perception of the exciting fact, and that our feeling of the same changes as they occur is the emotion. Common sense says, we lose our fortune, are sorry and weep; we meet a bear, are frightened and run; we are insulted by a rival, are angry and strike. (James, 1884, p. 190)  27  Cognitivists, alternatively, considered thought as the essential element of emotion and defined emotions in terms of mental states. Cognitivists reacted with two main objections to the feeling-centered theory of emotion: (1) the feeling-centered paradigm cannot explain the intentionality of emotions and (2) the feeling-centered paradigm cannot represent emotions as “objects of rational assessment” (Deigh, 1994, p. 825).  Indeed, generally, emotions have intentionality and an intentional object: we are afraid of a snake or we are happy because of a success, opposed to bodily sensations that do not have intentionality (the painful tooth is an unpleasant sensation that is not directed to anything). Cognitivists believe in the intentionality of emotion (Deigh, 1994; Rolls, 2000), however, as Deigh (1994) noted “intentionality is typical of emotions without believing that it is essential to them” (p. 826). Some emotions are objectless; an example could be in depression when somebody can feel anxious without any object.  In an excellent survey article, Deigh (1994) separated the philosophical cognitivism approach to emotion theory into two main paradigms: traditional and modern cognitivism. Both traditional and contemporary cognitivists conceived emotions as intentional “mental states in which the subject is cognized of some object” that result from operations like sensing, imagining, or remembering “as well as from the intellectual operations of understanding and judgment” (Deigh, 1994, p. 828). Emotion is not an isolated bodily sensation, but a complex phenomenon involving mental awareness: “In experiencing an emotion, something about our circumstances, our lives, or ourselves capture our attention, orients our thoughts, and touches our sensibilities” (Deigh, 1994, p. 829). Characteristic to traditional cognitivism were empirical investigation and introspection as main methodologies of research, and a very general definition of cognition entailing a “concept of thought broad  28  enough to apply to all states of mind with objective content” (Deigh, 1994, p. 827). Board (1971), a traditional cognitivist, defined emotions as experiences with an epistemological object opposed to pure feelings that do not have an object. The feeling-centered paradigm also take into consideration thoughts; however, for the traditional cognitivism, the relation between emotion and though is constitutive (emotions are mental states), opposed to the feeling-centered paradigm for which the relation is causal (e.g., for James-Lange emotions are constituted by feelings aroused by the though). Traditional cognitivism was superior to the feeling-centered theories of emotion as Hume’s and was able to explain their intentionality. According to Deigh (1994), the main differences brought by the contemporary cognitivism were related to the intellectualist position and the method of study. Contemporary cognitivists departed from the empirical investigation and exploration through introspection which were the methodologies employed by the traditional cognitivism. Exploration through introspection is limited and did not offer enough solid grounds for explaining or theorizing emotions, therefore intellectualist, conceptual and linguistic analysis were used and “concepts of anger, fear, envy, shame, pity, and so forth became the real subject of study, and in analyzing these concepts philosophers converged on the conclusion that each entitled thought in the intellectualist sense” (Deigh, 1994, p. 831). For contemporary cognitivists emotions have theoretical depth. Propositional logic and Socratic induction became “orthodoxy in the philosophical study of emotion” (Deigh, 1994, p. 831).  For something or someone to become the object of a specific emotion, the subject should interact (directly or indirectly via imagination) with that object, have knowledge of that object (i.e., to know the nature of that object) or to have the belief that that object has a certain characteristic (i.e., to believe that the object is dangerous). Deigh (1994) observed  29  that belief is an important constituent of processes that generate emotions: “the object of an emotion can have, in the subject’s mind its evaluative character only if the subject believes or judges it to have this character” (p. 836). The subject’s beliefs can create a dissonance between an object’s actual character and the impression of that object in the subject’s mind: “a dangerous man would not be feared if he were not known or believed to be dangerous, and someone with a terminal disease would not be pitied if no one even suspected he was ill” (p. 834). Contemporary cognitivists used prepositional logic to analyze an emotional situation: “a person feels pity for z implies that the person believes z to be of some distress” (Deigh, 1994, p. 831). The second critique of feeling-centered paradigm related to the rational/irrational nature of emotion represents an important debate between feeling-centered theorists and cognitivists. The feeling-centered theorists offered a weak explanation in this respect: “because they assimilate emotions to bodily sensations, cannot explain how an emotion can sometimes be unreasonable or irrational and so (by implication) at other times can be reasonable or rational” (Deigh, 1994, p. 846). For example irrational fear occurs in an individual without a real danger. Deigh (1994) considered that all cognitivist theories should be able to answer the question of rationality. In Deigh’s opinion, belief is a key concept and can shed light on the problem of rationality: he mentioned Davidson (1980), who defined emotions as “logical outcomes of desires and beliefs that combine to produce them” and “can be described as reasonable or unreasonable, rational or irrational, according as the beliefs and, more controversially, desires that combine to produce them are reasonable or unreasonable, rational or irrational” (p. 846). Without considering belief, cognitivist theories alone cannot answer the emotion questions. As Deigh (1994) noted regarding a text of  30  Proust, the artist described “a paradigm of an experience of primitive emotion infused with and altered by belief but nonetheless intelligible without it” (p. 853).   2.3.2. Current Findings in Neuroscience and Psychology  Deigh (1994) observed contemporary cognitivism from a philosophical perspective only. He does not include any mention of important psychologists like Ekman, Lazarus, Russell, Scherer and Zajonc, and neuroscientists like Damasio, LeDoux, and Rolls who made important contributions to emotion theory. Discoveries in neuroscience and experiments on people with brain lesions have proven that emotion is processed in specialized brain areas offering an important cognitivist argument. Therefore in the second part of the twentieth century, an important movement in social studies and sciences shifted to a bi-directional cognitive-emotion paradigm, which considered cognitive process as a prerequisite of emotional experience, as well as emotion being an important factor in thinking and decision making. Intentionality and theoretical depth was augmented by science, clinical studies, and technological advances. However, not all contemporary emotion theories are cognitivist: several feeling-centered theories in which emotion precedes cognition had been proposed, the most known being offered by Zajonc (1980, 1984) who also argued that cognition and emotion belong to independent neural structures of the brain. An important current standpoint is that emotion and cognition are interrelated: Lazarus, the inventor of appraisal theory demonstrated in his laboratory and in theory that causality is bi-directional: events initiate evaluations that lead to emotions that produce physiological response and trigger coping mechanisms (Lazarus, 1982, 1984, 1991), Damasio (1994) considered that emotion is influenced by cognition, precedes cognition and influences decision-making, and Rolls (2000) argued that emotion influence cognitive  31  processes by facilitating “continuity in the interpretation of the reinforcing value of events in the environment” (p. 181). Several theories of emotion associate emotion, cognition and motivation, first noted as the “trilogy of mind” by Plato. Current cognitivist theories of emotion consider that emotion is highly connected to the brain activity and embedded in the brain structure. Many scientists like Lazarus and Rolls agreed with LeDoux (1996) that “brain systems that generate emotional behaviors are highly conserved through many levels of evolutionary history” (p. 17). Appraisal theories (Lazarus, 1984; Rolls, 2000) involves a relational aspect, a motivational aspect, and a cognitive aspect and agree that stimuli provoke appraisal of events (generally known as primary appraisal) which generate emotion and emotion in turn generates a new evaluation (secondary appraisal) that results in decision making. Appraisal theories are substantiated by the studies of brain mechanisms and the neural basis of emotion: when a stimulus is received, it will circulate fast from the sensory organs (eyes, ears) to the amygdala (amygdalae are two almond-shape brain structures located deep within the medial temporal lobes of the brain), and be processed immediately for fast generation of an appropriate emotion that will activate the cortex for example in the case of fear to provide a fast response to danger (LeDoux, 1995). Clinical studies demonstrated that lateral nucleus of the amygdala respond to auditory stimuli and are essential in auditory fear conditioning and that neurons in the amygdala respond to visual emotion processing and recognition; for example bilateral damage to the human amygdala impairs retrieval of emotional and social information from faces (LeDoux, 1995, 2007). Information from the amygdala and also from the sensory system arrives at the hippocampus (part of the brain that is responsible with  32  memory forming, organizing, and storing) through the perirhinal cortex and will generate the second appraisal.  In contemporary theories, emotion is observed not only internally (cognitive — as brain activity or in terms of feelings — as physiological changes inside the subject), but as complex cognitive-affective states observed in relationship not only to an object but to the environment (Deigh, 1994; Lazarus, 1984; Rolls, 2000). According to Lazarus (1984), emotion reflects “a constantly changing person-environment relationship” (p. 124) and cannot be justified by feeling-centered theories as physiological arousal. Generally conceptualized as “an organic mix of action impulses and bodily expressions, diverse positive or dysphoric (subjective) cognitive-affective states, and physiological disturbances” (p. 125), emotion should be defined as a combination of “behavior, subjective reports, or physiological changes: its identification requires all three components” (Lazarus, 1984, p. 125). In Lazarus’ (1982, 1984) opinion, appraisal and cognitive activity are a pre-condition of emotion because “to experience an emotion, people must comprehend” (p. 124). Lazarus offered a very appealing explanation of how emotion happens: “What would transform sensory states into emotions? The transformation necessary to produce an emotion out of sensory states is an appraisal that these states are favorable or damaging to one’s wellbeing” (p. 126). However, although Lazarus believes that “cognition (of meaning) is a necessary precondition of emotion, this does not imply that emotions, once elicited, do not affect cognition. Emotions appear to be powerful influences on how we think and interpret events. They are the result of cognition but in turn affect cognition” (p. 126).  Lazarus’ work was heavily critiqued by Zajonc (1984) who believed in the independence of cognition and emotion and in primacy of affect: “affect can be aroused  33  without the participation of cognitive processes and it may therefore function independently for those circumstances” (p. 119). Even if Lazarus offered empirical evidence as proof of his theory, Lazarus explained his differences with Zajonc from a larger perspective: he argued that their differences are due to philosophical and not just scientific arguments. It is interesting to observe that Lazarus took a philosophical position when responding to Zajonc, declaring himself a constructivist and Zajonc as neo-positivist, and arguing in a postmodernist way that there are “ many styles of explanation that can be scientifically rigorous” (p. 126).  In LeDoux’s (1995, 1996) view, emotion is a brain process corresponding to a circuit in the brain that computes the value of an experience: “only a label, a convenient way of talking about aspects of the brain and its mind” (p. 16). Emotions cannot be treated uniformly because different classes of emotions are mediated by different neural systems: “The system we use to defend against danger is different from the one we use in procreation, and the feelings that result from activating these systems—fear and sexual pleasure—do not have a common origin…There is no such thing as the ‘emotion’ faculty and there is no single brain system dedicated to this phantom function” (LeDoux, 1996, p. 16).  Based on research on brain function and design with regard to how emotion is elicited, a definition of emotion is provided by another important figure in neuroscience, Rolls (2000) who offered a neo-behaviorist theory of emotion based on adaptation when defined emotion as based on reward and punishment and deeply embedded in evolution: “Why is the brain built to have reward and punishment systems, rather than in some other way? Raising this issue of brain design and why we have reward and punishment systems, and emotion and motivation, produces a fascinating answer based on how genes can direct  34  our behavior to increase fitness.” (p. 177). In his view, emotions are “states elicited by rewards and punishments, including changes in rewards and punishments” (p. 178). Rolls agreed with the general cognitivist perspective that emotion is intentional, has an object and consists of cognitive processes. Rolls also offered a definition of moods as affective (emotional) states without intentional object that “may occur in the absence of an external stimulus “(p. 179). Rolls’ (2000) view of emotion is functional and motivational as emotion and motivation are equated: “emotion is motivation” (p. 180). According to Rolls (2000), emotion is needed for eliciting autonomic and endocrine responses; provide “flexibility of behavioral responses to reinforcing stimuli” (p. 179) and is used in communication, social bonding, and in memorization for both facilitating information storage and recall of memories. Rolls offered also a model of emotion based on intensity of reinforcement contingencies. Rolls (2000) also argued that emotion influences cognition, and storage and recall of memories. Rolls’ theory of emotion was accepted and approved in the scientific community but with reservations; his view of emotion as a solution to adaptation and his schema based on reward and punishment was critiqued especially for reductionist and behaviorist reasons; rewards and punishments alone cannot explain emotion.  As indicated, an important issue related to the conscientious / unconscious nature of emotion was popularized by James (1884) in his famous question whether we run from an animal because of fear (conscientious) or the fear occurs because we run. By associating emotions to body sensations (running from danger will generate fear), feeling-centered theorists argued that emotion is not conscientious. Running from danger is characteristic to all primates and this suggests that fear is an emotion conserved in evolution, but is it unconscious?   35  Current cognitive scientists depart from Descartes’ idea that mind is just conscientious and agree that many brain activities are unconscious. According to LeDoux (1996) the unconscious consists of “processes that take care of the mind’s routine business without consciousness having to be bothered” (p. 30) and even speech is based on unconscious processes: “Speech, consciousness’ favorite tool, is also the product of unconscious processes. We do not consciously plan the grammatical structure of the sentences we utter. There simply isn’t enough time. We aren’t all great orators, but we usually say things that make sense linguistically. Speaking roughly grammatically is one of the many things that the cognitive unconscious takes care of for us” (p. 31). Many defense responses like freezing or flight and fight response as well as facial expressions and visceral reactions accompanying these situations are based on unconscious processes and are not learned, but hard-wired into our nervous and endocrine systems (LeDoux, 1995). However, LeDoux (1996) argued that emotion needs consciousness: “Emotional feelings result when we become consciously aware that an emotion system of the brain is active. Any organism that has consciousness also has feelings” (p. 302).  Rolls (2000) also attempted an answer to the question of consciousness of emotion with his HOLT (High-Order Linguistic Thought) theory of consciousness which consider that somebody can have consciousness only if has a concept of thought and language to describe that thought, therefore animals and even babies lack phenomenal consciousness. Emotion is related to language and early acquisition of basic-level concepts (Deigh, 1994; Ortony & Turner, 1990; Rolls, 2000). Deigh (1994) noted: “what marks human beings as rational creatures and sets them apart from other species and the very immature of their own species is the special importance of language in human life. Its pervasive impact on human  36  thought and feeling is obvious to anyone upon self-reflection. Human beings, as they mature, learn to speak and to encode their thoughts in language. As their facility for language improves and the store of their encoded thoughts enlarges, they develop an increasingly powerful system of beliefs on which they rely in negotiating their way through life.” (p. 849). Deigh (1994) observed that contemporary cognitivists use this as an argument to restrict their theories of emotion to human psychology, leaving out the very young and animals. Rolls’ theory was critiqued for its reductionist nature and because it cannot explain emotions that do not have an objective motivation (e.g., fear of a nonexistent event like a disaster that is not eminent or even probable). Several authors critiqued HOLT: Korb and Nicholson (2000) believed that HOLT is not a well-considered theory of emotion: “Rolls holds that the distinguishing feature of consciousness is that first-order thoughts (e.g., about the world) become objects of a higher-order thought (e.g., a belief about the first-order thought)” but “dismisses as evidence of consciousness what are commonly taken as paradigmatic, such as suffering, joy, and so forth” (p. 206). Frijda (2000) noted that many rewarding conditions like “altruism, courtship, or solving an intellectual problem” do not “fit the notion of stimuli” (p. 199) and emotions “may not just equal the instigations for solving adaptational dilemmas. They are, I would say, instigations for maintaining or changing relationships with the environment, whether for solving adaptational dilemmas, for fitting in with elementary social satisfactions, or for answering relational urges that just are part and parcel of certain biological autonomous systems. These issues surrounding “reward,” innate forms of behavioral instigation, and motivational phenomena are essential in understanding emotions, but remain mysteries in the emotion theories that are limited to a reinforcement basis” (Frijda, 2000, p. 200).  37  2.4. Emotion and Cognition The relationship cognition/emotion is very important and controversial, and needs more attention. To synthesize previous discussions, there are several theories: cognition precedes emotion (Lazarus, 1982, 1984); emotion precedes cognition (Zajonc, 1980, 1984); cognition and emotion are independent (Zajonc, 1980, 1984); emotion is just cognition (Rolls, 2000); the cognitive, emotional and motivational systems are interrelated (Lazarus, 1982; 1984; LeDoux, 1995; Ortony & Turner, 1990; Scherer, 2001b). Modern philosophers, scientists and theorists generally follow a cognitivist model, without necessarily agreeing with a cognition-first approach. Generally, the role of emotion is downplayed in favor of cognition or reason. In Western cultures from the period of the nineteenth century up until the early twentieth century reason was considered the bonding element of public life, missing an essential observation that a society is first built on cultural and religious affinities and traditions which are in fact representations of emotions shared by its members (Nicholson, 1997).  In the early 1980s, a strong cognitivist movement in emotion theory took place in the scientific community based on discoveries of neural pathways of emotion and cognitive appraisal theories, and having as a main thesis the idea that cognition, emotion and motivation are inseparable. However, theories of primacy of affect and independence of cognition from emotion were also established.  Robert Zajonc, a social scientist, published in 1980 an article in the American Psychologist titled “Feeling and Thinking: Preferences Need No Inferences” which received the Distinguished Scientific Contribution Award from the American Psychological Association and generated an important debate regarding the primacy of affect over  38  cognition. His main thesis is that “affect and cognition are under the control of separate and partially independent systems that can influence each other in a variety of ways, and that both constitute independent sources of effects in information processing” (Zajonc, 1980, p. 151). Zajonc (1980) argued that experiments have proven that affective reactions cannot be suppressed and are often the first reactions of the organism, do not depend on cognition, and can be produced in the “total absence of recognition memory” (p. 151).  The architect of appraisal theory, Lazarus (1982) a distinguished researcher and professor emeritus of psychology at the University of California, argued that emotions require cognitive mediation (appraisal), cognition and emotion are “fused in nature”, and the full emotional response is a combination of “thoughts, action impulses, and somatic disturbances” (p. 1019). In his view, cognitive appraisal mediates the relationship between the person and the environment meaning that “the way one interprets one’s plight at any given moment is crucial to the emotional response”. Emotion results from sensory states, which are evaluated to determine if they are “favorable or damaging to one’s wellbeing” (p. 126). However, the link between thought and emotion can be broken and cognitive coping processes such as “isolation and intellectualization (or detachment), which are aimed at regulating feelings, can create a dissociation between thoughts and feelings” for example attack can happen without anger and “avoidance without fear” (p. 1019).  Zajonc (1984) critiqued Lazarus’ model considering that emotion in fact precedes cognition. An advocate of the independence of the emotional system from the cognitive system, Zajonc argued that emotion can be aroused without the participation of cognitive processes, that the emotional and cognitive systems have separate neuroanatomical structures (emotions are under the control of the right hemisphere) bringing as argument that emotions  39  generated by olfactory and gustatory stimuli are immediate, and affective states are induced by non-cognitive and non-perceptual procedures. In 1980 Zajonc wrote: “processing of affect is probably an even stronger candidate for the right hemisphere than the processing of pictures” and all sorts of judgments are faster and more efficient for pictures than for words, and this may be so just because pictures are able to evoke an affective reaction more directly and faster than words. An affective reaction aroused early in the encoding process earlier than it is possible for the interoceptive and motor memories to become effective—might facilitate a complex cognitive encoding sequence by an initial categorization along affective lines, which as we have seen, requires minimal stimulus information” (pp. 168-169). The idea of separation between cognition and emotion belongs to the old philosophical argument that cognition is rational and emotion is not, and was one of the main critiques of the feeling-centered approach. There are many reasons to disagree: for example, emotions can be anticipated (it had happened to everybody to only imagine at an unpleasant situation and have a negative emotion of fear or sadness), and more importantly can be suppressed and educated, opposed to reflex processes (like startle) that cannot be controlled.  Zajonc (1980) noted: “in contrast with cold cognitions, affective responses are effortless, inescapable, irrevocable, holistic, more difficult to verbalize, yet easy to communicate and to understand” (p. 169). Zajonc offered a very important theory and brought into attention the importance of affect. However, Zajonc’s idea of separate brain structures and models of processing for cognition and emotion is not yet substantiated by research; Lazarus (1984) considered Zajonc’s view as reductionist and ungrounded. However, there are differences between the two brain hemispheres with respect to emotion processing; more recent studies by LeDoux (1996) suggest that the left hemisphere is  40  responsible with positive emotions: “Studies of affective changes in patients with unilateral brain injury suggest that the left frontal region is particularly important for certain forms of positive affect and when this region is damaged, depression is a likely consequence” (p. 114) From a neuroscientific position, LeDoux (1995, 1996) offered an appraisal theory similar with Lazarus’ considering that emotional and cognitive systems influence each other and are mediated by interacting systems of the brain. More recent studies of cognitive sciences indicated that the amygdala is an important part of the emotional memory system and the hippocampus is part of the brain that is involved in cognitive and declarative form of memory, however this “does not prove that the systems operate by different information processing rules, but it certainly leaves open the possibility” (LeDoux, 1995, p. 226). LeDoux (1996) recommended a neuroscientist approach in studying emotions: “I view emotions as biological functions of the nervous system. I believe that figuring out how emotions are represented in the brain can help us understand them. This approach contrasts sharply with the more typical one in which emotions are studied as psychological states, independent of the underlying brain mechanisms, Psychological research has been extremely valuable but an approach where emotions are studied as brain functions is far more useful” (p. 12).  An important question arises: is cognitive appraisal just information processing? I agree that Zajonc (1984) had a point in bringing into attention critical questions such as: “what is the minimal information process that is required for emotion, and if untransformed, pure sensory input can directly generate emotional reaction or not?” (p. 122). Responding to Zajonc’s criticism, Lazarus (1982, 1984) noted the difference between information processing (simple retrieving of information, encoding, and storing) and meaning making:  41  the “most serious mistake in Zajonc’s analysis lies in his approach to cognition…In this approach information and meaning stem from the conception of mind as an analogue to a computer” (p. 1020). Information processing is only one of the elements needed in meaning making, and in fact “we do not have to have complete information to react emotionally to meaning. We can react to incomplete information, which in fact we do in most ordinary transactions” (Lazarus, 1982, p. 1021). According to Lazarus (1982), not only humans but also mammals are equipped with emotion and emotional response based on appraisal: “Probably all mammals meet the minimum requirements of emotion if one permits the concept of appraisal to include the type of process described by ethologists in which a fairly rigid, built-in response to stimulus arrays differentiates danger from no-danger” (Lazarus, 1982, p. 1023).  I agree with appraisal theories: emotion is preceded by cognitive processes. An example can be how news is received: if news is good, positive emotions are elicited opposed to bad news that arouses sadness. However, if news is given in a language that is not understood, no emotion is being elicited: in order to react emotionally, the content of the message should be comprehended. What about a circumstance of fear associated with an immediate threat? The same situation can happen: if the threating situation is not comprehended as dangerous, fear will not happen. Some emotions depend more on cognitive activities than others: for example the reaction of fear to an immediate dangerous situation like the attack of a wild animal is processed in the amygdala and happens very fast, opposed to a similar emotion, anxiety, which involves symbolic representations, is anticipatory, and can occur without a real threat (Lazarus, 1982).   42  I argue that you cannot draw a line between emotion and cognition and in many cases they are so interleaved that it is hard to identify where a cognitive process ended and an emotion started. The way we think influences that way we feel; however this does not contradict in any way that the way we feel also influences the way we will evaluate a future situation. Lazarus (1984) invited research to clarify “how various personal agendas, such as values, goals, and commitments, as well as beliefs or expectations about oneself and the world, shape cognitive appraisal over the life span and, in so doing, affect that propensity to experience certain emotions in particular environment contexts” (p. 129).  Klaus Scherer, an important contemporary theorist and specialist in the psychology of emotion, proposed the component process model (CPM) of emotion. According to this theory, emotions are synchronized to many different cognitive and physiological components. In the CPM framework, emotions are defined as episodes of “interrelated, synchronized changes in the states of all or most of the five organismic subsystems in response to the evaluation of an external stimulus event as relevant to major concern of the organism” (Scherer, 2001b, p. 93). Scherer identified the organismic subsystems as: information processing, support, executive, action and monitor. Therefore, emotions are episodes that occur for a short time interval when the organismic subsystems are coupled and synchronized to produce an adaptive reaction.  Appraisal theories are currently predominant for explaining the relationship between cognition and emotion and also the most utilized perspective in designing of artificial intelligent systems (Marsella, Gratch, & Petta, 2010).    43  2.5. Emotion Classifications  Some emotions had been studied in more details than other; for example fear was studied well because of precise experimental procedures for eliciting and measuring, but other emotions, especially positive emotions are less understood. LeDoux (1995) invited research to focus on positive emotions: creating “new models of positive affect is also important” (LeDoux, 1995, p. 222). Emotions are very different, are based on different mechanisms of perception and should be studied independently: “We shouldn’t mix findings about different emotions all together independent of the emotion…Unfortunately most work in psychology and brain science has done this” (LeDoux, 1996, p. 16). Theories of emotion use very different mechanisms for classifying emotion: scientists like Ekman and LeDoux argued for establishing a set of distinct emotions with neurological underlying processes and well distinguishable effects. Ekman argued that his six basic emotions act as building blocks of human emotional repertoire and can be used to distinguish emotions from other affective phenomena based on a set of characteristics and muscular feedback from facial expressions (Ekman, 1999).  Several taxonomies of emotions have been developed (Ekman & Friesen, 1999; Ortony, Clore, & Collins, 1988; Rolls, 2000; Russell, 2003). Ekman and Friesen developed in 1978 a taxonomy of facial expressions (FACS or Facial Action Coding System) that allows a codification based on a set of 32 action units corresponding to constriction or relaxation of facial muscles. Many studies of ITSs (De Silva & Ng, 2000; Fellenz et al., 2000; Sebe et al. 2002) used Ekman’s (1993) six basic categories of emotion: happiness, anger, sadness, surprise, disgust and fear within the FACS taxonomy. Ekman’s framework was not restricted to facial expressions: studies of emotion recognition based on voice or  44  textual information also used Ekman’s six basic emotions (Nasoz, Alvarez, Lisetti, & Finkelstein, 2004). Theorists from classical psychology argued that these basic emotions are universally displayed and recognized (Pantic & Rothkrantz, 2003). However, in 1999 Ekman revised his previous views and concluded that there are differences in facial expressions of emotions between cultures and even within a particular culture, and emotions are dependent of the social context (Ekman, 1999).  Other theories define emotions according to one or more dimensions: for example, Russell and Feldman Barrett’s (1999) model based on emotion’s arousal and valence and Rolls’ (2000) model based on emotion’s intensity. Ortony, Clore, and Collins (1988) offered a framework of emotion known as OCC that is widely used in artificial intelligence (especially in game development) for the development of intelligent agents (emotion synthesis). The OCC model has “established itself as the standard model for emotion synthesis” (Bartneck, 2002, p. 39). Bartneck (2002) reported that several studies employed the OCC model for the design of characters that express emotions.  An important question that arises is to what extend are for example Ekman’s set of six emotions (or basic emotions identified in other taxonomies) basic? Theorists argued for different sets of basic emotions: Ortony and Turner (1990) mentioned Mowrer’s (1960) set for only two emotions pleasure and pain, Watson’s (1930) three basic emotions of fear, love, and rage, and Oatley and Johnson-Laird’s (1987) theory of happiness, anxiety, anger, and disgust. The set of basic emotions proposed by different theorists and scientists often does not even overlap: some proposed emotions like courage (Arnold, 1960) or anticipation (Plutchik, 1984) are not included in any other list. The controversy doesn’t stop at the number and nature of basic emotions, but what qualifies as an emotion is debatable. Some  45  emotions might be described under different terms: for example Ortony and Turner (1990) believed that Panksepp (1982) when identified expectancy and Plutchik (1984) when identified anticipation perhaps both referred to desire. However, it is remarkable to observe that nearly all taxonomies included fear, happiness, and sadness (Ortony & Turner, 1990).  Figure 1 represents the selection of lists of basic emotions from Ortony and Turner (1990, p. 316). Ortony and Turner noted that not all theorists represented here are “equally strong advocates of the idea of basic emotions” (p.316). Some consider that basic emotions are an important concept (e.g., Panksepp, 1982; Plutchik, 1980; Tomkins, 1984), whereas for others (e.g., Mowrer) it is of “peripheral interest only, and their discussions of basic emotions are hedged” (p. 316).    46   Reference Fundamental emotion Basis for inclusion Arnold (1960) Anger, aversion, courage, dejection, desire, despair, fear, hate, hope, love, sadness Relation to action tendencies Ekman,  (1999) Anger, disgust, fear, joy, sadness, surprise Universal facial expressions James (1884) Fear, grief, love, rage Bodily involvement McDougall (1926) Anger, disgust, elation, fear, subjection, tender-emotion, wonder Relation to instincts Mowrer (1960) Pain, pleasure Unlearned emotional states Oatley & Johnson-Laird (1987) Anger, disgust, anxiety, happiness, sadness Do not require propositional content Panksepp (1982) Expectancy, fear, rage, panic Hardwired Plutchik (1980) Acceptance, anger, anticipation, disgust, joy, fear, sadness, surprise Relation to adaptive biological processes Tomkins (1984) Anger, interest, contempt, disgust, distress, fear, joy, shame, surprise Density of neural firing Watson (1930) Fear, love, rage Hardwired  Figure 1. Selection of lists of basic emotions. Adapted from Ortony and Turner (1990, p. 316). Ortony and Turner (1990) questioned the necessity of having a set of basic emotions. They agreed that commonly the reason for proposing basic emotions is to provide “an explanation of some routine observation about emotions” (p. 317). Important conclusions  47  arise from related literature: emotions are universal and occur in all cultures and in some animals, some emotions seem to be “universally associated with and recognizable by characteristic facial expressions”, and some emotions “appear to serve identifiable biological functions related to the survival needs of the individual and of the species” (Ortony & Turner, 1990, p. 317). Generally, basic emotions are decided based on two main criteria not independent of each other: biological, based on evolutionary reasons like protection and procreation which can be found across cultures and in other species, and psychological. The psychological view “often starts on the position that there is a limitless number of emotions” and the basic emotions are needed as primitives “in the study of other, nonbasic emotions by developing some kind of combinatorial model” (Ortony & Turner, 1990, p. 317) Another important discussion related to emotion definition is related to what criteria should be used to classify a state as emotional. As found in literature, states are generally differentiated between affective (or emotional), cognitive and motivational. For example, startle was debated if qualify as an emotion or not, considering the lack of cognitive appraisal: Lazarus (1984) noted that startle is not an emotion because it cannot be inhibited opposed to emotions that can be inhibited. Ekman, Friesen and Simons (1985) dedicated an entire study to “startle” (feeling startled) that was inconclusive: in some respects (the uniformity in facial expression and the brief latency) startle reassembles an emotion; however in some other respects (i.e., easiness to be elicited, not totally inhibited, and because it cannot be simulated with the correct latency) is not an emotion. Deciding whether startle is an emotion requires scientific inquiry which would support “Zajonc’s and contradict Lazarus’s claim about whether or not cognitive appraisal is a prerequisite for emotion” (Ekman, Friesen, & Simons, 1985, p. 1424), and a philosophical perspective. According to  48  the feeling-centered theories startle could qualify as emotion. It is interesting to note that surprise, which is very similar to startle, is included in several lists of emotions (see Figure 1). Ekman’s set of six basic emotions include surprise; however, Ekman, Friesen and Simons (1985) argued that someone feels different when startled from being surprised.  I agree with Ortony and Turner (1990) that very important in classifying a state as emotional or not, is determining whatever or not that state is an affectively valenced state, positive or negative. Based on this rule, fear would qualify as an emotion because it has a negative valence when startle and even surprise would not, because are unvalenced. Ortony and Turner (1990) considered that being” affectively valenced is a necessary condition for a state to be an emotion” and argued that surprise can be better qualified as a cognitive than an affective state (p. 317). Other states that are disputable are interest and desire (see Figure 1); in Ortony and Turner standpoint they both qualify as motivational states. Emotion and motivation are interrelated (see also LeDoux, 1995); however, “the fact that emotion and motivation are often causally related does not entail that they are reducible one to the other” (p. 318). Basic biological emotions like happiness, sadness, fear, terror, pleasure possess evolutionary significance; possible their response is hardwired, and therefore, should be universal across cultures. Facial expressions related to happiness, anger, disgust, sadness, and fear are indicated by similar facial expressions in Europe, the United States, Japan, and Africa, and even in cultures such as “Dani of Iran and the South Fore of New Guinea” which had “virtually” no contact with the Western world (Ortony & Turner, 1990, p. 320). However, it was noted that facial expressions typical of one emotion can be observed in relationship with another emotion, therefore a more “prudent approach” is to “admit that the  49  universal production of distinctive facial expressions is neither necessary nor sufficient for (basic) emotions” (Ortony & Turner, 1990, p. 321).  In regard to basic emotions as psychologically primitive, Ortony and Turner (1990) explored the use of different criteria. Possible criteria used for determining the basicity of an emotion are: (1) elementary eliciting conditions and the (2) lack of other emotions as constituents. For example frustration is not on the list of basic emotions, but anger is; it can be judged that frustration is a kind of anger, but equally anger can be considered as a form of frustration. I would add that based on the first criterion, frustration can be elicited before anger and therefore frustration should qualify as basic. Same with distress; fear can be viewed as a special reaction to distress and therefore distress can qualify as a primitive emotion. However, as Ortony and Turner (1990) noted: “The difficulties with regarding anger and fear as psychologically primitive and the absence of frustration from most list of psychologically basic emotions suggests that emotions might sometimes be proposed as being basic on grounds that are irrelevant to whether they are basic in the psychological sense” (p. 325). Another criterion can be the saliency of an emotion with respect to culture: in this case, embarrassment would qualify as primitive, however, it was not proposed as basic emotion by any theorist.  Emotion is very important in the context of human communication and interaction. Consider the analogy between emotion and natural languages offered by Ortony and Turner (1990):  the best analogy that can be drawn is one between There are hundreds of human languages, and many more are possible. Yet, linguists do not seek to explain them by postulating a small set of basic languages out of which all others are built (even though linguists acknowledge that there may have been an  50  evolution from a few early ones). At the same time, they certainly recognize that there are constraints on possible languages, and they recognize that there are basic building blocks of languages such that any one language comprises some particular subset of a finite (but large) number of basic constituent elements (e.g., phonological properties and syntactic properties). Furthermore, some of the constraints have their roots in the biological nature of people. What is basic, however, are the constituents of languages, and these constituents are patently not themselves languages. So, too, with emotions. We conclude that the study of emotions is no more dependent on the existence of a nontrivial subset of basic emotions in terms of which all other emotions can be explained than is the study of language dependent on the existence of a small subset of elemental languages, or the study of animals on a set of basic animals. (p. 329)  I agree with Ortony and Turner (1990) that in fact there is not a “satisfactory criterion of basicness” (p. 329) and other strategies should be used to classify emotions like perspective and specificity of field of research and study. For this research, I propose a set of emotions not based on basicness criteria, but on ALTs, ITSs and affective computing perspectives, and on the meaning of a specific emotion with regard to learning.  2.5.1. Dimensional Models of Emotion: Core Affect  An important contribution was made by Russell and Feldman Barrett (1999) when opposed models based on basicness and proposed in 1980 a model of emotion based on two dimensions: arousal and valence. Since 1980, Russell and Feldman Barrett revised their model acknowledging that the original model did not provide “a sufficient rich account of prototypical emotional episodes” (Russell, 2003, p. 150). Russell and Feldman Barrett (1999)  51  and Russell (2003) critiqued theories based on basic emotions: a problem in using concepts like fear or anger as psychological primitives is incorrect, because they are more complex and imply cognitive structures. Russell (2003) agreed with Oatley and Johnson-Laird (1987) and other cognitivists (Lazarus, 1984; LeDoux, 1995, 1996) when argued that emotions are not independent, but about something therefore there is an intentional object; however, “an emotion directed at an Object is a complex event, not a primitive element” (p. 147). Instead, the term “core affect” is more suitable for a primitive because is object free: pleasure or displeasure, tension or relaxation, depression or elation. Core affect represents the “elemental feelings included with prototypical emotional episodes” (Russell & Feldman Barrett, 1999, p. 806). Core affect is a neurophysiological state “consciously accessible as the simple raw (nonreflective) feelings evident in moods and emotions” Russell, 2003, p. 148). Therefore, Russell and Feldman Barrett (1999) proposed a model where emotions (more complex entities) are mapped on core affect (primitives). See Figure 2: the inner circle represents the core affect and the outer circle corresponds to emotional episodes.  Figure 2. Framework of emotion. Adapted from  Russell and Feldman Barrett (1999, p. 808).   52  2.5.2. Emotion as a Consequence of Reward or Punishment  Rolls (2000) observed that “emotions can be produced by the delivery, omission, or termination of rewarding or punishing stimuli” (p. 178) and proposed a framework of emotions based on reward or punishment. His taxonomy takes into consideration the intensity of emotion: for example the intensity increases in the case of a positive reinforcer from pleasure, to elation and to ecstasy ( see Figure 3). In his notation, “s+” is a positive reinforcer and “s-” is a negative reinforcer; “s+!” is the termination of a positive reinforcer and “s-!” is the termination of a negative reinforcer. Emotions can be accounted based on a number of factors including the “reinforcement contingency (e.g., whether reward or punishment is given or withheld” (on the vertical dimension), the intensity of the reinforcer (the distance from the origin), the termination of a reward (on the left side) or a punisher (on the right side), and a “number of different reinforcement associations. For example, a stimulus might be associated both with the presentation of a reward and of a punisher, allowing states such as conflict and guilt to arise” (Rolls, 2000, pp. 178-179). If a negative reinforcer is used, for example the sight of a stimulus that is about to produce pain, depending on the intensity, the resulting emotion can be apprehension, fear, or terror. Then a negative stimulus (punishment) is removed (termination), the subject will feel relieved; however, when a positive reinforcer (reward) terminates, depending on the intensity, and the nature of the reward either frustration, anger and rage is perceived, or sadness and grief. In Rolls’ (2000) view, the “great majority of rewards and punishers are external stimuli not related to internal need states such as hunger and thirst, and these stimuli do produce emotional responses” (p. 179).  53   Figure 3. Framework of emotion based on reinforcement. Adapted from Rolls (2000, p. 179).    2.5.3. Emotion Synthesis: OCC Cognitive Theory Ortony, Clore, and Collins (1988) proposed a cognitive theory of emotion known as the OCC model.   The OCC model (see Figure 4) specifies 22 emotion categories based on the valenced reactions to situations concerning consequences of events (i.e., fear, joy, hope and pity), actions of agents (i.e., admiration, pride, reproach and shame), and aspects of objects (i.e., love and hate). Two categories combine to form a group of compound emotions, namely emotions concerning consequences of events caused by actions of agents: anger, gratification, gratitude and remorse (Ortony, Clore, & Collins, 1988).   54  In the OCC model, appraisals are psychological aspects of situations that distinguish one emotion from another. The model synthetizes emotion into five phases: classification, quantification, interaction, mapping, and expression. Emotions result as consequences of situations that are classified by a person based on goals, standards and attitudes, are quantified (evaluation of intensity) in interaction with the occurring events, actions and objects, mapped on the 22 emotion categories, and finally expressed (Ortony, Clore, & Collins, 1988).    Because emotions are defined in terms of events, actions, and objects, the OCC model is widely used in artificial intelligence for the development of intelligent agents. Several studies in related research used the OCC model: Conati et al. for designing the affective student model in the educational game for learning factorization Prime Climb (Conati & Maclaren, 2009) and Trabelsi and Frasson (2010) for the Emotional Machine, an online framework developed to “understand” and to predict user's emotional reaction.   55   Figure 4. Global structure of emotion types. Adapted from Ortony, Clore and Collins (1988, p. 19). 2.6. Affective Computing Emotion is not only theorized, but studied at the neurological level, classified, modeled, and simulated; Ekman and Friesen (1999), Ortony, Clore, and Collins (1988), Rolls (2000), Russell (2003) developed taxonomies of emotions; Balkenius and Moren (2001), Gartch and Marsella (2001; 2004) and Scherer (2009) offered computational models; and scientists like Conati, D'Mello, Frasson, Gratch, Lester, Marsella, and Picard included emotion in the design of artificial intelligence and human-computer interaction models.  56  The association between intelligence and emotion in computing led to the development of a new branch of artificial intelligence: affective computing. Affective computing is a multidisciplinary field involving cognitive science, computer science and psychology concerned with computational modelling of emotion and design and implementation of intelligent agents capable of expressing and responding to emotion. Theoretical and practical research directives are considered: (1) theoretical aspects: identify the mechanisms of cognitive appraisal and emotion-cognition interactions, define architectural requirements for emotions, and the role of emotion in the adaptive behavior, and (2) practical aspects in terms of enhancing the human-computer interaction for affect-adaptive interfaces, affective user models, and virtual affective agents (Hudlicka, 2006). The field expanded with important contributions from theorists and scientists from artificial intelligence, human-computer interaction and psychology: e.g. Aleven, Baker, Conati, D'Mello, Frasson, Graesser, Lester, Marsella, Petta, Rodrigo, and Scherer.  Adapting to affect and affect modeling are important areas of current research. Scherer, who is the current director of the Swiss Center for Affective Sciences in Geneva and one of the most influecial contemporary theorists in the field of affective computing suggested that appraisal theories of emotion constitute the most comprehensive and effective way to represent the complexity of the emotion process “spanning the whole gamut from low-level appraisal of the eliciting event to high-level influence over behavious” (Scherer, 2010, p. 19). Additionaly, they have the advantage of offering specific “hypoyheses for the underlying mechanisms that received consistent support in experimental research” (p.19). Appraisal theories of emotion are appealing to computer scientists because cognitive activities can be logically simulated, therefore designed and implemented into intelligent  57  agents. Currently, the large majority of computational models of emotion are derived from appraisal theory which is the “predominant force among psychological perspectives on emotion and arguably the most fruitful source for those interested in the design of symbolic AI systems, as it emphasizes and explains the connection between emotion and cognition” (Marsella, Gratch, & Petta, 2010, p. 27).  The information processing component is straightforward to comply, when the meaning making element is still a very complex problem to be solved as information processing as an “exclusive model of cognition is insufficiently concerned with a person as a source of meaning” (Lazarus, 1982, p. 1020). More recently, Scherer (2010) argued that “we are still many decades away from the moment in which an emotional Turing test would fail to identify the artificial alternative” (p. 311). 2.7. Cognition, Emotion and Learning Learning is an active process of combining new and existing information and building new mental or physical skills that results in a change in attitude, behavior, performance, mental skills, and relationship with the environment and other members of the community. Learning doesn’t equate to cognition; emotion, memorization, motivation, participation and socialization are important aspects of the learning process.  The amygdala, two almond-shape brain structures consisting of a number of distinct nuclei, are involved in emotion processing, memory consolidation, and learning (Balkenius & Morén, 2001; LeDoux, 1995, 2007; Rolls, 2000; Sylwester, 1994). Amygdala, recognized as a brain formation in the early 19th century, is a complex brain structure for a long time neglected, which became the object of study in several areas of psychology and cognitive science especially in relationship with emotional functions and processing. New technologies  58  like magnetic resonance imaging (MRI) allowed for observation of brain areas that are activated during cognitive and emotional processes, and understanding of neurotransmitter systems. Several emotions had been demonstrated to be correlated with the amygdala: fear (as in Pavlovian fear conditioning when a conditioned stimulus is paired with a painful shock, resulting in a fear-motivated avoidance conditioning tasks), and emotional states associated with aggression, maternal, sexual and ingestive (eating and drinking) behaviors. Not only emotional, but also cognitive functions like reward learning are correlated to amygdala (LeDoux, 1995, 2007).  External or internal stimuli are transmitted to the amygdala through two pathways: a shorter thalamic pathway (the thalamus is the center for incoming sensory information which sends very fast information to the amygdala for triggering emotion), and a longer thalamo-cortico-amygdala pathway that processes and represents stimuli in more detail. The direct shorter pathway to amygdala is needed for survival; it triggers rapidly emotions, for example the fight-or-flight response to fear. Rolls (2000) noted that the amygdala is activated in both pleasant and unpleasant emotions and is involved in “learning the emotional and motivational value of stimuli” (p. 185). LeDoux (1995) argued that the lateral nucleus of the amygdala “may be the crucial site of the cellular changes that underlie learning” (p. 216) and “plasticity in the amygdala could represent the integrative (stimulus- and response-independent) aspects of learning” (p. 217). Learning at the cellular level involves changes in synaptic transmission between neurons (LeDoux, 1995). Proven in the case of fear, the stabilization of memory via protein synthesis after learning (called consolidation), and re-consolidation when a fear memory was retrieved, occur in the lateral amygdala (LeDoux, 2007).   59  An adjacent structure to the amygdala, the hippocampus is responsible with consolidation of short-term memory into long-term memory. The two structures, the amygdala and the hippocampus collaborate in modulating the subjective and objective strength of memory: amygdala processes the subjective feelings associated with an event, and the hippocampus processes the objective details: location, time, actions, etc. (Sylwester, 1994).  The amygdala is not the only structure of the brain involved in emotion and learning: after stimuli are evaluated in amygdalae, signals are sent to other structures of the brain that further process emotions. For example, the hypothalamus is responsible with motivational control, and the frontal cortex is involved in motor function and also in short-term working memory and more detailed emotional processing (Balkenius & Morén, 2001; Rolls, 2000).  Emotional respondents (like the behavioral and visceral reactions that occur in an emotional situation, for example fear generated by danger) are not learned but hard-wired in our brain (LeDoux, 1995). Respondents are activated by outputs from the amygdala and are affected by Pavlovian conditioning. However, after emotional respondents are expressed, emotional operants occur (for example deciding what strategies are more appropriate to escape from danger). Emotional operants are learned through “instrumental (operant) conditioning procedures” (LeDoux, 1995, p. 228). Emotional respondents had been studied in details, but less was researched with respect to emotional operants (which are more complex) and coping responses (LeDoux, 1995). Making a connection with the appraisal theories, emotional respondents are results of primary appraisal and emotional operants are results of secondary appraisal.    60  2.8. Emotion Models with Respect to Learning “Although learning and emotion are interdependent, the influence of emotion on learning has to be theoretically analysed and empirically investigated” (Hascher, 2010, p. 15). Important questions need to be answered: what emotions are actually happening during the process of learning? What emotions influence learning and how? What emotions are beneficial to learning? What conditions would generate them?  Common sense as well studies indicated that positive emotions are facilitating cognitive processes involved in learning by promoting knowledge construction and problem solving (Ingleton, 2000; Isen, Daubman, & Nowicki, 1987; Sylwester, 1994; Um, Song, & Plass, 2007). Are in fact only positive emotions facilitating learning? And last, but not least, how and to what extent can designers of educational software or authors of learning material create conditions for eliciting these emotions?  Hascher (2004, 2010) disagreed that students’ emotions in schools are related to school events extrinsic to learning. In Hascher’s studies, students reported emotions that were intrinsic to learning processes, however were characterized by high variability as they  were induced “by teacher instruction, partner work, classroom discussions, single learning activities, achievement situations, etc., and these situations elicit a variety of emotions like pride, anger, frustration, happiness, and sadness” (Hascher, 2010, p. 18). She recommended more research to be conducted on emotion and learning indicating that current studies on student emotions are “loosely connected” and the field is “highly fragmented” (p. 19).  Although emotion and learning can be investigated from different perspectives, this study focused on identifying what emotions should be considered when designing ALTs and ITSs, and  61  how emotional cues and affordances can be incorporated in these devices for better learning, memorization and motivation. 2.8.1. Positive Emotions The role of positive emotions as direct or indirect factors of other processes was discussed by several authors in different contexts including positive psychology, a branch of psychology aimed to understand and foster factors that “allow individuals, communities, and societies to flourish” (Fredrickson, 2001), judgment and choice (Lerner & Keltner, 2000), creativity and motivation (Isen, Daubman, & Nowicki, 1987; Isen & Reeve, 2005) and multimedia learning (Um, Song, & Plass, 2007).  Isen, Daubman, and Nowicki, (1987) found that positive emotions improve problem solving by altering the cognitive context of the learning activity and Isen and Reeve (2005) argued that positive emotions improve the brain processes that provide motivation. According to Fredrickson (2001), positive emotions are important indicators of well-being and satisfaction and “positive emotions signal flourishing. But this is not the whole story: Positive emotions also produce flourishing” (p. 218). Fredrickson believes that positive emotions have more long-term benefits not necessarily related to evolution and adaptation:  In contrast to negative emotions, which carry direct and immediate adaptive benefits in situations that threaten survival, the broadened thought-action repertoires triggered by positive emotions are beneficial in other ways. Specifically, these broadened mindsets carry indirect and long-term adaptive benefits because broadening builds enduring personal resources, which function as reserves to be drawn on later to manage future threats. (Fredrickson, 2001, p. 221)   62  However, the role of positive emotions needs more research.  Fredrickson (2001) observed that although action tendencies (actions that a person takes when an emotion occur) have been determined for specific positive emotions, “the action tendencies identified for positive emotions are notably vague and underspecified” (p. 220). On the same note, Um, Song and Plass (2007) argued that even though “the facilitation hypothesis is dominant in positive emotions related research, the effect of users’ positive emotion in learning process is still not understood well” (p. 4177).  Responding to this challenge, Chaouachi, and Frasson (2012) explored the relationship between workload, engagement and emotional states during learning activities of trigonometry and concluded that “higher level of alertness and attention elicit a high emotional activation however these emotions tended to be positive when the mental demand is low and negative in the opposite case” (p. 70).  Using “affective priming” (p. 4), a masked priming technique that involves exposing participants interacting with an ITS to affective stimuli, Jraidi, Chalfoun and Frasson (2012) demonstrated that positive emotions are beneficial: pairing self-referential words such as “I” or the name of the participant (conditioned stimulus) with positively valenced words such as “efficient” or “success” (unconditioned stimulus) resulted in better learning outcomes, self-esteem and positive emotional reactions.       Positive emotions are not always seen as having positive benefits. Um, Song and Plass (2007) noted that emotions experienced during cognitive processing of learning materials can be viewed as “imposing unnecessary load in working memory, i.e., can be interpreted as extraneous cognitive load” (p. 4177). They mentioned research done by several authors including Ellis and Ashbrook (1987), Oaksford, Morris, Grainger, and Williams  63  (1996), and Seibert and Ellis (1991) that shown negative effect of positive emotions explained by cognitive load theory and “consistent with the suppression hypothesis that mood can take extra-task processing or task-irrelevant processing and it will have a negative effect on reasoning and performance” (p. 4177). In a study which examined to what extent positive emotions are enabling better multimedia learning, Um, Song and Plass (2007) used a mood-induction procedure to elicit states of happiness and sadness before learning sessions in which two designs of learning material aimed to generate either no emotions, or positive emotions were used: one described as “neutral emotional design”, and one that employed “positive emotional design” incorporating  better quality aesthetics, better color combination and immersion (p. 4180). They concluded that positive emotions induced before learning were maintained throughout the learning process, promoted knowledge construction and problem solving, and showed better transfer. Good-aesthetic instructional design of learning materials increased the positive emotions of those learners whose emotional state was neutral. The authors used the theory of cognitive load to explain interesting findings arising from their study: “if either positive emotions or good design quality of material is presented during the learning process, learners invested more mental effort, but if both manipulations are presented at the same time, they would not” (Um, Song, & Plass, 2007, p. 4185).  A critique of Um, Song and Plass’s (2007) study is their general use of the word emotion and their lack of differentiation between emotions of the same valence. First, it is important to distinguish between emotions that are exterior and irrelevant to the learning activity and emotions that occur during learning and have as objects the learning activity or material. Emotions that were induced before learning through mood-induction procedures  64  fall under the first category, whereas emotions produced by the aesthetic quality of the learning material are under the second.  2.8.2. Emotion in Instructional Design – FEASP (Fear, Envy, Anger, Sympathy, Pleasure)  Astleitner (2000) proposed an emotional design model that can be implemented in instructional design starting from the idea that cognitive, emotional and motivational processes are “related to the world in different ways”, and emotions although “interact with cognitive and motivational processes, are considered in research as an unique component of human mental states, experiences, and behavioral expressions” because they “may initiate, terminate, or disrupt information processing” (p. 169). He suggested a framework of emotional design of instruction (EDI) composed by strategies aimed to improve learning by decreasing negative and increasing positive emotions. Astleitner (2000) stated that emotions in instruction were neglected because (1) many teachers are not even considering addressing them in instruction because they “believe that emotional goals are so long range and intangible that regular classroom time restrictions prohibit the development of desired emotional outcomes” (p. 171) and (2) emotions are expected to disturb learning and therefore guidelines of instructional design not even include them: “emotions are no issue at all, at best, they should be avoided during learning” (p. 171). Progress was made in 1980s when instructional design included motivation. In the new paradigm, emotion was seen as a consequence of learning (pride, despair or tranquility felt after a school performance) and also as an element that stimulates motivation by “increasing curiosity” (p. 171). In the 1990s, an important improvement in instructional design took place represented by “affective” or “holistic education” in which “affects were not  65  considered as the positive or negative valence of an emotional experience” but as “attitudes…set toward or against certain objects or situations” (Astleitner, 2000, p. 172). However, strategies employed were not based on theories of emotion, and no differentiation was made with respect to different kinds of emotions. Another improved approach was represented by “emotional education” in which emotions are the content of instruction and students are taught how to express, interpret, manage and monitor emotions. As a result, emotion was introduced in the daily instruction as learning material with emotional-related content.  Astleitner’s EDI model of “emotionally sound instruction” is based on two important assumptions: (1) EDI is focused on conscious experiences: “emotional instruction should influence conscious mental states” (p. 173), and (2) emotions considered in EDI are selected based on four criteria. Criteria used to establish what emotions should be considered are: (a) emotions that are included in theoretical models of emotions; (b) do not need treatment; (c) are important in the social context of school; and (d) are “not conveyed by cognitive and motivational design approaches” (p. 174). Astleitner (2000) started with Plutchik’s (1984) set of basic emotions: joy, sadness, acceptance, disgust, fear, anger, surprise, and anticipation. He eliminated sadness because of criterion (b) (a sad child needs medical attention), disgust because the criterion (c), and surprise and anticipation because of criterion (d). Instead of acceptance, he considered two emotions: one positive (sympathy) and one negative (envy).  In Astleitner’s research, there are five emotions (three negative and two positive) that should be taken in consideration in instructional design for designing “instruction which is emotionally sound”: "Fear…arising from subjectively judging a situation as threatening or dangerous…Envy ….resulting from the desire to get something that is possessed by others or  66  not to lose something that one is possessing…Anger…coming from being hindered to reach a desired goal and being forced to an additional action…Sympathy…referring to an experience of feelings and orientations of other people who are in the need of help… and Pleasure…based on mastering a situation with a deep devotion to an action” (p. 175). Figure 5 illustrates the instructional strategies that are suggested by Astleitner (2000). Strategies F1 to F4 are related to fear, considered as a very important emotion for humans (see also LeDoux (1995, 1996) that main occurrence in schools is in “test anxiety” (Astleitner, 2000, p. 177). I agree with Astleitner's connection between emotional, and cognitive and motivational aspects of instructional design: strategy F1 “Ensure success in learning” can be obtained through “well-proven motivational and cognitive instructional strategies” (p. 191).    67   Figure 5. General instructional strategies. Adapted from Astleitner (2000, p. 191).   68  I also agree with Astleitner’s approach: it is crucial to start from a theory of emotions and focus on specific emotions and not on general positive/negative categories; it is important to clearly define the emotions and criteria used to select them, and most importantly, it is crucial to decide what emotions are relevant in the school/instruction context. However, Astleitner’s (2000) framework resolves problems related to only a subset of emotions: fear, envy, anger, sympathy and pleasure. His strategies are sometimes too general (i.e., F1, P1), too simplistic (i.e., A4), or difficult to implement (i.e., S1).  Astleitner’s (2000) study doesn’t indicate any findings resulted from empirical research implementing EDI in real-life situations. However, I believe that his model could be an important source of inspiration especially because of his proposed tactics that can be employed when emotion is detected. Astleitner (2000) invited instructional designers and researchers to further investigate emotional processes in the learning context on a larger scale. 2.8.3. Emotion, Identity and Learning From a constructivist position, Ingleton (2000) critiqued current learning theories which neglected emotion even if it is evident that emotion can influence learning in many ways: at the individual level inhibiting or promoting effective learning and approaches to study, and on a larger scale in a broader socio-cultural context. She offered a model of emotions in learning based on evaluations of real-life classroom experiences, and on Scheff’s (1997) theory of social bond that connects identity, emotion, and learning. Ingleton (2000) connected learning to the social bond theory, implying that social relationships of solidarity and alienation are essential in identity formation. In an earlier study, Salzberger-Wittenberg, Henry and Osborne (1983) analyzed the emotions felt in early learning experiences and  69  found that they continue in present. They also found that confidence is higher if social bond and self-esteem would have been high. The link between the social bond and confident expectation is a central element in the theory of emotion and learning proposed by Ingleton (2000). In her study, Ingleton found that pride and shame are “significant in the classroom experiences which make learning possible … as they are fundamental in the formation of confidence, anxiety and fear” and “are central in the construction of identity, and so are significant in the theorizing of emotion and learning” (p. 87). Opposed to confidence, anxiety and fear are prevalent for persons who have been denied acceptance and recognition.    pride  solidarity  confidence social relationships      disposition to learn   Shame  distance  fear  Figure 6. Model of emotion and learning. Adapted from Ingleton (2000, p. 88).   Ingleton (2000) agreed with Barbalet (1998) that confidence is a particular emotion because it” is the only emotion that has time as its object” (p. 88); based on confidence future is apprehended as possible and actions are decided. She offered a model based on two dimensions connected in a casual relation: social relationships and disposition to learn (see Figure 6). The model suggests that “relationships with teachers, peers and parents as giving rise to emotions of pride or shame…solidarity or distance, attendant with confidence or fear” and all these emotions influence “decision-making, conscious or unconscious, about immediate or future action” (p. 88) and therefore impact the disposition to learn. Ingleton validated her hypothesis in a qualitative study of confidence in the mathematics classroom,  70  her choice being based on the fact that often students and teachers report strong feelings related to learning and teaching mathematics and statistics. The study started with a stimulus statement: “Maths is constructed as an objective, emotion-free discipline, yet the study of maths generates strong negative emotions among many learners“(p. 91). Participants organized in two groups (six graduate students and five mathematics lectures) were invited to write narratives related to encouragement/discouragement in learning mathematics. Ingleton (2000) concluded that judgments related to correctness or incorrectness of performance are particularly more frequent in a mathematics classroom and generate shame and pride, which in turn are very important emotions in self-identity and self-esteem and form possible the “basis of maths anxiety” (p. 99).  2.8.4. Analytical Model of Emotions and Learning Picard, Kort and Reilly (2003) offered an analytical model of learning that incorporates emotions [also presented in Kort and Reilly (2002)]. Their model was built as part of a larger project aimed to design, develop, and evaluate a prototype named the Learning Companion. The Learning Companion is a computer program that reacts when a student shows emotional signs of becoming confused, distracted, anxious, worried, etc. (Picard, Kort & Reilly 2003). The purpose was to research a new educational model that can be applied in contemporary schools where students learn in the context of STEM (Science, Technology, Engineering and Mathematics). The proposed model started from observing the regular students/teacher interaction:  In an attempt to install/build/re-engineer the current state of educational pedagogy, educators should first look to expert teachers who are adept at recognizing the  71  emotional state of learners, and, based upon their observations, take some action that scaffolds learning in a positive manner. (Kort & Reilly, 2002, p. 8)  According to Kort and Reilly (2002), teachers or observers of a learning activity can assess the affective emotional state of learners based on external factors like tone of speech, body language, or facial expressions. Accurately identifying a learner’s cognitive-emotive state is a “critical observation that will enable teachers to provide learners with an efficient and pleasurable learning experience” (Kort & Reilly, 2002, p. 9). Using a model inspired from science and technology, they provided an in-depth description of emotional states that could facilitate learning. Kort and Reilly (2002) proposed a set of emotions not addressed in previous frameworks (e.g., determination, dispiritedness, hopefulness, and puzzlement).                 Figure 7. Analytical model of emotions and learning.  Adapted from Kort and Reilly (2002, p. 10). I II II IV Positive Affect Constructive Learning Negative Affect Un-learning Curiosity Satisfaction Determination Hopefulness Dispiritedness Frustration Confusion Puzzlement Investigate Fresh Research Discard Misconceptions Diagnosis  72  Kort and Reilly created an axis of emotions that are possible relevant to learning (Figure 7). According to this model, students begin their learning journey in quadrant I (Investigate) or II (Diagnosis) by being either fascinated or curious about a new topic, or motivated to solve issues of confusion. By exploring the new topic, dynamic processes occur that can lead, if the topic is difficult or the student gives a wrong solution, to quadrant III (Discard Misconceptions). The affective state of frustration is likely to occur in quadrant III leading to “un-learning”. During this stage, students start consolidating knowledge and are ready to progress to the next quadrant. Students move to quadrant IV (Fresh Research) where positive emotions of joy accompany new ideas and send them to quadrant I.  The purpose of learning is not to keep the student in quadrant I, but help her/him move from one quadrant to another or, if the student is mature enough, make her/him recognize the cyclic nature characteristic to learning processes. Kort and Reilly (2002) stated that the circle will not close, but evolve into a spiral on a different knowledge axis. With respect to different axes, a student can be simultaneously in all four quadrants. Kort and Reilly (2002) concluded that their model goes “beyond previous research studies not just in the range of emotions addressed, but also in an attempt to formalize an analytical model that describes the dynamics of a learner’s emotional states, and does so in a language that supports metacognitive analysis” (p. 13).  Kort and Reilly’s (2002) model is interesting because takes into consideration a large variety of emotions explained from a learner’s perspective; however, the paper lacks a justification of what theory of emotion was utilized when these emotions were selected and what learning theory was employed for designing the learning quadrants. Most importantly, the study lacks empirical validation.   73  2.9. Intelligent Tutoring Systems  ITSs were first introduced in Chapter One, and as indicated are characterized by a domain model, a student model, and a tutoring model (Shute & Psotka, 1996). ITSs have two loops:  outer loop executes once for each task, where a task usually consists of solving a complex, multi-step problem. The inner loop executes once for each step taken by the student in the solution of a task. The inner loop can give feedback and hints on each step. The inner loop can also assess the student’s evolving competence and update a student model, which is used by the outer loop to select a next task that is appropriate for the student. (VanLehn, 2006, p. 227)  Initially, ITSs were cognitive tutors (Anderson, Corbett, Koedinger, & Pelletier, 1995).  A cognitive tutor is an ITS that has a computational model that possess capabilities in solving problems in “the ways students are expected to solve the problems" (Anderson, Corbett, Koedinger, & Pelletier, 1995, p. 170). Cognitive tutors make decisions based on rules. According to Aleven (2010), the design of cognitive tutors faces two major challenges: “flexibility to adapt to students’ actual solutions, and cognitive fidelity to accurately correspond to the knowledge components students are actually learning” (p. 57). Several paradigms are used in designing ITSs: model tracing, knowledge tracing (knowledge tracing also requires model tracing), the constraint-based model, and more recently, example-tracing tutors (Aleven, McLaren, M., Sewall, & Koedinger, 2009).  The three separate models stated by Shute and Psotka (1996) are generally included in current ITSs: a knowledge domain (what should be learned), a student model intended to infer a student’s current knowledge, activity and state (e.g., affective or motivational), and a  74  tutoring model employing an instructional strategy (how learning materials are presented, organized, and evaluated).  In recent years, game-based learning environments attracted interest as technologies that harness motivation and support learning in various settings: classroom, university, training in industry and simulation. Many ITSs are designed as educational games (Baker, D’Mello, Rodrigo, & Graesser, 2010; Conati & Maclaren, 2009; Gratch & Marsella, 2004; Linehan, Kirman, Lawson, & Chan, 2011; McQuiggan, Robison, & Lester, 2010; Spires, Rowe, Mott, & Lester, 2011; Woolf, et al., 2009).  2.9.1. Intelligent Tutoring Systems and Emotions  Current research in ITSs focused not only on the cognitive aspects of interaction, but also on affect recognition and response, adapting to student’s metacognitive skills, and prediction of future actions (Baker, D’Mello, Rodrigo, & Graesser, 2010; Conati & Maclaren, 2009; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Jraidi, Chaouachi, & Frasson, 2013; Gratch & Marsella, 2004; McQuiggan, Robison, & Lester, 2010; Rodrigo, et al., 2012). Marsella, Gratch and Petta (2010) observed that more research is now dedicated to computational modeling of emotion and to emotion expression of animated characters:  A related trend in HCI [human-computer interaction] is the use of emotions and emotional displays in virtual characters that interact with the user… emotional displays in an artificially generated character can have the general effect of making it seem human or lifelike, and thereby cue the user to respond to, and interact with, the character as if it were another person. (p. 7)   75  According to Woolf et al. (2009), ITSs that recognize and adapt to affect are effective in class instruction providing an increase of 12% over traditional tutoring systems. In addition, students were more engaged and motivated when the tutor mirrored their emotions.  Conati and Maclaren (2009) argued that “educational systems may be more effective if they can trigger appropriate student affective states” (p. 4). Using the Prime Climb game they compared a system that adapts based on causes of emotions (predictive model) and effect of emotions (diagnostic model) with a model that is only predictive. Conati and Maclaren (2009) concluded that the predictive and diagnostic model had better accuracy in cases where the students’ affective state had a clear valence. One of the outstanding problems in diagnostic and predictive modelling for educational games is the identification of emotional states to model. Jraidi, Chalfoun and Frasson (2012) used implicit strategies of affective priming, which consisted of exposing the participants in the study during intervention to affective stimuli, like positive words or smilling faces, and cognitive priming in the form of an answer or hint (pp. 4-6). One difference in the use of answers and hints compared to studies by Conati et al. (2002, 2009, 2013) is that in Jraidi, Chalfoun and Frasson’s case these interventions were “aimed toward  positively enhancing specific cognitive processes such as reasoning or decision making toward the goal of implicitly enhancing knowledge acquisition” (p. 6) and not directly in response to the learners’ needs as in Prime Climb studies (Conati, 2002; Conati & Maclaren, 2009; Conati & Manske, 2009; Conati, Jaques, & Muir, 2013). Based on data collected with electro-encephalography (EEG), Jraidi, Chalfoun and Frasson (2012) concluded that both self-esteem and abilities to reason in a problem-solving environment can be augmented through the use of these techniques.   76  In the context of detecting emotions during interaction with educational software, Conati et al. (Conati, 2002; Conati & Maclaren, 2009) proposed a framework that models a set of emotions taken from the OCC theory of emotions (Ortony, Clore, & Collins, 1988) (admiration, joy, regret, reproach, pride and shame). Other work (Baker, D'Mello, Rodrigo, & Graesser, 2010; Rodrigo, et al., 2008; Rodrigo, et al., 2012) used a framework proposed by D’Mello and Graesser et al. (D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006). This framework includes boredom, confusion, delight, flow (also named engaged concentration), frustration, and surprise, as emotions considered relevant for learning. Graesser et al. (2006) found that emotion classification performed by trained judges was more reliable than classification performed by peer judges: an inter-judge Cohen’s Kappa agreement of 0.71 was achieved when two trained judges made judgements of affective states at voluntary timestamps; considerably lower values were obtained for all the other possible pairs: self, peer, and one trained judge.   The same framework has been used by various researchers to measure the likelihood of transition from one affective state to another: D’Mello, Tyler and Grasser (2007) focused on students interacting with Auto Tutor, a dialogue-based ITS. They concluded: However, it appears that learners experiencing negative affective states, such as boredom and frustration, are more likely to wallow in these states rather than transition into positive states of flow, delight, or even confusion. (D’Mello, Taylor, & Graesser, 2007, p. 278) McQuiggan, Robison and Lester (2008) investigated the likelihood of affective transitions in a narrative-centered educational game (Crystal Island). McQuiggan, Robison and Lester  77  (2008) found that engaged concentration was the predominant state for learning (reported 42% of the time by participants).  Baker, D’Mello, Rodrigo and Graesser (2010) studied the incidence, persistence, and impact of these emotions with three different learning environments. For all three studies, engaged concentration was the most common state (Cohan’s Kappa of 0.60 - average between the three studies) and confusion was the second most common state. Boredom was the most persistent state in each environment.  Rodrigo et al. (2008) found that two learning environments Ecolab and M-Ecolab were not able to disrupt the persistence of boredom and frustration. In a study from 2012, Rodrigo et al. found that an interactive affective agent Scooter the Tutor, although well-liked by students, did not have an effect on students’ affective states. A recent study by Jraidi, Chaouachi,and Frasson (2013) employed three-modality diagnostic variables that sense the learner’s experience: physiology, behavior and performance. They evaluated a hierarhical probabilistic framework for assessing three extreme trends: “flow (a perfect immersion within the task), stuck (a difficulty to maintain focused attention) and off-task (a drop out from the task)”, and the emotions associated with them stress, confusion, boredom and frustration (Jraidi, Chaouachi, & Frasson, 2013, p. 271). They reported that an accuracy of 82% was reached to assess positive vs. negative emotions and an accuracy ranging from 81% to 90% was achieved with respect to the four emotions.    78  2.10. Conclusion The emotional, cognitive, and motivational processes are dynamic and influence each other. The brain processes continuously information involving a neurotransmitter system and specialized areas responsible for emotional processing and responses; research in neuroscience determined the importance of amygdala as an important structure involved in both emotion processing and memorization, as well as the hippocampus involved in cognitive and declarative form of memory.  Emotion is necessary for survival (e.g., fear protects us from danger), evolution (e.g., the love we feel around babies or young animals, or the pleasure elicited by beautiful landscapes makes us protect youngsters and the environment), environmental interaction (we perceive the changes in the environment at cognitive level and respond emotionally) and social interaction (we tell others about ourselves and understand others through emotions). Emotion is an important way of communication: we express ourselves in an emotional way, and we understand others through facial expressions, modulations of voice, or body postures.  Emotion is a way of knowing: we learn not only our behavior through emotions, but our culture: meanings of things and concepts have deep emotional roots. The context in what learning occurs should produce positive emotions in learners, but equally, just learning new things has positive consequences. In my practice I witnessed many situations when solving a problem, mastering a difficult concept, or helping a colleague who was behind produced feelings of joy and satisfaction. What about negative feelings? Do negative feelings hinder learning? Based on my observations and my own experience, I would answer: “Not always.” Sometimes students are challenged by a difficult concept and feelings of frustration motivate them to work harder.   79  A critique of discourses of emotions grouped based on valence value is that they are too general. In fact it is very different if a student experiences a negative emotion of fear compared with a negative emotion of sadness. It is also important to take into consideration the intensity of an emotion: it is different if a learner is somehow happy or very happy. Even if positive emotions were demonstrated to produce well-being and better outcomes by promoting knowledge construction and problem solving (Isen, Daubman, & Nowicki, 1987; Isen & Reeve, 2005; Fredrickson, 2001; Um, Song, & Plass, 2007), their role was not clearly established and more theoretical and empirical studies are needed. Another important discussion is related to the particular relationship between each emotion and learning: e.g., confidence is a more productive emotion than happiness in a learning context.  Research should focus on particular emotions and their role in the learning process and not on general classifications of positive/negative valence. Attention should be given to emotional responses for identifying emotions in learners and help them developing coping skills. There are several challenges. First, emotions are not static; on a person’s face someone can identify in a very short time interval the passing of several emotions. Emotions are dynamic and include transitions from neutral to emotion, back to neutral and to another emotion. Another important challenge is the fact that emotions are very different and can be theorized or analyzed based on different dimensions. Some emotions are directed to self: happiness, pleasure, pride and sadness; some emotions are directed to others: love, compassion, solidarity, envy, sympathy, and empathy. There are emotions related to needs: happiness and pleasure are related to satisfied-needs, and anger, frustration and sadness to unsatisfied-needs. Happiness and pleasure are different because happiness is moral, when pleasure is physical. Both fear and pleasure are related to evolution: fear is associated to  80  danger and pleasure will promote well-being for survival of species. Several emotions can be classified as in Rolls (2000) from the reward/ punishment perspective: anger, grief, sadness, happiness, and pleasure, but emotions like solidarity and confidence do not meet this criterion.  Adapting to affect is desirable and necessary. Several emotion-cognition theories are used for emotion recognition and modeling and for designing intelligent-affective agents: cognitive theories, appraisal theories, Ekman’s framework of basic emotions, dimensional models, and the OCC model. Emotion recognition is the first step needed for student model acquisition. User input can be extracted from facial expressions, speech, auditory features, psychological signals, textual information, multimodal information and context information. Research efforts are increasingly directed toward emotion classification and designing affect modeling frameworks. Interfaces that adapted to affect generally depend on application. Some frameworks are focused on emotion synthesis, especially in a context of educational games employing animated characters.  An important challenge is related to the lack of unified definition and interpretation of emotion. There is a large body of literature related to emotion classification based on a set of emotions (generally Ekman’s framework of basic emotions) but there is evidently no formal theory of which emotions should be recognized for adapting to affect and to what extent they can be recognized.  Also, the most used framework, Ekman’s basic emotion theory, was based on the assumption that emotions are universal and facial expressions for emotions like fear, sadness, disgust, surprise and happiness are independent of culture (Pantic & Rothkrantz, 2003). Later in his career Ekman (1999) revised his original model and acknowledged that facial  81  expressions are culturally and socially sensitive. Also, strong emotions that could be easily distinguished had been considered, with the exception of Conati and Maclaren’s (2009) study that observed emotions with milder or conflicting valence. Another limitation is a lack of evaluation of models in a real-world situation, with many studies based only on laboratory experiments.  Chapter Three discusses the design of the research and the procedures for data collection.    82  Chapter Three: Methodology  3.1. Introduction The following chapter has two sections. The first section includes the description of game used in this study. The second section is dedicated to the study design (a description of the participants, and the data collection instruments designed and used in this study), and to the data collection methods in an attempt to communicate how data were collected for this study and to describe the quantitative and qualitative methods applied in analysis. 3.2. Section One: Heroes of Math Island The game Heroes of Math Island was used for experiments and data collection.  3.2.1 Game Description The mathematics game Heroes of Math Island developed for this study is intended for students in grades five through seven. Based on my observation and classroom practice, this age group is suitable because at this stage students start learning more complex concepts and also enjoy playing games. Several other studies with similar games by Conati et al. and Rodrigo et al. also targeted this age group (Conati & Maclaren, 2009; Conati & Manske, 2009; Rodrigo, et al., 2008; Rodrigo, et al., 2012). The game is implemented on an XNA professional gaming platform (a runtime environment provided by Microsoft for game development) that allows for implementation of rich game mechanics. Similarly to McQuiggan, Robinson, and Lester (2008) the game interaction is rich, comparable with commercial video games.   83  The game has a narrative and activities happening on an island employing as a central site a castle where students get “quests” from a king or queen. My long-term goal is to create several islands each of them intended to master a specific topic: mathematics, language, geography, etc. Currently there is one island with a potential of five quests, and one quest implemented: the mine quest. Similar to Rodrigo et al., Heroes of Math Island has an agent (the monkey) that uses emotional expressions that respond to situations in the game (Rodrigo, et al., 2012). Game-based learning in a rich interactive environment employing activities on an island was also explored by Lester’s research group using the Crystal Island game (McQuiggan, Robison, & Lester, 2010). The idea of an island (also found in several commercial games, i.e., World of Warcraft) is related to adventure on an enclosed imaginary territory.  The game design started in June 2010 and the game was ready for experiments in March 2012. The game has been designed in close collaboration with Dr. Stephen Petrina and the graduate team of researchers assembled in the How We Learn Lab (Media and Technology Across the Lifespan) (http://blogs.ubc.ca/howwelearn). Components of the game development have been funded through Dr. Petrina’s SSHRC SRGs at The University of British Columbia (UBC) and the VP Research Seed Fund at British Columbia Institute of Technology (BCIT). I also worked closely with and directed a group of BCIT students, in designing and programming this game. The game design and implementation had several prototypes and iterations that were evaluated in presentations and brainstorming sessions with graduate students from UBC and students from BCIT. A graphics artist was hired to create the game graphics and animations.   84  Design was based on several principles of game design: avatars, non-player characters (monkey, queen, king, etc.), content (a narrative accompanying each task), control (players use the keyboard and the mouse to navigate through the game), repeatability (a player will repeat a set of actions for mastering a task), and levels of difficulty (the game’s level of difficulty increases over time) (Salen & Zimmerman, 2004). Several general educational principles employed in classroom practice were considered for game design: defining learning objectives, planning learning activities, assessment and evaluation, and knowledge construction.  Heroes of Math Island was designed with five possible challenges or “quests”: forest, mine, mountain, sea shore and swamp (see Figure 8). Each quest is intended to have a set of activities. For this version of the game only the mine quest was designed and implemented. The mine quest is constructed based on learning outcomes involving prime numbers and factorization based on British Columbia Ministry of Education Guidelines for grades five through seven  (Ministry of Education Curriculum, 2011). Three activities have been designed and implemented: divisibility, prime numbers, and de-composition.   Figure 8. Heroes of Math Island game and scene.    85  3.2.1.1. Learning Outcomes The following learning outcomes and achievement indicators were followed (Ministry of Education Curriculum, 2011):  Demonstrate an understanding of factors and multiples by:  determining multiples and factors of numbers less than 100   identifying prime and composite numbers   identifying multiples for a given number   determining all the whole number factors of a given number   identifying the factors for a given number   sorting a given set of numbers as prime and composite  identifying that 0 and 1, negative and fractional numbers are neither prime nor composite Not included in the Ministry of Education guidelines but included in the game were:  determining multiples and factors of numbers less than 400  using the divisibility rules for 2, 3, 5, and 10   using the square root number method for finding if a number is prime or composite  The mathematics content of the game is used for various reasons, including that I can act as a content specialist. The game and its mathematical content been evaluated in three usability studies involving (i) Alayne Armstrong an elementary mathematics teacher who taught grades six through eight mathematics for 14 years in Greater Vancouver area (two studies), and (ii) two instructional designers: Jennifer Zhao from UBC (eLearning technologist and instructional designer, who have been working in this field for eight years)  86  and Youdan Zhang from BCIT (instructional development consultant with ten years’ experience; specialty e-learning and curriculum development). The game has many aspects learned and inspired from the Prime Climb educational game for number factorization (Conati, 2002; Conati & Maclaren, 2009; Conati & Manske, 2009; Conati, Jaques, & Muir, 2013). There are similarities with respect to mathematical content and learning outcomes (prime numbers), design (both games use hints) and methodology (pre- and posttest and the questionnaire were inspired and adapted from previous Prime Climb studies). I intend to expand the game however, for the moment Heroes of Math Island is non-adaptive compared with Prime Climb that adapts to affect, has a student model and a pedagogical agent based on a probabilistic model.  The game is fully functional and includes all standard game mechanics for play and animation, and a set of characters [avatars (boy/girl), commander, king, queen, wiseman/wisewoman and monkey]. Figure 8 presents scenes from the current implementation of the game. A quest is a symbolic hero’s journey towards a goal that appears in the folklore of many cultures. In this context, quests involve mathematical problem solving. The student is asked to go to the mine and save a disastrous situation. At the beginning of the game, students can choose their avatar. On the island there is a library (the wiseman’s library in this version of the game) where students can get help. For this version of the game, the library existed along with a sample book, but there was no implementation for help, therefore students asked experiments for help.   87  3.2.1.2. Emotional Character Similar to Rodrigo et al. (Rodrigo, et al., 2012), Heroes of Math Island has an agent (the monkey) that uses emotional expressions to respond to the student’s math performance in the game. For this study the monkey displays a neutral state, two positive (happy and confident) and two negative (sad and frustrated) emotional states. Happy and sad were states that indicated a small change in the student’s performance, when frustration or confidence indicated a more persistent situation. The emotional state displayed by the monkey is based on a student success score calculated based on his/her absolute score (number of mistakes – number of correct responses) and the trend of the most recent actions. The monkey has a neutral state and will display a positive or a negative emotion based on the student success score. If the student starts to make mistakes, the monkey displays sadness and if the trend continues, frustration. If the student starts to improve, the monkey goes first into the neutral state, later to happiness and ultimately to the confident state. If the trend changes the transition from positive to negative emotions is always done through neutral (i.e., confidence -> neutral, frustration -> neutral). I wanted the monkey to be encouraging; therefore the monkey is slightly smiling in the neutral state. I started with a simple set of four emotions with the intention to later expand the set of emotions based on results from this study. The monkey character is present continuously on the screen during the math activities (see Figure 8, right image). The emotional states of the monkey are presented in Figure 9.   88   Figure 9. The monkey’s emotions (from the left): frustrated, sad, neutral, happy and confident.  3.2.2. Game Mechanics The following metaphor is used for the mine quest: Miners are sick and need help to finish the mining work. Numbers are represented by rocks: prime numbers are hard rocks that cannot be broken and should be separated. Composite numbers can be broken with picks and transformed into gold. Once in the mine, students solve problems generated by the system with an increase level of difficulty based on student’s performance. A panel on the top of the screen indicates the current level in the activity. For each activity similar problems will repeat for five times, each time with an increase level of difficulty. The game provides progressive hints to help students overcome errors, similar to the Prime Climb educational game for number factorization (Conati, 2002; Conati & Maclaren, 2009; Conati & Manske, 2009). At the first error of a kind, a hint appears acknowledging the error, e.g., “You picked 10, and that is incorrect”; if a similar error is repeated, the next hint suggests what the student should be looking for, e.g., “There are still some rocks with prime numbers that you can pick”; at the third error, an example on how to solve the problem is given. Hints are given based on the type of error, e.g., if the number “1” is selected as a prime number the hint given by the system is “1 is not a prime number; the smallest prime number is 2”. On the  89  fourth error, the student is sent to visit the wiseman’s library. The visit to the library results in re-starting the activity. Experiments indicated that this technique prevented “gaming the system” (guessing the correct answer).  Figure 10 presents a screen shot with a student receiving a hint.    Figure 10. Divisibility activity: A hint is displayed.     90  Activity 1 - Divisibility Divisibility problems involve finding divisors. On the screen appear rocks representing integers. When the student hits a rock, the rock gets into foreground and all other rocks become gray and several “picks” are available on the right side of the screen. In Figure 11, the student selected a rock having the number 40 on it. On the bottom of the screen it is displayed 40 = 5 x ?. If the student selects the pick with 8 on it (the correct answer), the rock transforms into gold.     Figure 11. The divisibility activity.      91  Activity 2 – Prime Numbers Activity 2 is concerned with recognition of prime numbers. The system generates a set of rocks representing positive integers (prime and composite), zero, negative integers, and numbers with decimals (see Figure 12). Prime number rocks, if selected, will be removed. If a non-prime number is selected, it will result in an error and the rock turns gray. When the student believes that all prime numbers were removed, he/she should click a “Done” button.     Figure 12. The prime numbers activity.        92  Activity 3 - Decomposition This activity is concerned with decomposing a composite number into prime factors. For the decomposition activity the system generates only positive integers greater than 2. This activity is similar with the divisibility activity. Similarly, when the student hits a rock, the rock gets into foreground and all other rocks become gray and picks are available on the side (see Figure 13). The student should keep re- decomposing the number till the final factor in order to get access to other rocks. In the end the number 1 is displayed and the rock turns to gold. The decomposition string is displayed, e.g., 12 = 2 * 2 * 3.     Figure 13. The decomposition activity.      93  3.3. Section Two: Study Design This study used the DBR paradigm by designing an educational game to address the research problem and observing participants in interaction with the game. DBR methodologies are suitable for studies of ALTs and ITSs in that they blend “empirical research with the theory-driven design of learning environments” and help in understanding “how, when, and why educational innovations work in practice” (Design-Based Research Collective, 2003; p. 5). Sometimes called design experimentation, DBR “simultaneously pursues the goals of developing effective learning environments and using such environments as natural laboratories to study learning and teaching” (Sandoval & Bell, 2004; p. 200). Sandoval and Bell (2004) noted that on the design side researchers draw from the “fields of computer science, curriculum theory, instructional design, and teacher education” and on the research side researchers draw from “multiple disciplines, including developmental psychology, cognitive science, learning sciences, anthropology, and sociology” (p. 200). DBR has as elements of theory-driven pedagogical intervention and ID implemented and evaluated in real-world situations (Joseph, 2004). Converging experiments with design is an important characteristic of DBR. The study was designed to include two parts: (1) a quasi-experimental study (one-group pretest-postest design) and (2) affect analysis by trained judges. For the purposes of this research, the lack of a control group was acceptable within this specific quasi-experimental design. The quasi-experimental study included: pretest, intervention (gameplay), posttest, post-questionnaire, and interview. Affect analysis involved the process of identifying what emotions should be observed (emotion labeling), and video annotations.  94  In order to get better insights of the emotion analysis, a formal interview was conducted with the judges involved in affect analysis. This study employed triangulation mixed methods design. Quantitative data resulted from affect analysis, pre-and posttests and questionnaires, and qualitative data from interviews of participants and judges involved in the affect analysis process. Triangulation was used for cross-checking data from these sources. 3.3.1. Participants and Recruitment The participants in this study were 15 students (seven boys and eight girls) grades six and seven. The mean age was 11.4 and the median age 11. Participants for research were recruited with recruiting flayers posted on several locations (schools, daycares, and at BCIT and UBC) and during the BCIT Open House 2012 (April 2012). The recruitment flayers were very successful with parents and teachers visiting the BCIT Open House. The recruitment flyers are included in Appendix 1.   3.3.2. Procedure   The following section provides a summary of steps and activities required to be performed in order to carry out the study and reach the research goals. The two parts, (1) quasi-experimental study and (2) affect analysis (emotion labeling process and video annotations), are described and explained. Each component mentioned in the procedure section will be clearly identified and described in details in subsequent sections. It is important to note that part one and two temporarily overlap.  Figure 14 presents the study design:     95     Experiments                                                 Quantitative analysis + Qualitative analysis        Conclusions    Emotion   Video  labeling    annotations   Figure 14. Study design.  3.3.2.1. First Part: Quasi-Experimental Study  Students participated in a quasi-experimental study described by the protocol included in Appendix 2. The quasi-experimental study consisted of: pretest (Appendix 4), intervention, posttest (Appendix 5), post–questionnaire (Appendix 6) followed by a semi-structured interview. The interview guide is included in Appendix 7.  I conducted the experiments together with three BCIT students who were knowledgeable with respect to this study and involved in the design and implementation of the game. We conducted the experiments and observed the participants; I will refer to a person observing the experiments as “observer”. Later on, I will refer to the person who performed affect analysis by video annotations as “judge”.  The protocol used for this part included a short tutorial given by one of the observers, including definitions and examples (see Appendix 3), followed by the experiment: pretest, game play, posttest, post-questionnaire, and interview. Total time for an experiment was 1 ½ to 2 ½ hours and the time used for game play in the mine quest was from 15 to 48 min (M = 32.3 min; SD = 10.3 min). The game interaction was videotaped: one video camera recorded  96  the student’s face and one the computer screen and the two videos were merged and synchronized together.  The tutorial given before the experiment was intended to bring students to the same level of required math knowledge. The pre- and posttests were similar but not identical and contained 23 questions (12 divisibility, 5 prime numbers, and 6 number de-compositions). When students had questions, they asked the observers. Figure 15 presents the primary setting during experiments.  Figure 15. The experiment settings.  The post-questionnaire was adapted from one designed in 2004 by Heather Maclaren, under the supervision of Dr. Cristina Conati and used in Prime Climb studies (Conati & Maclaren, 2009; Conati & Manske, 2009). The adapted post-questionnaire was composed of 48 Likert scale items: statements regarding the general game experience (14), statements regarding the “fun” of the game experience (13), statements regarding learning mathematics  97  in this game (7), statements regarding interest in video and math games and willingness to play the game again (10), reports of mastery of the three topics (divisibility, prime numbers and decomposition) before playing the game (3), and the attitude towards mathematics (1), and 5 open-ended questions. The post- questionnaire is included in Appendix 6. Data extracted from pre- and posttests and questionnaires were statistically analyzed with the SPSS software, graphed and displayed. An inferential statistics pair-difference t-test was used to determine if there was a significant difference between the means of the pre- and posttest scores. A paired t-test is suitable in this situation because the pre- and posttest scores are from the same participants.  Descriptive statistics were used to describe the student’s responses in the post-questionnaire. Correlation analysis was used to identify whatever and to what degree there is a relationship between different variables like interest in mathematics, enjoyment of game play, features of the game, and learning. A detailed description of these variables will be provided in the next chapter. Data collected from post-questionnaires because of the rank format (Likert scale) were analyzed with Spearman’s correlation coefficient which is suitable for rank data: Spearman’s rank-order correlation (rho, also signified by rs) is the non-parametric equivalent of the Pearson correlation and measures the strength of association between two ranked variables. Spearman’s correlation coefficient and subsequent significance testing requires data to be ordinal and monotonically related: these two requirements were verified. According to the statistics literature Spearman’s correlation coefficients values are considered as follows:    98  Table 1. Spearman’s Coefficients Spearman’s Coefficients  Meaning  0.2 – 0.39 Weak 0.4 – 0.59 Moderate 0.6 – 0.79 Strong 0.8 – 1.0 Very strong  A two-tailed test of significance was used: significances less than the 0.01 level (2-tailed) and less than 0.05 (2-tailed), in combination with correlation coefficients above 0.6 were taken into consideration to identify relationships. Post-questionnaire data were analyzed independently and summated to create scores for groups of items classified according to the categories: affective domain, cognitive domain, learning, in contrast to attitude towards the game, the content (mathematics), and the emotional agent of the game (see Table 2).  Table 2. Questionnaire Blueprint  Affective domain Cognitive domain Learning  Attitude towards the game 1.1; 1.10; 2.1; 2.7; 2.11; 2.12; 2.16; 4.5; 4.6; 4.8; 4.10 1.5; 1.11; 4.12 1.6; 1.13; 3.1; 3.9; 4:11 Attitude towards math 1.7; 1.8; 2.8; 2.9; 2.13; 2.14;  1.8; 1.16 1.13; 3.1; 3.2; 3.3; 3.4; 3.6; 3.9; 3.10; 4.11 Attitude towards the monkey 1.3; 1.9; 1.12; 1.15; 1.17; 2.2; 2.3; 2.4; 2.5     99  The design of this questionnaire was intended to provide information needed to respond to the research questions. The rows correspond to the students’ subjective attitude towards the game, mathematics and the monkey (who is the emotional agent in this game). Columns indicate the following domains: affective domain, cognitive domain, and learning. Intersection of rows and columns correspond to the themes of this study. Besides showing emotional reactions, the monkey did not contribute in other ways to the game interaction; therefore only the affective domain refers to him. Some statements were used to characterize more than one category: for example the statement 1.15: “I felt shame when I made mistakes and the monkey was sad” refers to the cognitive domain (mistakes), the affective domain (i.e., shame, sad) and the monkey; and statement 4.11: “I believe that I can learn math better by playing this game” is about learning, and the attitude towards both the game and mathematics. However, for simplicity I analyzed each statement in only one single category, the most appropriate, given priority to the affective domain and learning. Statements 2.8 and 2.9 refer to the experiences students had during pre- and posttest. The post-questionnaire also included statements related to the prior attitude towards mathematics, video games and math games: statements 4.1; 4.2; 4.3; 4.4 and 4.13 (not included in the blueprint). The statements related to reports of mastery of the three topics (divisibility, prime numbers and decomposition) before playing the game, are also not included in the blueprint.  After students completed the post-questionnaire, I conducted interviews together with the other observers. The interview guide (see Appendix 7) included a set of questions related to the game experience, affective states, perception of mathematics cognitive gains, and willingness to play again. Interviews were semi-structured for allowing for free discussion:  100  questions were not restricted to the interview guide. Issues noted during game interaction and reported in post-questionnaire were clarified in interviews. After interviews, observers had a discussion and collaboratively wrote a report based on notes taken during experiments. Data collected from interviews were used to better describe, clarify and understand the findings resulting from the quantitative analysis. A coding system for emotion labeling was developed in parallel with experiments. A separate document was created in a checklist matrix format for recording and describing the results (see Appendix 10). A matrix based on the questionnaire blueprint was used to assemble descriptive data from the statistics resulted in the quantitative analysis. The matrix was analyzed with a quasi-statistic method for estimating the frequency of each theme, and with logical analysis for outlining causation and identifying the logical process. 3.3.2.2. Second Part: Affect Analysis The second part had two stages (1) the emotion labeling process and (2) video annotations by trained judges.  3.3.2.2.1. Stage One: Emotion Labeling Process An important research objective of this study is the identification of what emotions were expressed by students within game play. Selecting what emotions to observe, and designing the protocol and instrument for affect analysis and video annotations required a long, involved process. The process of emotion labeling used the DBR paradigm, by combining exploration with design (Design-Based Research Collective, 2003). Exploration was conducted in parallel with design and targeted the identification of a set of emotions that are relevant to game interaction and learning. This part was needed because existing affective frameworks, though each had emotions that were relevant to this study, none of them was  101  complete or fully adequate. I started from the idea that emotions that are relevant to learning and to the game play should be taken into consideration. Emotions should not be separated from their context, and judges should see what the student was doing, therefore video software was used to process and synchronize both the student’s face and the game screen.  The process of labeling emotions took several iterations between April and mid October 2012, happened in parallel with experiments, and consisted of the following phases:  Phase 1: Derived a set of emotions that can be relevant to learning in the context of game play by looking at existing literature in education, emotion theory and affective computing  Phase 2: Ran a pilot during which I investigated which emotions happened during game play and trained judges in observing emotions  Phase 3: Ran a pilot to evaluate whether students should be used as judges of their own emotion, in addition to trained judges.   Phase 4: Ran a pilot during which I evaluated multiple emotion-reporting at 20-second time interval, revised the emotion set, developed an observer’s guide, and established a protocol and an instrument for emotion reporting.  Phase 1    To define an initial set of emotions, I looked at the following emotion models: (i) the affect framework proposed by D’Mello and Graesser (D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006), which considered the following emotions: boredom, confusion, delight, engaged concentration (also known as flow), frustration, and surprise; (ii) models found in the education literature: Astleitner’s (2000) FEASP model of emotions in the  102  context of instruction (including anger, envy, fear, pleasure, and sympathy); Ingleton’s emotion model in learning: confidence, distance, fear, pride, shame and solidarity (Ingleton, 2000); and (iii) the OCC cognitive theory of emotions (Ortony, Clore, & Collins, 1988). All these models had merits and were relevant to this study: D’Mello and Graesser’ s model was used in several studies of ITSs, Astleitner’s model is well known in the educational literature, Ingleton’ s model, also from education, was designed with respect to mathematics, and the OCC model was used in studies of ITSs and games, and in related research (Conati, 2002; Conati & Maclaren, 2009; Trabelsi & Frasson, 2010). From each model, I selected the emotions that would be relevant for this study and that could form a reasonable set to pilot test. From D’Mello and Graesser’s framework (D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006) I selected boredom, confusion, delight, engaged concentration and frustration. Note that delight is equivalent to what Astleitner (2000) and OCC (Ortony, Clore, & Collins, 1988) referred to as pleasure, so this emotion was referred with both names. Surprise was not included in the first set of emotions because I wanted to focus on emotions only, and surprise is not considered an emotion by important emotion theorists (Lazarus, 1984; Ortony & Turner, 1990). Ortony and Turner considered that being “affectively valenced is a necessary condition for a state to be an emotion” and argued that surprise can be better qualified as a cognitive than an affective state (p. 317). From Ingleton’s (2000) model, I selected confidence, pride, and shame. Ingleton argued that confidence creates a “disposition to learn” opposed to negative emotions such as anxiety, grief and dejection, which can prevent learning, leading to inactivity and isolation (p.88). Pride and shame are playing a role in identity and self-esteem which influence the formation of confidence (Ingleton, 2000). Pride and shame are also part of the OCC model and were  103  explored in studies with the Prime Climb as emotions that occurred because of the student’s actions (Conati, 2002; Conati & Maclaren, 2009; Conati & Manske, 2009). Furthermore, they were especially relevant in this game because the monkey is showing emotions that are explicit reactions to the student’s game performance. From Ingleton’s model I excluded anger, distance, fear and solidarity. Anger (also found in the OCC and Astleitner’s models) was excluded because I felt that the already included negative emotion of frustration is more appropriate for this study. Frustration is the negative emotion most encountered when interacting with computers (Klein, Moon, & Picard, 2002). Fear (also present in the OCC and Astleitner’s models) was excluded because it is described as “fear of failure” in the context of learning (Astleitner, 2000; p. 212) and the study participants were in a relaxed environment without any school pressure. Fear is an emotion that depends less on cognitive activities: the reaction of fear to an immediate dangerous situation is processed in the amygdala and happens very fast, opposed to a similar emotion, anxiety, which involves symbolic representations, is anticipatory, and can occur without a real threat (Lazarus, 1982). I argue that fear in Astleitner (2000) and Ingleton (2000) is in fact anxiety and contemplated adding anxiety to the emotion set [also found in McQuiggan, Robison, and Lester (2008)], but did not do it at the beginning, waiting to see how students react. Distance and solidarity from Ingleton’s model (2000) were excluded because I judged them to be more suited to classroom instruction. From Astleitner’s model (2000) I included pleasure; the remaining emotions of envy and sympathy were considered more suited for classroom instruction. The OCC model (Ortony, Clore, & Collins, 1988) contains 22 emotions in three categories: emotions resulting from consequences of events, actions of agents, and aspects of objects. I did not consider reactions to aspects of objects because I felt that there are no objects in the  104  math game that can trigger substantial emotions. Regarding the emotions resulting from consequence of events to self, I assumed that, for this initial round, the already included positive emotion of delight and the negative emotion of frustration could cover the OCC emotions of joy, pleasure, satisfaction, displeasure, disappointment and distress. With respect to emotions for agents other than self, (e.g., admiration, gratitude, remorse, and reproach), I decided to exclude them at this time because the only agent in the game is the monkey and it does not perform any game actions that can directly help or hinder the student’s game performance. The monkey’s affective displays may trigger emotions such as reproach, but this did not seem to be the case when I observed participants in the experimental component of the study. The model included neutral to give judges a way to report no emotion, as opposed to a situation of invalid interval or missing data. This process resulted in eight emotions (plus neutral) to be pilot tested in the second phase: boredom, confidence, confusion, delight/pleasure, engaged concentration, frustration, pride, and shame. An initial instrument for emotion reporting was designed with vertical rubrics for each emotion in the set above, plus an entry for observed emotions not listed in the set. As part of the protocol for emotion judgment, I adapted from Baker, D'Mello, Rodrigo and Graesser (2010) an observer’s guide that had a short description for each emotion (see Appendix 8). The labelling system and the instrument’s format were evaluated in several sessions during which observers met and watched videos of pilot game segments on a large display and observed, judged and refined the set of emotions and the process that should be implemented for video annotations.     105  Phase 2    Emotion labelling in similar studies (Baker, D'Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Rodrigo, et al., 2008; Rodrigo, et al., 2012) involved trained judges assessing emotions occurring during 20-second intervals. During this phase, I explored the process and the length of the interval of observing emotions. Observers reported emotions during 3-minute intervals from portions of the videos that will not be later used for video annotations. There were instances reported for all emotions from the given list, although observers more frequently reported engaged concentration, confidence and confusion. However, there were also reports for emotions not included in the original set: curiosity, surprise, and tentativeness (also described by observers as hesitancy). Therefore, the set of emotions was extended to include these three. Surprise was added because, even if it is not consistently considered an emotion, it was observed in pilot studies and also included in many important previous emotion studies (Baker, D'Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor, & Graesser, 2007; Ekman, 1999; Graesser, et al., 2006; Rodrigo, et al., 2008; Rodrigo, et al., 2012). The outcome of this phase was a revised set of 11 emotions (plus neutral): boredom, confidence, confusion, curiosity, delight/pleasure, engaged concentration, frustration, hesitancy, pride, shame, and surprise. This pilot phase also confirmed that, as expected, too many emotions happen in a 3-minute interval. Therefore, for the subsequent phases, a 20-second interval was adopted. It should be noted, however, that in previous work judges were asked to report only one emotion per interval, even when more than one was observed, except Graesser et al. (2006) in which all emotions were marked but only the most pronounced was indicated. In contrast, I decided to  106  allow judges to indicate all observed emotions, to have a better sense of whether 20-second intervals represent an adequate granularity for this process.  Phase 3    An emotion judging session was employed with “Student 1” (the first participant in this study) as a judge. She observed herself for 16 min 40 sec minutes with a granularity of 20 seconds (50 intervals) and reported emotions seen during each interval. The student reported all emotions except surprise offering confidence that the emotion set was adequate: shame (3), boredom (3), frustration (12), confusion (15), tentative (1), surprise (0), curiosity (6), engaged concentration (15), delight/pleasure (7), confidence (6), pride (4), and neutral (5). She did not have difficulties reporting emotions, but it took her 45 minutes and in the end she indicated that the whole process was a tedious job. Student 1 was a more mature student; I considered that it would be even more difficult for the younger students in the group to judge their emotions. Previous studies by Graesser et al. (2006) found that trained judges have an inter-judge reliability of Cohen’s Kappa = 0.71 offering confidence in this method. The outcome of this phase was the decision to use trained judges for video emotion analysis as in several other previous studies (Baker, D'Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Rodrigo, et al., 2008; Rodrigo, et al., 2012). Phase 4  Phase 4 was dedicated to process’ and method’s validation. To increase the observers’ confidence with the emotion labeling process, I conducted more group observations and brain-storming sessions. Observers processed together three 8-minute segments videos from different students (that will not be used for video annotations). They reported all affective states that were noted during the 20-second interval, and revised the observer`s guide  107  developed in phase 1 and improved in the sequential phases. After the video was observed once, they observed it again from the beginning with the intention of correction. Based on feedback from the observers, the set of emotions was revised again by adding rubrics for (i) a negative emotion of disappointment or displeasure (also found in the OCC model); (ii) excitement (also found in emotion studies with the Cristal Island game (McQuiggan, Robison, & Lester, 2010), to allow observers to report separately situations similar to delight/pleasure but of a higher intensity. Confusion and hesitancy were merged together because we found difficult to differentiate between the two. The final set included several emotions that are described with two names for joining emotions hard to discriminate (i.e., confusion and hesitancy) and for avoiding a too-fine of a granularity. The final choice of affective states included 12 emotions (plus neutral): boredom, confidence, confusion/hesitancy, curiosity, delight/pleasure, disappointment/displeasure, engaged concentration, excitement, frustration, pride, shame and surprise. The observer guide adapted from Baker, D'Mello, Rodrigo and Graesser (2010) in phase 1 (see Appendix 8) was improved and included as part of the protocol for observing affective states. The protocol used for observing affective states including an updated observer’s guide,  is found in Appendix 9. The form used as an instrument for video emotion analysis is found in Appendix 10.  During phases 3 and 4, I attempted recording the intensity of emotions on three levels of intensity: low, medium and high. Student 1 and the observers involved in the process of emotion labeling fond the process confusing and had difficulties to discriminate between the three levels of intensity. Therefore, I decided to not record the intensity of emotions in this study.   108  3.3.2.2.2. Stage Two: Video Annotations by Trained Judges  The idea of affect analysis by trained judges using video annotations came from literature (Baker, D'Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Rodrigo, et al., 2012). I actually had a conversation with Dr. D’Mello who explained how to train judges: he suggested that a “trained judge” should be very familiar with the research, understand the goals of the study, and understand how to label emotions. He suggested training judges on some 5 min of recording coding at a lower granularity level (20 sec). Based on literature review, his recommendations, and the particularities of this study, I successfully trained judges and implemented the protocol for observing affective states and the instrument for emotion analysis. During the first part (see Section 3.3.2.1) participants were videotaped, but no affect judgment was employed. However, general observations regarding the game interaction (including affect) were reported by observers. Participants also reported emotional states in the post- questionnaire and in interviews. The two videos (face and screen) were synchronized based on the audio recording, merged together in one video that included on one half the video of the student and on the other half the video of the screen, and trimmed to the game interaction starting from the beginning of the mine quest till the end of the quest or the end of the game interaction for students who decided to stop earlier. In order to proceed with video annotations, the three observers and I went through several sessions of training in observing and understanding emotional states (as described in Section 3.3.2.2.1).  One of the observers was not available for judging. Two judges (one of the observers and I) performed video analysis on videos from the 15 students. Videos were observed by the two judges individually, in a quiet environment. Judges made judgments every 20 seconds interval reporting all affective states that were observed during the 20-second interval  109  following the document attached in Appendix 9. A form was used for judgments with columns for all affective states and rows for each time interval (see Appendix 10). Judges were allowed to re-play the video. Judges did not discuss their observations or results. A third judge (another observer whom I will call him Judge 3) who participated in training did not complete video analysis; he declined to continue because he found the task too challenging. Both judges, including the one who declined to judge participated in a formal interview. A detailed discussion will follow in Chapter Four. The interview was semi-structured with questions grouped on the following categories:  Process: clarity, merit, challenge  Emotions: what was observed, challenge  Participants: particularities, challenge The interview guide for judges is included in Appendix 11.  Descriptive statistics were used to describe the results of affect analysis. A Cohen’s Kappa analysis was used for inter-judge reliability.  The results from the first and second parts were combined into a casual network based on the attitudes and domains specified in Table 2. The casual network employing cross-site procedures was used for the final data analysis, and provided interpretative and explanatory results (Miles & Huberman , 1984).  Final conclusions were drawn by counting, noting patterns and themes, clustering data into relevant groupings, and identifying relations between variables. For internal validation, triangulation between data collected from affect analysis, pre- and posttests, questionnaires and interviews was used.   110  3.4. Conclusion This chapter presented the methodology used in this study including detailed descriptions of the game and procedure. The procedure included two parts: the quasi-experimental study and affect analysis. The affect analysis employed two stages: the emotion labeling process and video annotations by trained judges. Descriptions of the participants and the quantitative and qualitative techniques were also provided.  Chapter Four addresses data analysis and findings.   111  Chapter Four: Data Analysis   4.1. Introduction The previous chapter described the research methodology employed for this study. This chapter focuses on data analysis and interpretation. By combining exploration with design, this study used a DBR paradigm (Design-Based Research Collective, 2003). With respect to methods, this study used triangulation mixed methods design: quantitative and qualitative data are collected concurrently throughout the same study (Gay, Mills, & Airasian, 2006). Three stages of data analysis were employed in the data analysis process of this study: affect analysis (quantitative approach), descriptive and causal-comparative analysis (quantitative and qualitative approach), and triangulation. Data analysis included descriptive statistics, t-test, and Cohen’s Kappa analysis (using the SPSS software) categorization, content analysis, and matrix analysis.  4.2. Demographics Gender and Age Distribution The participants in this study were 15 students (seven boys and eight girls) grades six and seven. The mean age was 11.4 and the median age 11. The participants in this study are named based on their IDs: Student 1, Student 2, etc.     112  4.3. Affective States Analysis This section focuses on the main research question of this study: What affective states are important with respect to a student’s interaction with an educational game? In order to answer this question the following question is first answered: What affective states are elicited during the Heroes of Math Island gameplay? In the Methodology chapter I described the process employed for emotion labeling. The emotions analyzed were observed on videos by trained judges during the students’ interaction with Heroes of Math Island. This section presents the detailed analysis of the emotions experienced by students playing the game. Twelve emotions (plus neutral): boredom, confidence, confusion/hesitancy, curiosity, delight/pleasure, disappointment/displeasure, engaged concentration, excitement, frustration, pride, shame and surprise were observed in videos of 15 study participants and were classified by two judges from the original team of observes. I refer to them as Judge 1 and Judge 2. I was Judge 1 and Judge 2 was one BCIT graduate student (she also had a Bachelor of Science degree from the University of British Columbia). The criteria used to select these emotions and the process employed to train judges were described in details in Chapter Three (see Section 3.3.2.2.1). Overall, 1082 data intervals of 20 seconds were analyzed for the 15 students. All emotions in the emotion set were present in the game interaction. Judges reported in each 20 second time interval as many emotions they observed. There was no limit on how many emotions would be reported; however, no more than 4 emotions were reported per interval. Time intervals when the study participants talked to observers or no emotion was reported (e.g., there were situations when the camera tilted or the face of the participant was not  113  visible) were removed from the data set. Figure 16 presents the raw agreement between the two judges.  Figure 16. Judges report on emotions: raw agreement. The least frequent emotions/states observed (below 1.2%) were: curiosity, excitement, neutral, pride and shame. Boredom, negatively associated to learning (D’Mello, Taylor, & Graesser, 2007) was also rarely reported (1.7% by Judge 1 and 1.4% by Judge 2), followed by surprise (3.1% by Judge 1 and 2.3% by Judge 2), delight/pleasure (5% by Judge 1 and 3.8% by Judge 2) and then by confusion/hesitation (26% by both judges). Engaged concentration was the state where students spent the majority of the time (83.9% reported by Judge 1 and 74.4% by Judge 2).  While for the emotions listed so far, frequencies of reports from the two judges are quite similar, there are noticeable differences with respect to (i) confidence, for which Judge 1 reported 24.9% and Judge 2 reported 49.5%, (ii) displeasure/disappointment (Judge 1 reported 12% compared to Judge 2 who reported only 0.3%), and (iii) frustration reported  114  8.1% by Judge 1 and 1.6% by Judge 2. Both judges followed carefully the protocol and were very conscientious; however I am an experienced educator whereas Judge 2 was a graduate student without experience in teaching. I believe this may be the reason for Judge’s 2 higher tendencies to interpret students’ behaviors in terms of positive affect (more discussion on this point will follow in a subsequent section). In the next subsections, I provide a more detailed analysis of inter-rater agreement, at different level of granularities. 4.3.1. Agreement over One Emotion per Interval In this analysis, I discuss the level of agreement between the two judges if only one emotion is selected per interval, even when several emotions were reported, to mimic the assumption made in previous emotion studies relying on the 20-second interval approach. I report Cohen`s Kappa scores for each student and for the aggregated data over all 15 students. When several emotions were reported per data point, only one emotion was taken into consideration to build the confusion matrix for agreement/disagreement, following this selection criterion:   If there was agreement on one emotion only, that emotion was selected.  If there was agreement over more than one emotion, one of the agreed-upon emotions was selected, choosing one that seemed more prominent if possible [similar to Graesser, et al. (2006)] otherwise at random, unless one of them was engaged concentration. In that case, engaged concentration was excluded from the selection, because engaged concentration was observed more than other states.   115   If there was no agreement over the emotions in the interval, the pair most likely to be mixed up was selected (e.g., if Judge 1 indicated confusion and curiosity and Judge 2 indicated frustration and boredom, we have chosen confusion and frustration because they are more likely to be mixed up). Note that in most previous studies, judges reported the first emotion observed in an interval. I could not use observation time as selection criterion, because I had no information on the sequencing of emotions within each interval. The aggregated Cohen’s Kappa was 0.676. Although there is no unified criterion to interpret Cohen’s Kappa values in the literature, values in the 0.6-0.7 range are generally considered good (Fleiss, 1981) or even substantial (Landis & Koch , 1977). The best values of Cohen’s Kappa achieved in previous studies were: 0.63 (Baker, D'Mello, Rodrigo, & Graesser, 2010), 0.68 (Rodrigo, et al., 2012), 0.71 (Graesser, et al., 2006), and 0.73 (Rodrigo, et al., 2008).   Figure 17. Cohen’s Kappa for individual students.   116  In addition to the aggregated Cohen’s Kappa, I also computed Cohen’s Kappa scores for each individual student (see Figure 17), to ascertain the impact of individual differences in the observed subjects on the reliability of emotion labeling. As the figure shows, the inter-judge reliability was quite varied for some students, although it was substantial on average (M = 0.688, SD = 0.19). Five students had Cohen’s Kappa values at or above 0.8 (considered excellent), whereas four had values generally considered low (Student 8 with 0.297 and Student 12 with 0.388), or moderate (Student 11 with 0.504 and Student 6 with 0.526).  Table 3 shows the confusion matrix on the aggregated data, to give a sense of which emotions where harder to discriminate. Reports from Judge 1 are in the rows, those for Judge 2 in the columns. Note that excitement is not in the matrix, because it always appears in intervals with other emotions, with no agreement, and was never picked by the selection criterion (Judge 1 reported it 7 times, and Judge 2 zero times).  Table 3. Confusion Matrix: Aggregated Data  Judge 2Bo Sh Fr CH Dis Su Ne Cu Eng PD Co Pr TotBo 1 0 0 2 0 0 0 0 2 0 2 0 7Sh 0 0 0 0 0 0 0 0 0 0 0 0 0Fr 0 0 9 6 0 0 0 0 10 0 0 0 25CH 1 1 0 116 0 1 1 0 34 0 19 0 171Judge 1 Dis 3 0 0 4 1 0 0 0 7 0 3 0 18Su 0 0 0 0 0 8 0 1 0 0 1 0 9Ne 0 1 0 1 0 0 1 0 1 0 2 0 6Cu 0 0 0 0 0 0 0 2 1 0 1 0 4Eng 0 0 0 7 0 1 1 0 530 2 100 0 641PD 0 0 0 0 0 0 0 0 1 23 1 0 24Co 0 0 0 0 0 0 0 0 0 0 175 0 175Pr 0 0 0 0 0 0 0 0 0 0 1 1 3Tot 5 2 9 136 1 10 2 2 586 25 304 1 1082 117  A confusion matrix is a table that indicates on the diagonal the instances when both judges agreed in the same interval with respect to a specific emotion. For example, confusion/hesitation (CH) was agreed in 116 intervals; however when Judge 1 reported confusion/hesitation, Judge 2 reported boredom (1), shame (1), surprise (1), neutral (1), engaged concentration (34) and confidence (19). When Judge 2 reported confusion/hesitation, Judge 1 reported boredom (2), frustration (6), disappointment/displeasure (4), neutral (1), and engaged concentration (7).    As discussed earlier, Judge 2 generated many more reports of confidence than Judge 1. The highest source of disagreement (100 instances) is between Judge 2 reporting confidence and Judge 1 reporting engaged concentration, a rather intuitive outcome because both states involve a positive attitude towards and an active involvement with the task at hand. Engaged concentration is described as immersion, focus, and concentration on the system, with the appearance of the positive engagement. Confidence is described as knowing what to do and solving problems fast: behaviors include solving problems quickly, without hesitation, statements such as “I know this” or “this is easy”; searching for help when needing. The body language of confidence is sitting tall, without slouching or fidgeting. A confident participant is asking questions about the subject matter knowing what to ask. The confusion between the two affective states is not surprising; in fact confidence implies engaged concentration. When Judge 2 indicated confusion/hesitation Judge 1 reported boredom (2), frustration (6), disappointment/displeasure (4), and only 1 instance of neutral and 7 instances of engaged concentration. Confusing confusion/hesitation with boredom, frustration, or disappointment/displeasure is not unusual. More surprising are the situations  118  in which Judge 2 reported a positive valenced emotion and Judge 1 reported a negative valenced one: 19 instances in which Judge 2 reported confidence and Judge 1 reported confusion/hesitation, 34 instances in which Judge 2 reported engaged concentration and Judge 1 reported confusion/hesitation and 10 instances in which Judge 2 reported engaged concentration and Judge 1 reported frustration.    I interpreted these differences as due to Judge 1’s experience as an educator, resulting in a higher ability to detect affective signs of students having difficulties during learning (Picard, 2000). However, this result should be further explored in future studies. 4.3.2. Agreement over Multiple Emotions per Interval   This section is dedicated to analysis of number of emotions reported per 20-second interval. Figure 18 summarizes the frequency with which different numbers of emotions were reported per interval, showing that intervals with two emotions are almost as frequent as interval with one emotion. There is a non-negligible number of intervals with 3 emotions, and even a small percentage of intervals with four emotions.   Figure 18. Emotions reported per interval.    119  I also analyzed in a confusion matrix to what extent the number of emotions reported by the two judges in the same interval was the same. Table 4 shows the confusion matrix indicating how many times judges indicated 1, 2, 3, or 4 emotions for the same interval. The Cohen’s Kappa is negative (K = -0.011) indicating less than chance agreement between judges on this point.  Table 4. Confusion Matrix: Number of Emotions per Interval  1 2 3 4 Total 1 241 233 21 1 496 2 238 197 29 2 466 3 38 58 9 0 105 4 3 3 6 0 15 Total 520 494 65 3 1082  I analyzed how often judges agreed on emotion type for 2, 3 4 emotions, regardless of how many they disagreed upon. There were 732 intervals (67.7%) in which judges agreed on one emotion, 131 (12.1%) intervals in which judges agreed on the type of 2 emotions, 12 intervals (1.1 %) with agreement over 3, and no interval with agreement over 4. In 207 cases (19.1%) there was no match (see Figure 19).   120   Figure 19. Number of emotions that matched per interval.  I looked at how the two judges agreed with respect to each individual student. Looking at matches over individual students (see Figure 20), shows a very high variance on raw agreement (M = 67.9%, SD = 8.0% for one match; M = 12.8%, SD = 8.5 for 2 matches; M = 1.2%, SD = 1.9 for 3), once again indicating that subject differences can affect accuracy of emotion labeling. However, the variance is higher in the case of 2 and 3 emotions. There was no match for 4 emotions.  121   Figure 20. Agreement for 2 or 3 emotions per interval for individual students.    Finally, I looked at the instances when judges matched in both number and type of emotion (Figure 21). In 225 cases (20.8%), there was a perfect match, with a higher occurrence in the case of one emotion (142 intervals, or 13.1%) followed by two emotions (77 intervals, or 7.1%) and three (6 intervals or 0.6 %).   Figure 21. Percentage of total instances when judges agreed on emotion type and number for 1, 2, 3, or 4 emotions.  122   There was again a high variance between students (see Figure 22). The highest numbers of absolute matches were for Student 2 (38%), Student 1 (33.3%) and Student 9 (31.3%); the least are for Student 8 (2.2%), Student 12 (6.7%) and Student 11 (10.5%).   Figure 22. Absolute agreement for 1, 2, or 3 emotions for individual students.    The above results indicate that there was a high variance of reporting agreement over different students. Although it is common knowledge that different people have different propensity for showing their emotions, these results indicate that these differences can be quite substantial, and may make it difficult to obtain reliable ground truth emotion information, and subsequent accurate affective models, for some individuals.  4.4. Descriptive and Causal-Comparative Analysis The following section is concerned with the analysis of data resulted from pre- and posttests, post-questionnaires, and interviews. As mentioned in Chapter Three, I used inferential statistics to determine if there is a significant difference between the means of the pre- and posttest scores, and I used correlations to determine whatever and to what degree a relationship exists between quantifiable variables resulted from the pre- and posttests and  123  post-questionnaires. Pearson’s correlation coefficient was used for linear numerical data (scores of pre- and posttests) and Spearman’s correlation coefficient was used for rank numerical data resulted from post-questionnaires. Results from the quantitative analysis were augmented by the students’ responses resulted from interviews. Data collected from post-questionnaire (see Appendix 6) and interviews, was analyzed and interpreted. For analysis, I used a matrix based on the blueprint described in Chapter Three (see Table 2), including the following categories: affective domain, cognitive domain, and learning with respect to the game features (e.g., game elements, monkey, etc.,) and content (i.e., mathematics). I eliminated from the original blueprint statements that were not responded by all students: 2.13, 2.14 and 4.3. I made an exception for statements 4.6 and 4.8 which, even if responded by only 14 students provided interesting data for this study.  4.4.1. Attitude Before Play Besides the usual demographic questions, I wanted to know the students’ attitude towards the game content (i.e., mathematics) and what their previous experience with video and mathematics games was. The following statements related to the attitude prior playing the game were included in the in post-questionnaire (Table 5): Table 5. Attitude Before Play Statement ID Statement 4.1 I played before video games. 4.2 I like playing video games. 4.3 I played before math games. 4.4 I like playing math games. 4.13 I like math.   124  Statement 4.3 (“I played before math games”) was not given a response by all participants and therefore I eliminated it.  As expected, the majority of students in the group played video games 93 % (14). Only one student (Student 21) reported that she did not play video games. The majority of students agreed or strongly agreed 73.3% (11) with the statement: “I like playing video games.” Only three students were neutral (20%) and one disagreed (Student 21). With respect to the statement “I like playing math games”, not as many students strongly agreed, however the agreements (agree or strongly agree) were also high: 73.3% (11). For this statement, two students were neutral (13.3%), one disagreed (6.7%) and one strongly disagreed (6.7%). Student 13 strongly disagreed, and Student 8 disagreed. Figure 23 presents the bar chart:  Figure 23. Attitude before play: I like video / math games.  From the 15 students, 13.3% (2) students reported that strongly disagreed with the statement “I like math”, 26.7% (4) students were neutral, and the rest agreed 20% (3) or strongly agreed 40% (6). Figure 24 presents the bar chart:  0123456789101 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI like playing video gamesI like playing math games 125   Figure 24. Attitude before play: I like math.   Generally, students in the study demonstrated good attitude and interest. However, they were more interested in mathematics and math games than the average student population. I do not have statistics of how many students in the general population like math games, but I argue that numbers are higher for this group. This is justified by the fact that participating voluntarily in a study involving a math game generally implies a good attitude towards mathematics. The students who reported that they did not like mathematics were Student 10 and Student 13. Interestingly to notice, both students were in fact strong in mathematics and demonstrated focus and good skills. Further analysis will explore in more details their attitude towards the game and the subject matter.  4.4.2. Affective Domain The following sub-section is intended to interpret data from post-questionnaire and interviews with respect to the question: What are the students’ subjective reactions with respect to Heroes of Math Island and to the underline mathematical content? 0123456781 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI like math 126  Besides specific statements, the questionnaire also included an open-ended question with respect to emotions: “Please tell us what emotions you experienced when you played the game”. Table 6 presents the students’ responses. Table 6. Affective Domain: Students’ Responses Name Response Student 1 Interesting, effective Student 2 Happiness, sense of accomplishment, frustration Student 4 Happy (because of the monkey); Annoyed (monkey blinking)  Student 5   Student 6 It was fun Student 7 Happy, frustration Student 8 I had fun, but when I kept getting sent back to the beginning I got frustrated Student 9 Interest; a little bit frustrated sometimes Student 10 Happy, thoughtful, tired Student 11 Happy, confident, interested, frustrated, mixed-up, confused Student 12 Happy Student 13 I liked the part at the castle and I was frustrated when I answered incorrectly Student 15 Proudness, frustration, fun Student 18 I did not see the monkey react, so I put neutral Student 21 It was fun, sometimes a bit frustrated, but it was a good game  It is interesting to observe that the most reported state was happiness or a related state of “fun” and the second one was frustration, but reported to a lesser extent. Other positive emotions were “interesting” which I equate to curiosity, sense of accomplishment, and pride. Results from the interview analysis were consistent with these reports; details will follow in the next sections were post-questionnaire data and interview responses are analyzed together. Table 7 contains the statements related to the affective domain (the other two columns are cleared for clarity: they will be discussed in subsequent sections). Please note that statement 1.8 which belongs to both affective and cognitive domains will be discussed in the context of the cognitive domain (see section 4.4.3.2).  127   Table 7. Affective Domain: Questionnaire Blueprint  Affective domain Cognitive domain Learning  Attitude towards the game 1.1; 1.10; 2.1; 2.7; 2.11; 2.12; 2.16; 4.5; 4.6; 4.8; 4.10   Attitude towards math 1.7; 2.8; 2.9   Attitude towards the monkey 1.3; 1.9; 1.12; 1.15; 1.17; 2.2; 2.3; 2.4; 2.5    4.4.2.1. Affective Domain: Attitude Towards the Game  The following table contains the statements related to the attitude towards the game:  Table 8. Affective Domain: Attitude Towards the Game Statement ID Statement 1.1 I wanted to have fun when I was playing the game. 1.10 I didn’t care whether I had fun or not. 2.1 I had fun when I chose my avatar. 2.7 I had fun when I finished the quest. 2.11 I had fun when the rocks turned into gold. 2.12 I had fun when prime number rocks were removed. 2.16 I had fun when I visited the castle. 4.5  This game is boring. 4.6  This game was more interesting than video games that I played. 4.8 This game was more interesting than math games that I played. 4.10  I would like to play the game again.  Statements 1.1 and 1.10 were opposite to each other. Generally, students responded that they wanted to have fun when they played the game: 66.7% (10) strongly agreed and  128  13.3% (2) agreed), nobody disagreed and only three (20%) were neutral. The opposite statement (“I didn’t care whether I had fun or not”) was responded in a complementary mode indicating that students were consistent in their responses. Figure 25 describes the results:  Figure 25. I wanted to have fun.  4.4.2.1.1. Game Mechanics Statements 2.1, 2.7, 2.11, 2.12 and 2.16 were related to different aspects of the game: avatar, quest, castle, and game metaphors (turning rocks related to correct answers into gold or prime number rocks being removed). As expected for this age group, more student participants reported having fun when choosing the avatar, finishing the quest and visiting the castle and only a minority of students were neutral. Only one student (Student 9) strongly disagreed that she had fun when she finished the quest. However, during the interview, she reported that she was delighted when she finished the quest (it might be possible that she misunderstood the question):    0246810121 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI wanted to have fun when Iwas playing the gameI didn’t care whether I had fun or not  129  Interviewer: Were you delighted? Student 9: Yeah Interviewer: When? Student 9: When I finished it, cause I‘ve gotten through. Even if a minority of students disagreed (nobody strongly disagreed) of having fun when the rocks turned into gold (6.7% or one student) and when prime number rocks were removed (13.3% or two students), higher percentages were neutral (33.3% or five students and respectively 46.7% or seven students). Still, 60% (9) in the first case and 40% (6) agreed or strongly agreed that they had fun. Figure 26 presents the bar chart.  Figure 26. I had fun.  In order to evaluate the students’ reaction to game mechanics, I computed a summative frequencies chart with results from the above discussed five statements (2.1, 2.7, 2.11, 2.12 and 2.16). The analysis concludes that only a small percentage of responses were not favorable: 68% of responses from 15 students to the five statements indicated that 024681012I had funwhen I chosemy avatarI had funwhen Ifinished thequestI had funwhen therocks turnedinto goldI had funwhen primenumber rockswereremovedI had funwhen Ivisited thecastle1 Strongly Disagree2 Disagree3 Neutral4 Agree5 Strongly Agree 130  students agreed or strongly agreed (18.7% strongly agreed and 49.3% agreed); 26.7% were neutral and only 5.3% disagreed or strongly disagreed (4% disagreed and 1.3% strongly disagreed). These results are optimistic, indicating that the game mechanics currently implemented in the game are adequate and attractive for this age group (see Figure 27).  Figure 27. Summative game mechanics.   4.4.2.1.2. Gender Issues Even if the above results are optimistic, interviews revealed that there are gender issues. According to Schell (2008) five things differentiate a male/female player of a video game: for males it is important to have mastery, competition, destruction, spatial puzzle, and trial and error; and for females: emotion, real world, nurturing, dialog and verbal puzzles, and learning by example. However these are only tendencies; we observed very competitive female students (e. g., Students 1 and 9) and male students who did not use trial and error, for example the very cautious Students 6 and 18. 05101520253035401 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeSumative game mechanics 131  The current implementation of the game was aimed to satisfy both genders: the game had mastery, destruction (breaking the rocks), emotion (the monkey), real world (the castle, the mine), and nurturing (helping the miners). The intention of this game was to be as gender neutral as possible; however, the story, the metaphors, and the symbols currently implemented in the game are oriented toward boys or a masculine audience: mining, breaking rocks, using picks, and getting the quest from a king. Observers noticed the gender aspect during the experiment with Student 4 who brought the issue in interview and offered design suggestions: “I think that we should be able to buy new clothes for the avatar and maybe a staff for the monkey, too. When we get enough money we can unlock the monkey store”. Some other female students offered ideas: Student 13 suggested instead of rock composite number to have flower buckets and the action happening in a garden. Male students also observed the gender imbalance: Student 2 pointed out the male-oriented narrative of the mine quest. However, not all female students disliked the mine quest: for example Students 1, 9 and 21 had nothing against a game with male-oriented activities and symbols. Future studies should focus on gender differences; the additional quests in Heroes of Math Island will be designed with this consideration and the mine quest will be revised.  4.4.2.1.3. Would You Play the Game Again? Statements 4.5, 4.6, 4.8, and 4.10 are related to the game. The statement 4.5 (“This game is boring”) was intended to determine the overall attitude towards the game. Only one student (6.7%) strongly agreed with the statement, none agreed, two students were neutral (13.3%), and a large majority of 73.3% disagreed and strongly disagreed: 20% (3) strongly disagreed and a 60% (11) disagreed (see Figure 28). The students who were neutral are Student 10 and Student 18. The student who disagreed is Student 13. These three students  132  were the least interested in playing the game (based on negative responses to other statements discussed in subsequent sections). Students 10 and 13 also reported a dislike of mathematics. The statement 4.10 (“I would like to play the game again”) complements the previous statement (4.5). Only one student, the same Student 13, strongly disagreed. Similarly, two students were neutral (13.3%), and a large majority of 73.3% agreed and strongly agreed with the statement (see Figure 28).  Both students (Student 7 and Student 15), who were neutral to the statement 4.10, disagreed that the game is boring. I conclude that their lack of interest in future play has different reasons than boredom.  Figure 28. This game is boring / I would like to play again.  Statements 4.6 (“This game was more interesting than video games that I played”) and 4.8 (“This game was more interesting than math games that I played”) also complement statement 4.5 (“This game was boring”). Unfortunately, only 14 students responded them, but I considered reporting the results as they are worthy. A majority of students 78.6% (11) 0123456789101 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeThis game is boringI would like to play the gameagain 133  considered that this game was more interesting than other math games (35.7% (5) strongly agreed, 42.8% (6) agreed), and only 14.2% (2) were neutral, nobody disagreed, and only 7.1% (one student) strongly disagreed, the same Student 13. Responses to the statement “This game was more interesting than video games that I played” was not responded in the same way: only 21% (3) of students agreed, more were neutral 42.8% (6), 28.8% (4) disagreed, and 7.1% (one student) strongly disagreed. Even if the perception of students was that the game is not more interesting than a video game, it is supportive to learn that it was considered better than other math games. See chart presented in Figure 29.   Figure 29. This game is interesting.   Students were asked in interviews what improvement would they suggest. Their suggestions were towards adding more content to the game (quests), and including awards and leaderboards. Some suggestions were related to game mechanics that are characteristics to video games targeting this age group, like collecting items and “dressing their avatar”.    012345671 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeThis game was moreinteresting than video gamesthat I playedThis game was moreinteresting than math gamesthat I played 134  4.4.2.2. Affective Domain: Attitude Towards Mathematics The following statements have been included with respect to attitude towards math: Table 9. Affective Domain: Attitude Towards Mathematics Statement ID Statement 1.7 I felt upset when I did not answer correct. 2.8 I had fun when I responded to math questions before play 2.9 I had fun when I responded to math questions after play  An important percentage of students (40% or 6 students) disagreed with the statement “I felt upset when I did not answer correct” and the same number of students reported neutral. 20% (3) of students reported feeling upset; however no participant strongly agreed or disagreed (see Figure 30).   Figure 30. I felt upset when I did not answer correct.  The reason why students did not feel (very) upset for making mistakes is because is common knowledge that learners learn from mistakes and at this age, participants in this study already had this idea. Student 11offered the following explanation: “I am not really sad when I have questions wrong because everybody has questions wrong. ‘Cause you learn 012345671 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI felt upset when I did notanswer correct. 135  from the mistakes”. The same idea was expressed by Student 21; asked when she learned during the game, she responded: “Well I learned from mistakes and from the questions and the answers and the hints”. Even more, Student 2 during interview explained that making mistakes increased his curiosity:  Interviewer: Did the game increase your interest in the topic? Student 2: Yeah, because I got so many questions wrong. I want to know more about it. Interviewer: So, when you keep getting something wrong you feel curious about. Student 2: Yeah I want to get this right, now. I also wanted to see the student’s reactions to the pre- and posttest (statements 2.8 and 2.9). The two statements were responded similarly: more students agreed and strongly agreed indicating a good attitude towards writing math tests: 53.3% (8) and 60% (9) agreed or strongly agreed with respect to pretest and respectively posttest (see Figure 31). There is a slight difference indicating more satisfaction with respect to the posttest which is consistent with the improvement of tests scores from before to after play (test scores will be discussed later in this chapter in Section 4.4.4.1). The percentage of students who disagreed or strongly disagreed with the statements decreased from four students (26.6%) in the case of pretest to only one student (6.7%) in the case of posttest. More details follow in the section dedicated to learning (see Section 4.4.4).  136   Figure 31. I had fun with the pre- and posttest.  4.4.2.3. Emotional Character: The Monkey. Is She Annoying? The monkey is the emotional character of this game. The monkey responded to the students’ play by showing emotions based on the students’ performance in the game: frustrated, sad, neutral, happy, and confident. The following table contains the statements related to the monkey. Table 10. Affective Domain: Attitude Towards the Emotional Character Statement ID Statement 1.3 The monkey was annoying. 1.9 The monkey reacted correctly to my emotions. 1.12 I was proud to see the monkey happy. 1.15 I felt shame when I made mistakes and the monkey was sad. 1.17 I didn’t mind to see that the monkey was sad. 2.2 I had fun when the monkey was happy. 2.3 I had fun when the monkey was sad. 2.4 I had fun when the monkey was confident. 2.5 I had fun when the monkey was frustrated.  0123456781 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI had fun when I responded tomath questions before playI had fun when I responded tomath questions after play 137  Interesting results were obtained with regard to the monkey. I started from the assumption that the monkey character would be attractive and appreciated by students. However, not all students liked the monkey: only 53.3% of students disagreed or strongly disagreed that the monkey was annoying; 26.6% of students were neutral and 20% of students strongly agreed that the monkey was annoying.  Figure 32 presents the results:  Figure 32. The monkey was annoying.  Some students complained in post-questionnaires and interviews of being distracted by the monkey’s animations and would like to have the choice to turn her off. Another group of students indicated that they did not even notice the monkey. When asked if he noticed the monkey, Student 2 responded: “I noticed the monkey in the beginning, but when he was here, [I] less noticed [him] because I was thinking about the questions and I really didn’t see it.” For example, an extract from the interview with Student 9 indicates this indifference: Interviewer: Did you notice the monkey? Student 9: Really, no. Interviewer: Was the monkey annoying? 01234561 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeThe monkey was annoying 138  Student 9: Maybe a bit, but I didn’t really notice it. Student 18 also did not notice the monkey: he reported in the open-ended portion of the post-questionnaire: “I did not see the monkey react”. In interviews he elaborated: “it did not bother me…I am not very observant”. Similarly, Student 12 liked the monkey but did not notice him during the game: “Well I thought he was cute. Didn’t see him very much though.” Student 4, even if she liked the monkey, she did not like the permanent blinking of the monkey. To the statement “Please tell us what emotions did you experience when you played the game”, she responded “Happy (because of the monkey); Annoyed (monkey blinking)”. She elaborated during the interview: “I was happy because the monkey kept making these really… he looked really happy, he was like confident. So I was happy and I was sort of annoyed at the monkey’s blinking.” Student 5 strongly disagreed with the statement “The monkey was annoying”; however he also mentioned during the interview being distracted by the monkey. He suggested: “Just do it one time (the animation), then, yeah, just smiling it’d work better. Like every five minutes, he’d jump up, so you have more of a chance to think.”  Student 13, besides not finding the game interesting, did not like the monkey:  Interviewer: Did you notice the monkey? Student 13: Yes, I didn’t like him Interviewer: You did not like him? Student 13: No. Interviewer: No? Why not? Student 13: He was annoying. Interviewer: How would you like him to be? Student 13: Maybe, there shouldn’t be a monkey.  139  Interviewer: Would you like to see another character maybe? Student 13: Well, I think that, maybe, if the monkey got too annoying, there should be a stop button. Student 13 indicated that she did not like mathematics and would like a writing game (interview excerpt is in Section 4.4.3.2). Later in the interview she indicated that having a monkey character that participates in a different way in the game was more attractive to her; however, she was very clear that mathematics content is not what would give her motivation to play the game: Interviewer: Would you like to see the monkey needing help, and maybe, [would you like] to help the monkey? Student 13: Yup Interviewer: Would that give you motivation to work, even if it is math? Student 13: No. Interviewer: Not for math? Student 13: (Shaking the head) No! Interviewer: For writing? Student 13: Yes! Interviewer: Did the monkey respond to what you did? For example, was the monkey happy when you were doing well or were happy, or the monkey was sad when you were frustrated or something like that?  Student 13: I didn’t notice that part.  140  With so many students not even noticing the monkey, it is easy to interpret why so many students strongly disagreed, disagreed, or were neutral with respect to the statement: “The monkey reacted correctly to my emotions”. Figure 33 presents the results:  Figure 33. The monkey reacted correctly to my emotions.  Only 20% (3) students agreed or strongly agreed. These students were Student 8, Student 15 and Student 21: these students also strongly disagreed that the monkey was annoying. Student 8 elaborated in the interview:  Interviewer: Did [the monkey] respond well to what you were feeling?  Student 8: It did, it kinda was doing what I felt like. When I was failing I felt mad and the monkey felt sad, so it kinda made sense. Student 11 liked the monkey without being able to give a reason, which is normal because people cannot always explain their feelings. He did not consider that the monkey responded to his feelings and the way the character was implemented created more frustration than good for him:  012345671 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeThe monkey reacted correctlyto my emotions 141  Interviewer: Did you notice the monkey in this game? Student 11: Ya! I like him. Interviewer: You like the monkey. Why? Student 11: I do not know. Interviewer: You don’t know? You just like him? Student 11: (big smile) Ya. Interviewer: Did you notice that the monkey was showing some emotional faces? Student 11: Ya. Interviewer: What did the monkey show? Student 11: (playing the monkey) Like confident. He is like happy and he is like sad… Interviewer: Did the monkey respond to your feelings? Student 11: I do not know. I felt frustrated when I was supposed to catch prime numbers from the last one but I got a little mixed up there and the monkey kept being sad and I was more frustrated. Student 21 in her interview reported the she did not notice the monkey “that much”; except for confidence (she even made a face imitating the monkey’s animation). The video confirms that Student 21, a strong mathematics student, avoided making mistakes and the monkey was in a happy state or confident state all the time.  However, the majority of students 53.3% (8) reported being proud to see the monkey happy: 40% (6) agreed and 13.3% (2) strongly agreed with the statement: “I was proud to see the monkey happy”. 20% (3) students were neutral, 6.6% (1) disagreed and 20% (3) strongly disagreed. Figure 34 presents the results.   142    Figure 34. I was proud to see the monkey happy.  Student 2 and Student 21 were the students who strongly agreed with this statement. Student 2, analogous to Student 21 was a strong math student who also played the game carefully to avoid making mistakes. Except to a moment at the beginning of the game when the monkey shows a sad animation, for Student 2 the monkey is happy and confident. Student 6 also was cautious in making mistakes and the monkey was only happy and confident for him, too, but he strongly disagreed with this statement.  The statement “I felt shame when I made mistakes and the monkey was sad” was disagreed upon or students responded neutral: only one student (Student 15) agreed (see Figure 35). This is consistent with the affect analysis results: shame was an emotion that was observed by judges very seldom: 0.1% by Judge 1 and 0.4% by Judge 2.  012345671 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI was proud to see themonkey happy 143    Figure 35. I felt shame when I made mistakes.   Statements 2.4 (“I had fun when the monkey was confident”) and 2.5 (“I had fun when the monkey was frustrated”) were responded as N/A by few students probably because the monkey did not react with these two emotions or they did not notice them. Therefore, I did not build a chart for them. All students responded statements 2.2 (“I had fun when the monkey was happy”) and 2.3 (“I had fun when the monkey was sad”). With respect to the statement 2.2, mostly students agreed or strongly agreed 46.7% (7) or were neutral: 40% (6). Only 13.3% of students (2) disagreed and none strongly disagreed. However, students did not like to see the monkey expressing sadness: one student (6.7%) agreed, 20% (3) of them were neutral, but the large majority disagreed 60% (9) or strongly disagreed 6.7% (1) to the statement 2.3 (see Figure 36). 0123456781 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI felt shame when I mademistakes and the monkey wassad 144   Figure 36. I had fun when the monkey was happy / sad.  Students had mixed feelings regarding the monkey character. Student 2 considered that showing emotions is beneficial and offered advice for future design: “I think the emotions are good and maybe even a voice too, like if you are doing good when he feels those emotions he could say like good job or keep trying, keep it up”. Student 9 (who did not notice the monkey) suggested that the monkey can participate in the game by offering hints to the student: “If you are making too many mistakes then I would want to go to the wiseman but if I got one wrong then instead of that thing popping up then maybe the monkey can give the hint”. Not all students wanted the monkey to help. Student 7 indicated that the wiseman or wisewoman should help and suggested a passive role for the monkey:  Interviewer: Which character would you like to be your helper, to give you help, to be the teacher? Student 7: Let’s see, the wiseman maybe? Interviewer: The wiseman? What would you like the monkey to do? 0123456789101 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI had fun when themonkey was happyI had fun when themonkey was sad 145  Student 7: Well, the monkey doesn’t really do anything so just, maybe, just sit, and sitting there and watches I think. Other students offered ideas of improvement of the monkey character not necessarily related to emotions or the game mathematics content, but related to achievement and awards. Similar to Student 4 (discussed above in Section 4.4.2.1.2) who suggested collecting items for the monkey, Student 12 suggested that the monkey should wear different hats based on the student`s achievements in the game. Similarly, Student 21 indicated that the monkey should respond to the student`s performance: Interviewer: What do you want the monkey to do? Student 21: Stuff. Interviewer: Like what? Student 21: Like, maybe if you got like, 98%, right? Then it could do a little victory dance of something. Interviewer: Do you see the monkey do a victory dance only in certain occasions? Because right now the monkey is all the time responding. Student 21: Yeah, like at the end, once you see your score, and if you did really good then it would do a victory dance. One important issue was the presence of the monkey on the screen all the time and the permanent movement that was indicated by several students as distracting. In conclusion, the monkey character needs to be re-designed. Results from this section will influence decisions of future design and will be represented as recommendations in the next chapter.    146  4.4.3. Cognitive Domain The cognitive domain was represented in the post-questionnaire by a smaller number of statements because of the study’s focus on the affective aspects of interaction, however,  it is hard to discriminate because, as discussed in details in Section 2.4, cognition and emotion are strongly interrelated and cannot be separated (Lazarus, 1982, 1984, 1991; LeDoux, 1995, Scherer, 2001b). Also, as I mentioned earlier, some statements correspond to more than one domain or attitude. Statements 1.5 (“The level of difficulty was right”) and 1.11(“The game was too easy or too hard for me”) are related to the level of difficulty; they could be judged with respect to the attitude towards the game and also towards mathematics. However, because a game design principle is to include levels of difficulties, I considered these two statements as attitudes toward the game. Similarly, statement 4.12 (“I would like to see more quests”) could have more meanings: more quests represent more cognitive load and also more learning. It is also hard to separate the game from its content: the level of the difficulty in the game is related to the mathematical level of difficulty. Results from analysis of statement 4.12 are indicators of the interest students have in playing the game in the future. Table 11 presents the cognitive domain questionnaire blueprint.    147  Table 11. Cognitive Domain: Questionnaire Blueprint  Affective domain Cognitive domain Learning  Attitude towards the game  1.5; 1.11; 4.12  Attitude towards math  1.8; 1.16   4.4.3.1. Cognitive Domain: Attitude Towards the Game The following table contains the statements that have been asked with respect to the attitude towards the game: Table 12. Cognitive Domain: Attitude Towards the Game Statement ID Statement 1.5 The level of difficulty was right. 1.11 The game was too easy or too hard for me. 4.12 I would like to see more quests.  Statements 1.5 and 1.11 complement each other. The results are consistent and indicate that the level of difficulty was generally perceived as appropriate. Figure 37 presents the chart diagram. Only one student (6.6%) strongly disagreed and one student (6.6%) disagreed with the statement “The level of difficulty was right” and the same number of students agreed and strongly agreed with the opposite statement: “The game was too easy or too hard for me”. The number of neutral responses was not the same: 26.7 % (4) and respectively, 46.7% (7); however a majority of students agreed or strongly agreed with the  148  first statement (60% or 9 students) and 40% (6) disagreed or strongly disagreed with the second one (see Figure 37). The reason might be that having the right level of difficulty is different from having a too easy or too difficult game. In the world of video games, levels of difficulty are desired and supposed to be challenging.   Figure 37. Level of difficulty.  Statement 4.12 was responded almost unanimously in a positive way: all students except one (Student 13) agreed or strongly agreed that they would like to see more quests (see Figure 38).  0123456781 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeThe level of difficulty wasrightThe game was too easy or toohard for me 149   Figure 38. I would like to see more quests.  This almost unanimous response gives me the confidence to conclude that students found a value in playing the game and would like to see more math topics. In interviews, all students except one (Student 13) indicated enjoyment in using this game to study. An interesting point was made by Student 2 who also strongly agreed that the level of difficulty was adequate. He considered that learning with the game even challenging, was easier to accomplish than learning from textbooks: Interviewer: Do you think that the [game] questions are hard? Student 2: These questions are harder than the textbook, but the textbooks’ … it’s harder to understand. However, some students would prefer both (i.e. Student 21) and some students still prefer only books: Interviewer: Would you study, for school, with this game? Or do you prefer something else, like a book or the teacher? Student 13: I would rather use a book. 0123456789101 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI would like to see morequests 150  4.4.3.2. Cognitive Domain: Attitude Towards Mathematics The following table contains the statements that have been asked with respect to the attitude towards mathematics: Table 13. Cognitive Domain: Attitude Towards Mathematics Statement ID Statement 1.8 I became curious about math (divisibility/prime numbers/factorization) by playing the game. 1.16 I didn’t want to think about math (divisibility / prime numbers / decomposition) when I was playing the game.  Two statements targeted the attitude towards mathematics with respect to the cognitive domain: statements 1.8 and 1.16. Statement 1.8 is also a statement that falls into two domains: affective and cognitive considering that curiosity was one of the emotions analyzed in videos. However, I decided to analyze this statement in the context of the cognitive domain. Statement 1.16 is intended to understand the relationship between the game play and content. Even if a good percentage of students (40% or 6 students) responded that their curiosity was increased by the game play, the same numbers were neural, a one student (6.7%) strongly disagreed and two students disagreed (13.3%). More effort should be spent in the future to understand what should be done to increase curiosity. The student who strongly disagreed was Student 18, and Students 8 and 13 disagreed (see Figure 39).   151   Figure 39. I became curious about math.  When I asked Student 18 about how he felt during the game play he responded “my mind was thinking”. He reported being frustrated sometimes but determined to finish the quest. Student 18 indicated that anticipation of what would be next (curiosity) motivated him to continue to play the game; he would not have stopped the game even if tired because he was curious about what was next. His curiosity was not towards the content of the game, but towards the game experience: Interviewer: When I asked if you wanted to have fun when playing the game you were indifferent, and when I asked if you wanted to learn math by playing the game you were also indifferent. What was your motivation to play the game? Student 18: My motivation…I think for me it wasn’t “I wanted to have fun or I wanted to learn”. I just did it hmmmm… not that I wanted to learn, but because I wanted to see what is was.  Student 13 was not interested in this game especially because of the mathematical content. Asked if she was curious about the game she responded: 012345671 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI became curious about math(divisibility/primenumbers/factorization) byplaying the game. 152  Interviewer: Were you curious about the game? Student 13: No. Interviewer: Are you curious about numbers, for example if numbers are prime? Student 13: (Shaking strongly) No. Interviewer: What kinds of game would you like to see, If not a math game? Student 13: Art, or English. Interviewer: English, ok, maybe spelling or what? Student 13: Writing. Statement 1.16 [“I didn’t want to think about math (divisibility / prime numbers / decomposition) when I was playing the game”] was responded very favorably: a large percentage of students (73.3% or 11 students) strongly disagreed or disagreed with 40% (6) strongly disagreeing and 33.3% disagreeing (see Figure 40). Two of students (13.3%) were neutral and the same number agreed. The students’ response is consistent with the expectation of the game to be based on mathematics. The response is also consistent with the level of engagement found in the affect analysis. In the next session dedicated to learning, a related statement (“I wanted to learn math by playing the game”), will be discussed in details.   153   Figure 40. I do not want to think about math.  4.4.4. Learning An important inquiry of this study is related to the attitude towards the mathematics content of the game and students’ achievements: “What are students’ levels of interest and achievement in the mathematics content areas after gameplay?” In the following section I will report and analyze the collected data with the intent to respond to this question.  4.4.4.1. Achievements: Pretest and Posttest The pre- and posttests consisted of problems related to the three game activities: divisibility, prime numbers and decomposition. Students reported different levels of previous knowledge with respect to the three topics: more students reported a good grasp on divisibility, than on prime numbers and decomposition.  Figure 41 represent in a chart the reported level of knowledge. Divisibility was a topic that was reported to be better known before the experiment. There is a strong correlation between previous knowledge of prime numbers and decomposition (rho = 0.809, rs = 0.000). 012345671 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI didn’t want to think about math (divisibility / prime numbers / decomposition) when I was playing the game.  154   Figure 41. Reported previous level of knowledge.  I rejected the null hypothesis of no difference between the pretest and posttest. One student (Student 5) did not write the posttest because he was too tired, therefore I computed the pre- and posttest scores for 14 students only. It should be noted that, for these 14 students, there was a significant improvement from pretest (M = 77.7%; SD = 9.3%) to posttest (M = 83.5%; SD = 8.7%), t (14) = 2.2; two-tailed p < 0.007) suggesting that students likely learned from their interaction with Heroes of Math Island.  Partial results in pre- and posttests reflected the students’ reports of previous knowledge. The total improvement from pre- to posttests was gained less from divisibility and more from prime numbers and decomposition (see Table 14). The standard deviation is low, indicating that there are no large differences between students. According to these results, the game improved the students’ knowledge especially with respect to prime numbers and decomposition. It is important to note that prime numbers and decomposition are harder topics than divisibility, and also were reported as less known by students.  0123456789101 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeGood at DivGood at PNGood at Decomp 155  Table 14. Pre- and Posttest Results   Divisibility Prime Numbers Decomposition Pretest M = 93.5%; SD = 0.6% M = 47.1%; SD = 0.8% M = 66.7%; SD = 2.1% Posttest M = 95.8%; SD = 0.7% M = 55.7%; SD = 0.9% M = 76.2%; SD = 2.1%  The post-questionnaire statements related to learning were observed with respect to the students’ attitude towards the game and towards mathematics. Many post-questionnaire statements belong to more than one category and this is normal: it is not possible to separate the game from its contents.  The statement 1.6 (“I wanted to learn when I was playing the game”) was eliminated because of overlap with statement 1.13 (“I wanted to learn math by playing the game”) and because statement 1.6 did not receive a response from all students.  Statement 3.4 (“I learned math when I got examples”) was also eliminated because was not responded by all students. A possible reason for the low response rate is that students were careful to not make mistakes and be sent to the wiseman’s library (which resulted in the re-start of the activity) and therefore not all received examples from the game (examples were generated when a student repeatedly made errors). Table 15 indicates these changes:        156  Table 15. Learning: Questionnaire Blueprint  Affective domain Cognitive domain Learning  Attitude towards the game   1.13; 3.1; 3.9; 4:11 Attitude towards math   1.13; 3.1; 3.2; 3.3; 3.6; 3.9; 3.10; 4.11 Attitude towards the monkey     4.4.4.2. Students’ Perceptions of Learning The following table contains the post-questionnaire statements related to learning:  Table 16. Learning: Attitude Towards the Game and Mathematics Statement ID Statement 1.13 I wanted to learn math by playing the game. 3.1 I learned math when I played the game. 3.2 I learned math when I made mistakes. 3.3 I learned math when I got hints. 3.6 I learned math when I helped the miners. 3.9 I learned math when I finished the quest. 3.10 I learned math when I had harder questions. 4.11 I believe that I can learn math better by playing this game.  Responses to the above statements were analyzed for frequencies. Generally students agreed or strongly agreed. A detailed analysis follows:  157  4.4.4.2.1. Before Game Play Students started with a good attitude towards mathematics: only one student (Student 10) disagreed and two students (Student 13 and Student 18) were neutral with respect of the statement 1.13: “I wanted to learn math by playing the game” (see Figure 42).  Statement 1.13 complements statement 1.16 [“I didn’t want to think about math (divisibility / prime numbers / decomposition) when I was playing the game”] discussed in the previous section. Responses to these two statements are opposite indicating consistency in students’ responses.  Figure 42. I wanted to learn math.  4.4.4.2.2. After Game Play From the group of 15 students, only one student (Student 13) responded neutral to the statement “I learned math when I played the game” and 93.3% of students agree or strongly agreed with this statement: from the 14 students, 6 agreed and 8 strongly agreed (see Figure 43).  01234567891 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI wanted to learn math 158   Figure 43. I learned math when I played the game.  Student 13, the student who responded neutrally, was completely uninterested in mathematics. When asked what would be her attitude towards the game if the game would be improved, Student 13 responded that the game would not interest her unless the math content would be removed and replaced with “more fun learning material”. As discussed earlier, Student 13 disliked mathematics. During the game, when observers noted that students were tired or struggled too much, they advised them to stop or play again the easier divisibility activity. It was interesting to note that students were determined to finish the quest. None of the students in this group stopped before finishing the quest. Observers agreed that especially Students 5 and 8 were very tired during the game play, but they did not want to stop and succeeded to finish the quest. Student 5 was so tired after the game play that he did not have energy to write the posttest. 01234567891 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI learned math 159    The third activity (decomposition) had an increased level of difficulty. For students who did not have strong concepts of mathematics (e.g., Student 4 and Student 5), I argue that finishing the quest was more important than being tired or overwhelmed by the task. Finishing the quest was a strong motivator. The students’ responses to “I learned math when I finished the quest” substantiate this idea: 80% of students (12) agreed or strongly agreed with the statement. Similar responses were given to the statement “I learned math when I helped the miners” which I interpret as another strong motivator. Few students were neutral, but none of the students disagreed (see Figure 44).  Figure 44. I learned math when I helped miners / finished the quest.  Learning from mistakes, hints, and harder question was not responded equally by all students. More students agreed or strongly agreed that harder questions helped them learn (73% or 11 students); however only 53.3% (8) were in the same category with respect of learning from mistakes. Students believed that they learned from harder questions: 80% (12) of students agreed (40%) or strongly agreed (40%) to the statement and only 6.6% or one 01234567891 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI learned math when I helpedthe minersI learned math when I finishedthe quest 160  student in each category strongly disagreed, disagreed or was neutral. Hints were not considered helpful for learning; in fact 40% (6) of the students responded neutral and 33.3% (5) disagreed or strongly disagreed (see Figure 45). This observation will be carefully taken into consideration for the future re-design of the game.  Figure 45. I learned math from mistakes / hints / harder questions.  I also generated a chart of all statements from the learning category. It is remarkable, and reassuring given the intention of the game, to observe that frequencies are concentrated on the right towards “agree” and “strongly agree”. Figure 46 summarizes these findings: 012345671 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI learned math when I mademistakesI learned math when I gothintsI learned math when I hadharder questions 161   Figure 46. Learning: Detailed histogram.  I also computed the summative frequencies for a better visualization. 71.6% of responses of the 15 students to the eight statements indicated that they agreed or strongly agreed (30% strongly agreed and 41.6% agreed); 18.3% were neutral and only 10% disagreed or strongly disagreed (6.6% disagreed and 3.3% strongly disagreed) with the learning statements. Figure 47 presents the results: 01234567891 StronglyDisagree2 Disagree 3 Neutral 4 Agree 5 StronglyAgreeI wanted to learn mathI learned math when I playedI learned math when I mademistakesI learned math when I got hintsI learned math when I helped theminersI learned math when I finishedthe questI learned math when I had harderquestionsI believe I can learn math byplaying 162   Figure 47. Learning: Summative histogram.   The above findings are indicators that learning likely happened during the game play. As expected there is a strong correlation between the general attitude towards mathematics (“I like math”) and the willingness to learn mathematics by playing the game (rho = 0.720; rs = 0.002). Students who wanted to learn mathematics by playing the game also believed that in the future they can learn mathematics by playing (rho = 0.706; rs = 0.003). I also found a strong correlation between learning from mistakes and learning from hints (rho = 0.616; rs = 0.015), and between learning from harder questions and finishing the quest (rho = 0.620; rs = 0.014). It is interesting to observe that students who reported being good at divisibility also reported learning from harder questions (rho = 0.695; rs = 0.004) and by helping the miners (rho = 0.682; rs = 0.005); however students who reported being good at prime numbers (a more complex task) reported to learn from harder questions (rho = 0.668; rs = 0.007) and from helping the miners (rho = 0.677; rs = 0.006), and also from mistakes (rho = 0.533; rs = 0.041) and by finishing the quest (rho = 0.641; rs = 0.010). Learning from harder questions is 01020304050601 2 3 4 5Learning Math (summative) 163  an indicator of a good attitude towards learning. Student 2 was one of the students who appreciated the harder level of difficulty implemented in the decomposition activity: Interviewer: Did you find the game difficult for decomposition? Or, it was the right level? Student 2: It was the right level because when you are making a moderate level of mistakes you are learning from it. He also reported that he learned from mistakes. Asked if he felt shame for making mistakes, he responded: Student 2: Well, then again, you might not feel shameful because there is nothing to be shameful of because you are learning the whole time. But some people do if they get lots of questions wrong. Interviewer: Let’s say you get lots of questions wrong, would you feel shame? Student 2: Not really, because I know that I am learning form my mistakes.  A Pearson correlation coefficient of 0.71 was found between pre- and posttest indicating that a correlation exists between the two scores: students improved theirs posttest scores propositionally to pretest scores. I also computed Spearman’s correlation coefficient matrix with respect to learning statements including the scores from pre- and posttests. It is notable to observe that besides the correlation between pre- and posttest, no other strong correlation was found indicating that the score was not a factor related to students’ perception of learning gains.  Student 2 (who gained 73.9% in pretest and 87% in posttest) reported that he felt that the better results in the posttest are consequences of the game play:  164  Student2: The pretest, I forgot a couple of things, but then I remembered [them], same with the posttest. And in the posttest, after the game, I learned a lot of stuff, and so I did better in the posttest, I think. A good attitude towards mathematics also increases the chance for a student to want to play a math game. Students who accept challenges (like learning from harder questions) are more immersed in the game and associate learning with game challenges like finishing the quest (accomplishing a task) and helping the miners (working towards a good cause). A good previous understanding of the subject increases the perception of learning gains, but because I did not find any correlation with the results of pre- or posttest, I argue that students’ perception of learning gains doesn’t depend on scores, but more on attitude and enjoyment. Prior insights regarding enjoyment were presented in previous sections; however it is noticeable to mention here the relationship between fun and learning resulted from the interview with Student 9: Interviewer: Did you have fun when you played the game? Student 9: Yeah Interviewer: Why did you have fun? Student 9: Because I learned and I learned though a game and I like learning through games instead of just text book stuff.     165  4.5. Triangulation: Judges’ Perception of Process and Student Interaction Affect analysis was performed by two judges: Judge 1 (I) and Judge 2 (one of the observers). One more observer (Judge 3) was intended to perform affect analysis. Judge 3 started but did not complete this task; however, he was an important contributor to this study by observing experiments and by participating in the process of deciding which emotions to consider. A third observer was involved in the process of emotion labeling and experiments, but because he left for a co-op job, was not available for affect analysis.   In order to get better insights of the affect analysis, a formal interview was conducted with Judge 2 and Judge 3 (see Appendix 11). As described in Section 3.3.2.2., defining and developing the set of emotions and the process for emotion analysis took a long time and several iterations. The purpose was to identify a set of emotions that is comprehensive and adequate to this study, and design a process that can be successfully followed by judges.  Regarding the 20-second interval, Judge 2 and Judge 3 agreed that it was suitable. Judge 2 brought the issue of emotions that were transient from one 20-sec time frame to the next one: “I think 20 seconds is enough to observe more than one emotion. However, sometimes the emotion was in transition from one to another so it was sometimes difficult to decide which 20 second interval to report it to. 10 seconds is too short I think.” This was an issue that was discussed thoughtfully. Finally, I decided on reporting an emotion observed continuously in all intervals of occurrence. During interviews, both judges confirmed that the process was adequate. Judge 2 said: “I think the task was clear and the system was easy to understand. However, it was the judgment of emotions that was difficult.”   166  All judges, not only Judge 2 found that the process of judging emotions was challenging. Judge 2 added: “it was always a challenge to distinguish between engaged and some of the subtle emotions”. For me, judging emotions was very demanding not only because it is a challenge to distinguish between emotions, but because of the high responsibility judges have to interpret other people’s affective states. On the same note, Judge 3 explained that he did not complete his task because he did not feel confident with his judgment. He made the following confession: I said that judging emotions was difficult. I suppose mostly it was because I questioned whether I'm as capable as other people in judging emotions. And while judging emotions, I wasn't always certain how to interpret them. Often times the student seemed not to display any emotion, and I might wonder whether I was missing something, or whether there really wasn't any emotion to report. The thing I tried to keep in mind was how much time to spend on deciding on how to judge an emotion, being decisive enough, on the one hand, while not rushing judgment and doing a sloppy job. Judges noted that engaged concentration was the main emotional state that students experienced. Judge 3 referred to discussions we had for the emotion labeling process:  Interviewer: What about engaged concentration? Did you find many instances? Judge 3: Almost all the time. Any time the student was playing the game. I think we discussed this one in our meetings because it seemed like it was difficult to find instances when there wasn't engaged concentration.  167  This observation is consistent with the results of the emotion analysis, the learning gains (the improvement from pre- to posttest) and the positive responses students gave to post-questionnaire statements related to learning. In interviews, participants in this study also reported a high engagement, some of them complaining that the emotional character (the monkey) bothered them and disturbed their focus. Some emotions were easier to observe than others. I found that emotions that involve stronger facial expressions and posture or verbal clues like confidence, frustration, pleasure/delight and surprise were easier to observe. Judge 2 confirmed my opinion: “Emotions like joy and confidence were very easy to judge. I felt that emotions having subtle differences were very hard to judge and I did not feel confident. For example, disappointment and boredom may have similar facial expressions and it was hard to judge.” I also found that it was sometimes hard to differentiate between engaged concentration and confusion/hesitation and even confidence because it was hard to decide if the student stopped working and did not decide yet what to do because he/she is confused or because is just thinking. On the same note Judge 2 said: “it was often very difficult to distinguish between engaged-confident as well as engaged-hesitant. It was difficult due to the amount of time the student time spent on the task and the students grew more comfortable with the game/situation over time.”  Not only emotions, but some students were more difficult to judge because they did not express emotions (e.g., Student 6). Other students such as Student 5, Student 15 and Student 21 appeared to enjoy the interaction to a lesser extent, but in interviews all indicated that they liked the game and disagreed or strongly disagreed in the post-questionnaire that the game is boring.   168  It is known that extrovert people express their emotions well, however Judge 2 indicated that passionate students about the game were also easier to judge: “I do not have the list of names but I remember one other girl (I believe she has red/blonde hair and was very energetic) who was easier to judge and she was confident and eager to play as well. The less interested the student was in the game and in math the harder it was it judge. Also, the more reserved/introverted the student was the harder it was to judge. However, I believe that the most important factor was the level of student interest in participating. Expressiveness is a lesser but important factor as well”. Judge 2 referred to Student 4. It is notable to observe that the Cohen’s Kappa for one emotion per interval as described in section 4.3.1 was the highest for Student 4: K = 0.976. One interesting aspect of this study is that judges did not report too many instances of pleasure (only 5% by Judge 1 and 3.8% by Judge 2); however the most reported state by students was happiness. I asked the two judges to elaborate with this respect. Two interesting explanations emerged. One valuable point made by Judge 2 is related to the particularities of this study: interaction with a computer and/or a video game even if pleasant or interesting doesn’t show on the face of the participant because it is a non-personal interaction with a machine:  I think that many people feel happy when using their computer (internet and games especially) but it is not the same joy experienced when interacting with friends so it is expressed differently. I feel happy to use my computer but I rarely find myself smiling unless it is a funny video or article (Judge 2). The second explanation was given by Judge 3 who believed that observing the participants interfered with their behaviors. This is a known problem in research: the context of the  169  experiment (e.g., the tools used for data collection, the environment, and the experimenters) adds extra variables that should be taken into consideration in data analysis. I agree with the points of view of the two judges, and I believe that one more issue should be remarked. Student 2 indicated a “sense of accomplishment” and other students described their experience with other words like interesting and effective, but the main words used by students were happy and fun. I believe that engagement, focus, and completion of work provide satisfaction that, because of the rewarding nature, is perceived as joy and reported by participants as happiness and fun.  4.6. Conclusion Judges of affective states analyzing students’ videos at a 20-second granularity reported 12 emotions (plus neutral): boredom, confidence, confusion/hesitancy, curiosity, delight/pleasure, disappointment/displeasure, engaged concentration, excitement, frustration, pride, shame and surprise. The results of affective analysis were consistent with results from the post-questionnaire and interview analysis in many respects:  Boredom was reported only 1.5% of the time by judges and it is consistent with the observation of students playing the game. None of students complained of boredom in interview or reported boredom in post-questionnaire.  The negative emotion of shame was rarely observed by judges. Students did not report it; however, some students were upset when making mistakes or when the monkey displayed the shame animation.   The only negative emotions that were reported by students are frustration and confusion, but to a lesser extent. Confusion is in fact not a negative emotion for learning because deep comprehension can happen in moments of confusion  170  (D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006). Future efforts should be invested to address frustration because it can lead to a “vicious circle” resulting in boredom. Boredom is negatively associated with learning (D’Mello, Taylor, & Graesser, 2007).  Students were mainly engaged and motivated to do well. Many of them explained in interviews their state of focus; one remarkable quote was from the interview with Student 18: “my mind was thinking”.   Students did not describe their emotional states based on the set of emotions that I labeled: they were not provided with the description of these emotions as judges were. In fact, the list of emotions and their descriptions were developed in parallel with experiments (following the DBR paradigm). When asked what emotions they experienced during the game, students generally responded happiness, and to a lesser extent frustration, confusion, confidence and pride. I argue that the “happiness” state that students reported represents the spectrum of positive emotions of confidence, curiosity, engaged concentration, excitement, pleasure/delight, and pride.  Even tired, students wanted to finish the quest. Finishing the quest was a strong motivator for completion.  93.3% of students agreed or strongly agreed and only one student responded neutral to the statement: “I learned math when I played the game”.  Even if easy to follow and clear, the emotion analysis process was challenging to judges. Issues of personality of participants and expressiveness of their face should be taken into consideration in future studies.   171  The next chapter addresses the summary, conclusions, and recommendations for further research and development.    172  Chapter Five: Conclusion   The purpose of this study was to investigate what affective states are important with respect to student’s interaction with an educational game, more precisely the Heroes of Math Island game (described in section 3.2), with the long-term goal of tracking and responding to emotions in real-time. Acquiring a deep understanding of the students’ interaction with educational games and more precisely the emotional aspects of interaction is very important for design of ALTs and ITSs and more generally for ID.  The main research question addressed emotion identification elicited during gameplay. Additionally, the study explored the participants’ subjective reactions with respect of the game and the mathematics content. Learning gains were analyzed together with levels of interest and achievement in mathematics. The main research questions of this study are: 1. What affective states are important with respect to student’s interaction with an educational game?   a. What affective states are elicited during the Heroes of Math Island gameplay? Additionally, this study responded to the following two questions: 2. What are the students’ subjective reactions with respect to Heroes of Math Island and to the underlying mathematics content? 3. What are students’ levels of interest and achievement in the mathematics content after gameplay?  This study was based on three important theoretical frameworks: emotion theory, emotion and learning, and affective computing. The first step in defining this study was an extensive  173  literature review. I analyzed theories of emotions and emotion frameworks (Ekman, 1999; LeDoux, 1995; Lazarus, 1991; Ortony, Clore, & Collins, 1988; Russell, 2003) and surveyed research suggesting that more empirical research is needed for a deeper understanding of the relationship between emotion and learning (Arnone, Small, Chauncey, & McKenna, 2011; Astleitner, 2000; Hascher, 2010; Ingleton, 2000; Petrina, 2007; Picard, Kort, & Reilly, 2003; Weiss, 2000). Several research groups are engaged in identifying and detecting emotions elicited by educational software (e.g., Baker, D’Mello, Rodrigo, & Graesser, 2010; Conati & Maclaren, 2009; Jraidi, Chaouachi, & Frasson, 2013; McQuiggan, Robison, & Lester, 2008; D’Mello, Taylor, & Graesser, 2007; Grasser et al., 2006; Rodrigo et al., 2008, 2012; Trabelsi & Frasson, 2010).  This study contributes to ALTs, ITSs and affective computing by: (1) reviewing sets of emotions important for learning derived from literature and pilot studies and proposing an original set of emotions observed during gameplay; (2) analyzing inter-rater or inter-judge agreement at the individual student level, to gain a better understanding of how individual differences in expression affect emotion recognition; (3) examining in detail what and how many emotions actually occur or are expressed in the standard 20-second interval used in literature and the challenge or ease in which judges identify them (Baker, D'Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Rodrigo, et al., 2008; Rodrigo, et al., 2012); and (4) designing a standard method including a protocol and an instrument intended for trained judges to label and analyze emotions over 20-second intervals.  This study contributes to ID by: (1) exploring to what extent learning happened in the context of the Heroes of Math Island game; (2) exploring the students’ subjective reactions  174  with respect to gameplay and mathematics content; and (3) investigating students’ levels of interest and achievement in mathematics. Finally, this study contributes to methodology. The protocol used to label and to analyze emotions employed in this study can be used in similar, future studies.  5.1. Summary of the Study Given the important role of affect in cognitive processes and learning, it is important to investigate what emotions are experienced by students during gameplay. The main goal of this dissertation was to gain a better understanding of emotional states of students during interaction with an educational game (Heroes of the Math Island) in order to design more effective ALTs and ITSs that improve students’ learning, motivation, and engagement by adapting to emotional expression. Investigation of emotions and students’ perceptions and reactions is the first step necessary to design emotion classifiers for emotion detection, and decision systems for anticipating or responding to emotion. The participants or students in this study were 15 students (seven boys and eight girls) grades six and seven. The mean age was 11.4 and the median age 11.  The purpose of this study resides in the need of better understanding of emotion in learning. From neuroscience, we know that the areas of the brain that process emotions are also concerned with learning. Even if emotion is considered beneficial and important to learning (Arnone, Small, Chauncey, & McKenna, 2011; Astleitner, 2000; Boler, 1999; Hascher, 2010; Petrina, 2007; Scherer, 2001a), the focus in classroom instruction is primarily focused on cognition (Astleitner, 2000; Hascher, 2010) and “educational research and models of learning have shed little light on the interrelationships between emotions and learning” (Ingleton, 2000, p. 86).   175  The study used as a testbed a mathematics game, Heroes of Math Island, developed for this study and intended for students in grades five through seven. Students solved problems related to divisibility, prime numbers and decomposition (number factorization) in game-like activities happening in a mine. The game has aspects inspired from Prime Climb, an educational game for number factorization (Conati, 2002; Conati & Maclaren, 2009; Conati & Manske, 2009). The game content and design was validated in usability studies conducted with a teacher and two instructional designers. Heroes of Math Island has an agent (the monkey) that uses emotional expressions to respond to the student’s math performance in the game, similar to Rodrigo et al. (2012). For this study the monkey displays a neutral state, two positive (happy and confident) and two negative (sad and frustrated) emotional states.  Methodologically, this exploratory study used an empirical identification of emotion with a DBR paradigm by combining exploration with design (Design-Based Research Collective, 2003), and a triangulation mixed methods design (Gay, Mills, & Airasian, 2006) employing a quasi-experimental study and affect analysis. The research was designed in the following parts and stages: 1. Part One: Quasi-experimental study: pretest, intervention, posttest, followed by post-questionnaire and interview  2. Part Two: Affect analysis (emotion labeling process and video annotations) a. Stage one: Emotion labeling process b. Stage two: Video annotations by trained judges Stages of data analysis were: affect analysis (quantitative approach), descriptive and causal-comparative analysis (quantitative and qualitative approach), and triangulation. Data  176  analysis included descriptive statistics, t-test, and Cohen’s Kappa analysis (using the SPSS software) categorization, content analysis, and matrix analysis. Participants’ reactions to the game were analyzed using a descriptive and causal-comparative analysis and were based on post-questionnaire responses and interviews. The post-questionaire blueprint included a matrix of aspects of attitudes toward (1) game, (2) mathematics, and (3) the emotional agent of the game over (a) affective domain, (b) cognitive domain, and (c) learning. The participants’ interview responses were analysed with respect to (i) feelings experienced during the game, (ii) mathematics content and (iii) emotional agent of the game. 5.2. Summary of Contributions This study contributes to related research in ALTs, ITSs, and affective computing with respect to identifying and detecting emotions elicited (Baker, D’Mello, Rodrigo, & Graesser, 2010; Conati & Maclaren, 2009; D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; McQuiggan, Robison, & Lester, 2010; Rodrigo, et al., 2008; Rodrigo, et al., 2012; Trabelsi & Frasson, 2010). Specifically, this dissertation makes the following contributions:  (1) Identification and proposal of an original set of emotions based on literature and observed during gameplay:  boredom  confidence  confusion/hesitancy  delight/pleasure  disappointment/displeasure  engaged concentration  frustration   (2) Analysis of inter-judge agreement aggregated and at the individual participant level.  177  (3) Detailed analysis of emotions occurring in a standard 20-second interval used in literature. (4) Standard method including a protocol and an instrument that relies on trained judges reporting emotions over 20-second intervals.  This study contributes to the field of ID by:  (1) An exploration of the Heroes of Math Island game (designed for the purposes of this study) as an ALT based on the students’ perception of learning.   (2) An exploration of students’ subjective reactions with respect to the gameplay and mathematics content. (3) An investigation of the students’ levels of interest and achievement in mathematics. This study also contributes to the methodology field by designing a method used to label and to analyze emotions. 5.3. Conclusions The following conclusions resulted from this study: 5.3.1. Identifying and Detecting Emotions  Twelve emotions were analyzed in this study: boredom, confidence, confusion/hesitancy, curiosity, delight/pleasure, disappointment/displeasure, engaged concentration, excitement, frustration, pride, shame and surprise. A subset of these emotions (boredom, confusion, delight, engaged concentration, frustration and surprise) was proposed by D’Mello and Graesser (D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006) and were used in related research (Baker, D'Mello, Rodrigo, & Graesser, 2010; D’Mello, Taylor,  178  & Graesser, 2007; Graesser, et al., 2006; Rodrigo, et al., 2008; Rodrigo, et al., 2012). However, more and subtle emotions were observed during gameplay. Some emotions were included in other frameworks or studies of emotion (i.e., disappointment, displeasure, excitement and hesitancy). Emotions like confidence (Ingleton, 2000) pride and shame (Ingleton, 2000; Ortony, Clore, & Collins, 1988) play an important role in self-esteem and are important for learning. Pride and shame are important even if not present much of the time in this research study. Curiosity is an important emotion in fostering motivation, interest and engagement in learning. Arnone, Small, Chauncey and McKenna (2011) recommend “reconsideration of research methods for conducting research into curiosity and learning in new media technology-pervasive environments” (p. 194). Even if these added emotions were not frequently observed (for example curiosity, excitement, pride and shame were reported below 1.2%) this does not mean that these emotions are unimportant for learning. Confidence on the other hand can be strongly recommended for inclusion in emotion frameworks and further evaluation. With respect to what emotions were reported, the most frequent emotion was engaged concentration followed by confidence. It is remarkable to notice that engaged concentration was reported 83.9% by Judge 1 and 74.4% by Judge 2 (more than related research) and confidence was reported 24.9% by Judge 1 and 49.5% by Judge 2. I attribute the variations in reporting to differences in experience between the two judges; Judge 1 with experience in teaching was able to readily identify when students needed help. Students interacting with this game were highly engaged. Several studies showed engaged concentration to be the most frequent state: 68% in Baker, D’Mello, Rodrigo and Graesser (2010); 67.4% in Rodrigo et al. (2008), 43.45% in Rodrigo et al. (2012) and 42% in  179  McQuiggan, Robison and Lester (2008); however in this study engaged concentration was on average 79% of instances between the two judges. Judges were interviewed regarding engaged concentration:  Interviewer: What about engaged concentration? Did you find many instances of that? Judge 2: Most of the time the students were focused a lot.  Judge 3 (the observer who declined to continue as a judge): Almost all the time. Any time the student was playing the game. I think we discussed this one in our meetings because it seemed like it was difficult to find instances when there wasn't engaged concentration. As one of the judges, I also observed engaged concentration. Even Student 13 (who reported that a game with mathematics content doesn’t interest her) was highly engaged (engaged concentration was reported for her 48.5% by Judge 1 and 65.2% by Judge 2). Overall, all students were mainly engaged and motivated to do well. Many of them explained in the interviews their state of focus: e.g., Student 18 reported “my mind was thinking”.  The third most frequently reported state was confusion/hesitation, similar to Baker, D’Mello, Rodrigo and Graesser (2010) who report confusion to be the second most frequently observed emotion. The frequency of confusion in this study is actually higher than in Baker, D’Mello, Rodrigo and Graesser (2010) — about 26% instead of 13% — but this finding is still consistent with the observed positive learning outcomes, since confusion is considered an emotional state that can trigger learning because deep comprehension can happen in moments of confusion (D’Mello, Taylor, & Graesser, 2007; Graesser, et al., 2006; Kort & Reilly, 2002).  180  Boredom is an emotion that is negatively associated with learning (D’Mello, Taylor, & Graesser, 2007). Low levels of boredom (in the 3% to 7% range) have been reported by Baker, D’Mello, Rodrigo and Graesser (2010), McQuiggan, Robison and Lester (2008) and Rodrigo et al. (2012). However, in this study boredom was reported only 1.5% of the time by judges (1.7% by Judge 1 and 1.4% by Judge 2) and it is consistent with the observation of students playing the game. None of students complained of boredom in interviews or reported boredom in post-questionnaire. The negative emotion of shame was rarely observed by judges. Students did not report it; however, some students were upset when making mistakes or the monkey expressed the frustration animation.  Students did not describe their emotional states based on the set of emotions that I initially identified: they were not provided with a description of these emotions as these emotions were decided in parallel with experiments: the original set of emotions and the protocol for emotion analysis was developed in parallel with observations. When asked what emotions they experienced during the game, participants generally responded happiness, and to a lesser extent confidence, confusion, frustration and pride. The “happiness” state that students reported represents the spectrum of positive emotions of confidence, curiosity, engaged concentration, excitement, delight/pleasure, and pride.  Even if well-defined, clear and easy to follow, the emotion analysis process was challenging to judges. Issues of personality of participants and expressiveness of their face should be taken into consideration in future studies.  This study’s results demonstrate that there is a high variance in reporting agreement over different students. Although it is common knowledge that different people have different propensities for showing their emotions, this study indicates that these differences  181  can be quite substantial for some individuals, and may make it difficult to obtain reliable emotion information and subsequent accurate affective models.  Another finding is that the 20-second intervals used for emotion reporting often included more than one emotion. Studies so far have adopted the simplified approach of only considering one emotion per interval, but this may not necessarily be the most relevant for learning. Ignoring the other emotions too often may result in an inaccurate account of which emotions an educational game (or tutor) should be able to detect and respond to. Results from this research however, showed low inter-judge reliability on the number of observed emotions per interval, and few instances in which judges agreed on emotion type and number when more than one was present.  The method implemented for identifying, labeling and analyzing emotions was adequate for this study. The judges (Judge 2, Judge 3—the observer who declined to continue as a judge, and I) found the process clear and easy to follow, and found the emotion descriptions helpful. However, judges brought important issues of subtle emotions, and challenges in judging particular students. 5.3.2. Attitudes Towards the Game The participants in this study generally responded that they wanted to “have fun” indicating a good attitude before the game, and reported after game play happiness, fun and pleasure. Even if the participants’ perception was that the game is not more interesting than a video game, it is validating that it has great potential as a mathematics game. The large majority of students found the game to be enjoyable or interesting. Responses indicated that all students would like to play the game again, except one (Student 13) who was confident that a mathematics game would not interest her and does not matter the game mechanics.   182  Interesting results were obtained from this study with regard to the affective character of the game: the monkey. I started from the assumption that the monkey character would be appreciated by students. However, not all students liked the monkey and some found her annoying; few students even did not notice her. There are several reasons why the monkey was not appreciated: the implementation of the monkey character with a continuous presence and movement was distracting to some. Also, the monkey’s emotions did not always correspond to what the students felt or was noticed by observers. A better implementation of the monkey would improve the students’ reactions with respect to an emotional character. We observed that the students were proud to see the monkey happy but generally did not feel upset when the monkey was upset.  Generally students found the level of difficulty of the game to be about right; however, a few students found the game too difficult. All students except one (Student 13) indicated that they would like to see more quests. These results are a strong indication that this type of game can be well received by students. 5.3.3. Learning Based on the results of this study I rejected the null hypothesis of no difference between pretest and posttest. One student (Student 5) did not write the posttest because he was too tired, therefore I computed the pre-and posttest scores for 14 students. For these 14 students, there was a significant difference between pretest (M = 77.7%; SD = 9.3%) and posttest (M = 83.5%; SD = 8.7%), t (14) = 2.2; two-tailed p < 0.007) suggesting that interaction with Heroes of Math Island could affect learning. The students’ perceptions of mathematics learning gains were consistent with the pre- and posttest results: only one student (Student 13) responded neutral to the statement “I learned math when I played the  183  game”. All other students (93.3%) agreed or strongly agreed with the statement. However, Student 13 improved her score by 4.3% (from 78.3% in pretest to 82.6% in posttest). Even if not motivated, she was engaged (engaged concentration was reported for her 48.5% by Judge 1 and 65.2% by Judge 2). Judge 1 found her frustrated 6.1% and Judge 2 found her confused but not frustrated. Neither of the two judges found her bored; her video suggests a high level of engagement. Several statements in the post-questionnaire addressed the students’ perception of learning gains with respect to gameplay, task accomplished (“I learned math when I finished the quest”), game design (hints, harder questions), mistakes, and social norm (“I learned math when I helped the miners”). Data analyzed in Chapter Four indicate that the students felt that learning happened. Data suggest that the hints designed into the game require revision. Difficult questions were generally welcomed. Participants suggested that their curiosity in the subject matter increased. When they were asked to stop or repeat the easier portion of the quest (divisibility activity) all students refused.  In order to be effective, an educational game should provide individualized support and adaptive interaction (Kardan & Conati, 2013). Algorithms aimed at detecting when help is needed and the predictive algorithms responsible for game adaptation to level of difficulty and response given to students involve artificial intelligence aspects of student modeling, machine learning and adaptive interfaces, aspects that will be included in the future re-design of the game. 5.4. Recommendations for Future Research This study emphasized the importance of affect in the context of learning with an ALT, ITS or educational game. Consistent with the DBR paradigm, this study can be viewed  184  as an iterative stage of a longitudinal process. In a recent study by Steenbergen-Hu and Cooper (2013) in the November issue of the Journal of Educational Psychology, based on reports between 1997 and 2010, it was found that effects of ITSs on K-12 math students are greater when ITSs are used for less than one year than over the school year or longer. The effectiveness of ITSs for helping students drawn from the general population was greater than for helping low achievers (Steenbergen-Hu & Cooper, 2013). A longitudinal study employing the Heroes of the Math Island game would be beneficial in confirning or informing these results. These studies should be aligned with research into gaming literacies (e.g., Krug, 2007). Besides extending this study longitudinally, two general directions are suggested for future work. First, several design issues and heuristics were either confirmed or emerged from results and are worth exploring in future research. Second, the results of this study call for investigation of the set of emotions proposed.  5.4.1. Recommendations for Design Issues and Heuristics Emerging from Results  This study provided a detailed analysis of data collected from fifteen participants. The study took place in a laboratory and observers were present with participants. I recommend that this work be continued in a longitudinal study with gameplay over a time period of students’ free time and also in a more naturalistic school setting. The design heuristics included in this game (e.g., quest, monkey, island, castle, activities in a mine, metaphors related to prime numbers being rocks that do not break) were adequate to the age group and provided engagement and enjoyment. Determination to accomplish tasks and finish quests ought to be considered among heuristics of educational game design.  185  Participants were keen to provide feedback and advise how features in the game should be designed and implemented. It is recommended that a study be conducted on game design with a group of students advising on and testing game mechanics and features that would keep them engaged and motivated. Gender is important for various reasons in game deign. Even with its mathematics content, the interaction offered by narratives and quests, Heroes of Math Island can be designed to address hidden gender cues and explore cultural and environmental factors that affect gaming and learning. More research is needed on the role that affective or pedagogical agents play in reinforcing or contradicting gender norms.  5.4.2. Recommendations for Further Investigation of the Set of Emotions Proposed In the context of this investigation the most relevant emotions were:   boredom  confidence  confusion/hesitancy  delight/pleasure  disappointment/displeasure  engaged concentration  frustration    Boredom is a good measure of lack of interest. Surprise was not relevant to this study; however, elements of surprise might be useful to remove students from situations of boredom and should be investigated in context of adaptive emotional agents.  Confidence should be further investigated as it provides, together with engaged concentration, an important measurement of engagement, focus and motivation during gameplay.   186   Confusion/hesitancy is not an emotion damaging to learning; however proper interventions in the form of hints and examples should be employed to allow students to resolve confusion or moments of hesitation to progress in the game.   Delight/pleasure is a measure of enjoyment as well as engaged concentration and confidence. Excitement was not noticed much in emotion analysis; delight/pleasure is enough for reporting special instances of positive emotions of fun or pleasure during gameplay.   Disappointment/displeasure ought to be separated from frustration, as it represents different nuances of negative emotions that hinder learning. This should be further evaluated and responded to appropriately.  Engaged concentration is closely interrelated with confidence and curiosity. More research into game design and gameplay is needed on the expression of engaged concentration, confidence and curiosity. Students sometimes often looked engaged and tense, or even worried to judges. Further analysis is needed.  Frustration was reported to a lesser extent in this study. However, frustration can lead to a “vicious circle” resulting in boredom and a disengagement from learning. D’Mello, Taylor and Graesser (2007) provide important findings for further investigation of this issue. This set of emotions has an advantage of simplicity, was derived from observation and affect analysis, and with its overlap validates the set proposed by D’Mello and Graesser (boredom, confusion, delight, engaged concentration, frustration and surprise) (D’Mello, Taylor, & Graesser, 2007; Graesser, McDaniel, Chipman, Witherspoon, D'Mello, & Gholson, 2006). Commonly reported in the literature, pride is an important emotion, but  187  should be further studied and considered in relationship to intrinsic and extrinsic achievements and rewards in game design. Empirical research is needed on the interface of ALTs, ITSs, agents and game companions, such as the monkey character utilized in this research: what effect would a less intrusive, more useful companion have (offering advice, cues, hints, help)? Should affective and cognitive companions be one in the same? When participants were asked if the monkey should teach them, they responded that a pedagogical role should be given to a different character (e.g., a wiseman or wisewoman in this game). Chalfoun and Frasson’s (2012) and Jraidi, Chalfoun and Frasson’s (2012) design and research into affective and cognitive priming offer productive avenues for further research on semantic or symbolic cues, hints, primes, and scaffolds. It is challenging to develop methods for affect analysis at fine levels of granularity, but it is an endeavor recommended for future studies.   188  Bibliography Aleven, V. (2010). Rule-based cognitive modeling for intelligent tutoring systems. In R. Nkambou, J. Bourdeau, & R. Mizoguchi, Advances in intelligent tutoring systems (pp. 33-62). Berlin: Springer. Aleven, V., Beal, C. R., & Graesser, A. C. (2013). Introduction to the special issue on advanced learning technologies. Journal of Educational Psychology, 105(4), 929-931. Aleven, V., McLaren, B., M., Sewall, J., & Koedinger, K. R. (2009). A new paradigm for intelligent tutoring systems: Example-tracing tutors. International Journal of Artificial Intelligence in Education, 19(2), 105-154. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. Journal of the Learning Sciences,, 4(2), 167-207. Arnold, M. B. (1960). Emotion and personality: vol. 1. Psychological aspects. New York, NY: Columbia University Press. Retrieved April 1, 2013, from http://www.ctherapy.com/Product_Home_Pages/feelings.asp Arnone, M., Small, R., Chauncey, S., & McKenna, P. (2011). Curiosity, interest and engagement in technology-pervasive learning environments: A new research agenda. Education Tech Research, 59, 181-198. Astleitner, H. (2000). Designing emotionally sound instruction: The FEASP-approach. Instructional Science, 28, 169–198. Baker, R. S., D’Mello, S. K., Rodrigo, M. M., & Graesser, A. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive- affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4), 223-241. Balkenius, C., & Morén, J. (2001). Emotional learning: A computational model of the amygdala. Cybernetics and Systems, 32, 611–636. Barbalet, J. (1998). Emotion, social theory and social structure. Cambridge : Cambridge University Press. Bartneck, C. (2002). Integrating the OCC model of emotions in embodied characters. Workshop on virtual conversational characters: Applications, methods, and research challenges, (pp. 39–48). Melbourne. Retrieved March 1, 2011, from http://www.bartneck.de/publications/2002/integratingTheOCCModel/bartneckHF2002.pdf Bechara, A., & Damasio, A. R. (2005). The somatic marker hypothesis: A neural theory of economic decision., 52(2), . Games and Economic Behaviour, 52(2), 336–372. Boler, M. (1997). Disciplined emotions: Philosophies of educated feelings. Educational Theory, 47(2), 203-227. Boler, M. (1999). Feeling power: Emotion and education. New York: Routledge. Bondareva, D., Conati, C., Feyzi-Behnagh, R., Harley, J., Azevedo, R., & Boucher, F. (2013). Inferring learning from gaze data during interaction with an environment to support self-regulated learning. Artificial Intelligence in Education, 7926, 229-238. Bower, G. (1981). Mood and memory. American Psychologist, 36(2), 129-148.  189  Broad, C. D. (1971). Emotion and sentiment. In D. Cheney, Broad's Critical Essays in Moral Philosophy. London: Allen & Unwin. Cannon, W. B. (1927). Bodily changes in pain, hunger, fear and rage. New York and London: D.Appleton And Company. Retrieved June 1, 2011, from https://archive.org/details/bodilychangesinp029647mbp Chalfoun, P., & Frasson, C. (2012). Cognitive priming: Assessing the use of non-conscious perception to enhance learner's reasoning ability. The 11th International Conference On Intelligent Tutoring Systems (ITS 2012). Chania, Greece. Chaouachi, M., & Frasson, C. (2012). Mental worlkoad, engagement and emotions : An exploratory study for Intelligent Tutoring Systems. The 11th International Conference On Intelligent Tutoring Systems (ITS 2012), (pp. 65-71). Chania, Greece. Conati, C. (2002). Probabilistic assessment of user's emotions in educational games. Journal of Applied Artificial Intelligence, Special Issue on Merging Cognition and Affect in HCI, 16(7–8), 555–575. Conati, C., & Maclaren, H. (2009). Modeling user affect from causes and effects. In U. '09 (Ed.), Proceedings of the 17th International Conference on User Modeling, Adaptation, and Personalization (pp. 4-15). Berlin, Heidelberg: Springer-Verlag. Conati, C., & Manske, M. (2009). Adaptive feedback in an educational game for number factorization. Proceedings of the 14th International Conference on Artificial Intelligence in Education AIED 2009, (pp. 518-583). Brighton, UK. Conati, C., Jaques, N., & Muir, M. (2013). Understanding attention to adaptive hints in educational games: An eye-tracking study. International Journal of Artificial Intelligence in Education, 23(1-4), 136-161. Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. Journal of User Modeling and User-Adapted Interaction, 4(4), 253-278. Creative Therapy Associates. (n.d.). (n.d.). Retrieved June 1, 2010, from http://www.ctherapy.com/Product_Home_Pages/feelings.asp Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New-York: Harper-Row. D’Mello, S. K., Taylor, R., & Graesser, A. C. (2007). Monitoring affective trajectories during complex learning. 29th Annual Cognitive Science Soc., 203–208. Damasio, A. (1994). Descartes’s error: Emotion, reason and the human brain. New York: Avon Books. Davidson, D. (1980). Essays on actions and events. Oxford: Clarendon. Deigh, J. (1994). Cognitivism in the theory of emotion. Ethics, 104, 824-854. Derbali, L., Ghali, R., & Frasson, C. (2013). Assessing motivational strategies in Serious Games using Hidden Markov Models. FLAIRS 2013, The 26th International FLAIRS Conference. St. Pete Beach, Florida, USA: AAAI Press. Descartes, R. (1649). The passions of the soul. (S. H. Voss, Trans.) Indianapolis: Hackett. Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. Ekman, P. (1999). Basic emotions. In T. Dalgleish, & M. Power (Eds.), Handbook of Cognition and Emotion (pp. 45-60). New York: Wiley. Ekman, P. (2004). What we become emotional about. In A. S. Manstead, N. Fijda, A. Fischer, & C. U. Press (Ed.), The Amsterdam symposium (pp. 119–135). Cambridge.  190  Ekman, P., Friesen, W., & Simons, R. (1985). Is the startle reaction an emotion? Journal of Personality and Social Psychology, 49, 1416-1426. Ellis, H. C., & Ashbrook, P. W. (1987). Resource allocation model of the effects of depressed mood states. In K. Fiedler, & J. Forgas, Affect, cognition and social behaviour. Toronto: Hogrefe. Fellenz, W., Taylor, J., Cowie, R., Douglas-Cowie, E., Piat, F., Kollias, S., . . . Apolloni, B. (2000). On emotion recognition of faces and of speech using neural networks, fuzzy logic and the ASSESS system. Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, 2 IJCNN 2000, (pp. 93-98). Fleiss, J. (1981). Statistical methods for rates and proportions (2 ed.). New York: John Wiley. Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56, 218–226. Frijda, N. H. (2000). Emotion theory? Behavioral and Brain Sciences, 23(2), 199-200. Gay, L. R., Mills, G., & Airasian, P. (2006). Educational research. New York: Merrill Prentice Hall. Graesser, A., McDaniel, B., Chipman, P., Witherspoon, A., D'Mello, S., & Gholson, B. (2006). Detection of emotions during learning with AutoTutor. In I. R. (Ed.) (Ed.), Proceedingsof the 28th Annual Meetings of the Cognitive Science Society (pp. 285–290). NJ:Erlbaum: Mahwah. Gratch, J., & Marsella, S. (2004). A domain-independent framework for modeling emotion. Cognitive Systems Research, 5(4), 269-306. Gutica, M., & Conati, C. (2013). Student emotions with an edu-game: A detailed analysis. Proceedings of the 5th International Conference on Affective Computing and Intelligent Interaction (ACII 2013), (pp. 534–539). Geneva, Switzerland. Hascher, T. (2004). Wohlbefinden in der Schule [Well-being in school]. Münster: Waxmann. Hascher, T. (2010). Learning and emotion: Perspectives for theory and research. European Educational Research Journal, 9(1), 13-28. Hu, H., Real, E., Takamika, K., Kang, M.-G., Ledoux, J., Huganir, R., & Mallnow, R. (2007). Emotion enhances learning via norepinephrine regulation of AMPA-receptor trafficking. Cell, 131(1), 160-173. Hume, D. (1739). A treatise of human nature. (L. A. Bigge, Ed.) Oxford: Clarendon. Ingleton, C. (2000). Emotion in learning - a neglected dynamic. (R. James, J. Milton, & R. Gabb, Eds.) Research and Development in Higher Education, 22, 86-99. Isen, A. M., & Reeve, J. (2005). The influence of positive affect on intrinsic and extrinsic motivation: Facilitating enjoyment of play, responsible work behavior, and self-control. Motivation and Emotion, 29(4), 297-325. Isen, A. M., Daubman, K., & Nowicki, G. (1987). Positive affect facilitates creative problem solving. Journal of Personality and Social Psychology, 56(6), 1122-1131. James, W. (1884). What is an emotion? Mind, 9, 188-205. Johnson, W. L., Rickel, J., & Lester, J. C. (2000). Animated pedagogical agents: Face-to-face interaction on interactive learning environments. International Journal of Artificial Intelligence in Education, 11, 47-78. Joseph, D. (2004). The practice of design-based research: Uncovering the interplay between design, research, and the real-world context. Educational Psychologist, 39(4), 235-242.  191  Jraidi, I., Chalfoun, P., & Frasson, C. (2012). Implicit strategies for Intelligent Tutoring Systems. The 11th International Conference On Intelligent Tutoring Systems (ITS 2012), (pp. 1-10). Chania, Greece. Jraidi, I., Chaouachi, M., & Frasson, C. (2013). A dynamic multimodal approach for assessing learner's interaction experience. 15th ACM International Conference on Multimodal Interaction (ICMI2013). Sydney, Australia. Kardan , S., & Conati, C. (2013). Comparing and combining eye gaze and interface actions for determining user learning with an interactive simulation. UMAP, 215-227. Klein, J., Moon, Y., & Picard, R. W. (2002). This Computer Responds to User Frustration. Interacting with Computers, 14, 119-140. Korb, K., & Nicholson, A. E. (2000). The essential roles of emotion in cognitive architecture. Behavioral and Brain Sciences, 23(2), 205-206. Kort, B., & Reilly, R. (2002). Analytical models of emotions, learning and relationships: Towards an affect-sensitive cognitive machine. Conference on Virtual Worlds and Simulation (VWSim2002), (pp. 1–15). Retrieved October 18, 2011, from http://affect.media.mit.edu/projectpages/lc/vworlds.pdf Krug, D. (2007). Virtual education: Cognition, media, and digital learning environments, Part 1. In C. Montgomerie, & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 2554-2559). Chesapeake, VA: AACE. Landis, J. R., & Koch , G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. Lange, C. G., & James, W. (1922). The emotions (Vol. 1). Baltimore: Williams & Wilkins Company. Retrieved June 1, 2011, from https://archive.org/details/emotionsvolumei007644mbp LaRock, E., & Kafetsios, K. (2004). Aristotelian affinities with contemporary emotion research. Theory and Psychology, 15(5), 15-25. Lazarus, R. S. (1982). Thoughts on the relation between emotion and cognition. American Psychologist, 37(9), 1019-1024. Lazarus, R. S. (1984). On the primacy of cognition. American Psychologist, 39, 124-129. Lazarus, R. S. (1991). Emotion and adaptation. New York, NY : Oxford University Press. LeDoux, J. E. (1995). Emotion: Clues from the brain. Annual Review of Psychology, 46, 209-235. LeDoux, J. E. (1996). The emotional brain: The mysterious underpinnings of emotional life. New York: Simon & Schuster. LeDoux, J. E. (2007). The amygdala. Current Biology, 17(20). Retrieved August 17, 2011, from http://www.sciencedirect.com/science/article/pii/S0960982207017794 Lerner, J. S., & Keltner, D. (2000). Beyond valence: Toward a model of emotion-specific influences on judgment and choice. Cognition and Emotion, 14, 473–493. Linehan, C., Kirman, B., Lawson, S., & Chan, G. (2011). Practical, appropriate, empirically-validated guidelines for designing educational games. Proceedings of the 2011 annual conference on Human Factors in Computing Systems (CHI '11), (pp. 1979-1988). Vancouver. Retrieved June 6, 2011, from http://mcs.mines.edu/Courses/csci422/PAPERS/p1979-linehan-edgames.pdf Locke, J. (1695). An essay concerning human understanding. (P. H. Nidditch, Ed.) Oxford: Claredon.  192  Marsella, S., & Gratch, J. (2003). Modeling coping behaviors in virtual humans: Don't worry, be happy. Second International Joint Conference on Autonomous Agents and Multi-agent Systems, Melbourne, Australia. Retrieved June 1, 2010, from http://people.ict.usc.edu/~gratch/MarsellaGratch-aamas03.pdf Marsella, S., Gratch, J., & Petta, P. (2010). Computational models of emotion. In K. Scherer, T. Bänziger, & E. Roesch, A blueprint for an affective computing: A sourcebook. Oxford: Oxford University Press. McDougall, W. (1908). An introduction to social psychology. London: Methuen. McQuiggan, S. W., Robison, J. L., & Lester, J. C. (2010). Affective transitions in narrative-centered learning environments. Educational Technology & Society, 13(1), 40–53. Miles, M. B., & Huberman , A. (1984). Qualitative data analysis. Thousand Oaks: CA: Sage Publications. Ministry of Education Curriculum. (2011). Retrieved November 20, 2011, from https://www.bced.gov.bc.ca/irp/course.php?lang=en&subject=Mathematics&course=Mathematics_K_to_7&year=2007 Minsky, M. (1986). The society of mind. New York: Simon and Schuster. Mowrer, O. H. (1960). Learning theory and behavior. New York: Wiley. Muldner, K., & Conati, C. (2007). Evaluating a decision-theoretic approach to tailored example selection . Proceedings of IJCAI 2007, 20th international joint conference in artificial intelligence , (pp. 483-488). Nasoz, F., Alvarez, K., Lisetti, C., & Finkelstein, N. (2004). Emotion recognition from physiological signals for presence technologies. Cognition, Technology and Work, 6(1), 4–14. Nicholson, L. (1997). Emotion in postmodern public spaces. In G. Hoffmann, & A. Hornung, Emotion in Postmodernism (pp. 1-23). Heidelberg: Universitatsverlag C. Norman, D. A. (2004). Emotional design: Why we love (or hate) everyday things. New York: Basic Books. O’Regan, K. (2003). Emotion and e-learning. Journal of Asynchronous Learning Network, 7(3), 78-92. Oaksford, M., Morris, F., Grainger, B., & Williams, J. M. (1996). Mood, reasoning, and central executive processes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 477-493. Oatley, K., & Johnson-Laird, P. (1987). Towards a cognitive theory of emotion. Cognition and Emotion, 1, 29-50. Ortony, A., & Turner, T. (1990). What’s basic about basic emotions? Psychological Review, 97, 315-331. Ortony, A., Clore, G., & Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge Univ. Press. Panksepp, J. (1982). Toward a general psychobiological theory of emotions. The Behavioral and Brain Sciences, 5, 407-467. Pantic, M., & Rothkrantz, L. (2003). Toward an affect-sensitive multimodal human-computer interaction. Proceedings of IEEE, 91(9), 1370-1390. Petrina, S. (2007). Advanced teaching methods for the technology classroom. Hershey, PA: Information Science Publishing. Picard, R. W. (1997). Affective computing. Cambridge, MA: MIT Press. Picard, R. W. (2000). Toward agents that recognize emotion. Vivek, 13(1), 3-13.  193  Picard, R. W., & Klein, J. (2002). Computers that recognize and respond to user emotion: theoretical and practical implications. Interacting with Computers, 14, 141–169. Picard, R. W., Kort, B., & Reilly, R. (2003). Exploring the role of emotion in propelling the SMET learning process. MIT. Retrieved October 18, 2011, from http://www.media.mit.edu/affect/AC_research/lc/nsf1.doc Plutchik, R. (1984). Emotions: A general psychoevolutionary theory. In K. Scherer, & P. Ekman, Approaches to Emotion (pp. 197–219). Hillsdale, NJ: Erlbaum. Proust, M. (1934). Remembrance of things past. (C. K. Moncrieff, Trans.) New York: Random House. Rodrigo, M. M., de Baker, R. S., Agapito, J., Nabos, J., Repalam, M. C., Reyes, S. S., & Pedro, M. O. (2012). The effects of an interactive software agent on student affective dynamics while using an Intelligent Tutoring System. Affective Computing, 3(2), 224–236. Rodrigo, M. M., Rebolledo-Mendez , G., Baker, R., Boulay , B., Sugay, J., Lim, S., . . . Luckin, R. (2008). The effects of motivational modeling on affect in an Intelligent Tutoring System. Proceedings of International Conference on Computers in Education, (pp. 49–56). Rolls, E. T. (1995). A theory of emotion and consciousness and its application to understanding the neural basis of emotion. In M. Gazzaniga, The cognitive neurosciences (pp. 1091–1106). Cambridge, MA: MIT Press. Rolls, E. T. (2000). Precis of the brain and emotion. Behavioral and Brain Sciences, 23, 177–233. Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172. Russell, J. A., & Feldman Barrett, L. (1999). Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology, 76, 805–819. Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Cambridge MA: MIT Press. Salzberger-Wittenberg, I., Henry, G., & Osborne, E. (1983). The emotional experience of learning and teaching. London: Routledge and Kegan Paul. Sandoval, W. A., & Bell, P. L. (2004). Design-based research methods for studying learning in context: Introduction. Educational Psychologist, 39(4), 199-201. Scheff, T. J. (1997). Emotions, the social bond and human reality . Cambridge: University of Cambridge Press. Schell, J. (2008). The art of game design. San Francisco: Morgan Kaufmann Publishers. Scherer, K. R. (2001a). The future of emotion: Foreword. Social Science Information, 40(1), 5-9. Scherer, K. R. (2001b). Appraisal considered as a process of multilevel sequential checking. In K. R. Scherer, A. Schorr, & T. Johnstone, Appraisal processes in emotion: theory, methods, research (pp. 92-120). New York, NY: Cambridge University Press. Scherer, K. R. (2009). Emotions are emergent processes: They require a dynamic computational architecture. Philosophical Transactions of the Royal Society, Series B, 364(1535), 3459–3474.  194  Scherer, K. R. (2010). Emotion and emotional competence: Conceptual and theoretical issues for modelling agents. In K. R. Scherer, T. Banziger, & E. Roesch, A Blueprint for Affective Computing. New York: Oxford University Press. Sebe, N., Lew, M., Cohen, I., Garg, A., & Huang, T. S. (2002). Emotion recognition using a Cauchy naive Bayes classifier. Proceedings of ICPR, 1, pp. 17–20. Seibert, P. S., & Ellis, H. C. (1991). Irrelevant thoughts, emotional mood states, and cognitive task performance. Memory and Cognition, 19, 507-513. Shen, L., Wang, M., & Shen, R. (2009). Affective e-learning: Using “emotional” data to improve learning in pervasive learning environment. Education Technology & Society, 12(2), 176–189. Shute, V. J., & Psotka, J. (1996). Intelligent tutoring systems: Past, present and future. In D. Jonassen, Handbook of research on educational communications and technology (pp. 570-600). New York: Scholastic Publications. Spires, H., Rowe, J., Mott, B., & Lester, J. (2011). Problem solving and game-based learning: Effects of middle grade students' hypothesis testing strategies on learning outcomes. Journal of Educational Computing Research, 44(4), 453-472. Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of Intelligent Tutoring Systems on K–12 students' mathematical learning. Journal of Educational Psychology. Advance Online Publication., 105(4), 970-987. Sylwester, R. (1994). How emotions affect learning. Educational Leadership, 52(2), 60-65. Tomkins, S. S. (1984). Affect theory. In K. R. Scherer, & P. Ekman, Approaches to emotion (pp. 163-195). Hillsdale, NJ: Erlbaum. Trabelsi, A., & Frasson, C. (2010). The emotional machine: A machine learning approach to online prediction of user’s emotion and intensity. ICALT 2010: 10th IEEE International Conference on Advanced Learning Technologies, (pp. 613 - 617). Sousse, Tunisia. Um, E., Song, H., & Plass, J. (2007). The effect of positive emotions on multimedia learning. In C. Montgomerie, & J. Seale, Proceedings of world conference on educational multimedia, hypermedia and telecommunications 2007 (pp. 4176–4185). VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227-265. Watson, J. B. (1930). Behaviorism. Chicago: University of Chicago. Weiss, R. (2000). Emotion and learning. Training and Development, 54(11), 44-48. Woolf, B., Burleson, W., Arroyo, I., Dragon, T., Cooper, D., & Picard, R. (2009). Affect-aware tutors: Recognizing and responding to student affect. International Journal of Learning Technology, 4(3-4), 129-163. Zajonc, R. B. (1980). Feeling and thinking: Preferences need no inferences. American Psychologist, 35(2), 151-175. Zajonc, R. B. (1984). On the primacy of affect. American Psychologist, 39(2), 117-123. Zhu, Y., De Silva, L., & Ko, C. (2002). Using moment invariants and HMM in facial expression recognition. Pattern Recognition Letters, 23(1-3), 83-91.     195  Appendix 1: Recruitment        Heroes of Math Island Grade 6 and 7 children are invited to participate in a research study that involves the design and implementation of a mathematical game. We are interested in the cognitive and emotional demands of this technology. This study addresses learning over time. Total time necessary for children to participate in the study is approximately four to six hours divided into 2 or 3 sessions of two hours.  If you want your child to participate in this study, please contact Mirela Gutica no later than August 31, 2012.   A New Fun Game to Improve your Math Skills  need your help!  University of British Columbia  196       Results of this research will be used in graduate theses and we intend to publish the findings of the study in professional journals and report them at conferences.  At no time will the actual identity of the participants be disclosed.  Participants will be assigned pseudonyms and these only will be used in publications.  We will maintain the strictest levels of protocols towards any and all information revealed in confidence.  Agreement on your part in no way obligates your child to remain a part of the study.  Participation is voluntary, and you may choose to withdraw from the study at any time.  Study The study is entitled How We Learn (Media & Technology Across the Lifespan), and is sponsored by the Faculty of Education at the University of British Columbia and will be conducted by Drs. Stephen Petrina and Franc Feng. Part of this study, we design and implementation a mathematical game: Heroes of Math Island. The study addresses learning and will be primarily based on observation of your child's interaction with the game. Your child will write a pre- and a post-math test, a post questionnaire, and simple questions about game interaction may also be asked. The interaction with the game will take 30 – 40 minutes and will be videotaped. The study will take place at UBC or BCIT (where is more convenient for you).        British Columbia Institute of Technology University of British Columbia LETTER OF INVITATION AND INFORMATION Heroes of Math Island Game and Scenes Contact Your grade 6 and 7 child is invited to participate in a research study of how children interact with an educational game.  We are interested in the cognitive and emotional demands of this technology. This study addresses learning over time. Total time necessary for children to participate in the study is approximately four to six hours divided into 2 or 3 sessions of two hours. The aim of this letter is twofold.  First, it describes the purpose and method of the research study.  Second, it requests that if you agree, you should do it in writing, to allow your child to participate in the study. If you agree, Assent Forms will be e-mailed to you.    If you want your child to participate in this study, please contact: Mirela Gutica no later than August 31, 2012. If you have any questions or desire further information with respect to this study, you may contact Dr. Stephen Petrina. If you have any concerns about your child’s treatment or rights as a research subject, you may contact the Research Subject Information line in the UBC Office of Research Services.   Privacy Dear Parent,  197  Appendix 2: Study Protocol Date: Experiment Number: Student name:  Grade:  Mother’s name:  E-mail address:  Time: 1:00pm Place:   Researchers:   Condition:  Heroes of Math Island Monkey’s Emotions: happy, confident, sad, frustrated, neutral Mine quest Divisibility, Prime numbers, Decomposition  Time for interaction: aim for 30 – 40 min Activities: tutorial, pretest, game, posttest, questionnaire, interview Papers: consent forms, questionnaire, pretest, posttest, questionnaire and interview guide Other: gift for participation 10$ Chapters Video recording: face and computer screen  Protocol: Before experiment: Invite parent E-mail consent forms (general, video)  Book video equipment (2) Print papers: consent forms, questionnaire, observer’s paper Prepare video  When student is here: Welcome the student Check for consent forms Give a description of the game and the tutorial see Appendix 3: Tutorial.   During game: Videotape Observe the play Take notes  Interview protocol:  Ask questions regarding the game experience  Be un-structured   After the experiment Thank the kid Give gift  After the kid leaves Write a detailed report  198  Appendix 3: Tutorial    Divisibility: meaning and examples Rules of divisibility by 2, 3, 5, and 10 Definition of prime and composite numbers  Determining if a number is prime: small numbers (e.g., 13, 19, 29) and large numbers (e.g., 113, 197, 239) Definition of prime factors  Example of prime factor de-composition: 8 = 2 x 2 x 2 Explain why 8 = 2 x 4 is not prime factor decomposition      199  Appendix 4: Pretest     Questions and Answers For Pretest  March 12, 2012    200     Mine Quest  Activity 1 Divisibility  1. Is 23,552 divisible by 5?  Answer: T/F Correct: F   2. 2?,602 is divisible by 3 if ? is:   Answer: 1 2 3 4 5 Correct: 2   3. 345, 78? is divisible by 2 if ? is:   Answer: 0 1 2 3 4 Correct: 0, 2 and 4   4. Is 678,777 divisible by 2?  Answer: T/F Correct: F  5. Is 8,799,110 divisible by 2?  Answer: T/F Correct: T   6. Is 900,120 divisible by 10?  Answer: T/F Correct: T  7. Is 901,120 divisible by 3?  Answer: T/F Correct: F  8. Type a single digit that makes this statement true.  115,43  is divisible by 2.   Correct: 0, 2, 4, 6, 8  201    9. Type a single digit that makes this statement true.  234,88  is divisible by 5.   Correct: 0, 5   10. Type a single digit that makes this statement true.  678,86  is divisible by 10.   Correct: 0   11. Is 32 divisible by 3?  Answer T/F Correct: F   12. Is 49 divisible by 7?  Answer T/F Correct: T  Activity 2 Prime Numbers  13. Which number is prime?  15, 2, 4, 27, none  Answer: 2  Which number is prime?  14. 1, 4, 21, 39, none  Answer: none  15. Which number is prime?  1, -4, -5, -6, none  Answer: none    202  16. Which of the following numbers are prime?  51, 13, 6, 17, none  Answer: 13, 17   17. Which of the following numbers are prime?  37, 11, 63, 47  Answer: 37, 11, 47   Activity 3 Decomposition  18. Which number is a prime factor of 20?  3, 10, 15, 21, none  Answer: none   19. Which number is a prime factor of 12?  2, 4, 9, 10, none  Answer: 2  20. Which number a prime factor of 27?  2, 6, 9, 12, none  Answer: none  21. Which number a prime factor of 27?  3, 7, 9, 12, none  Answer: 3      203  22. What is the prime factorization of 18?  2 x 2 x 2 2 x 2 x 3 2 x 9 2 x 3 x 3 none  Answer: 2 x 3 x 3   23. What is the prime factorization of 12?  2 x 2 x 2 2 x 2 x 3 2 x 6 2 x 3 x 3 none  Answer: 2 x 2 x 3        204  Appendix 5: Posttest    Questions and Answers For Post- Test  March 12, 2012    205    Mine Quest  Activity 1 Divisibility   1. Is 23,550 divisible by 5?  Answer: T/F Correct: T   2. 25?,602 is divisible by 3 if ? is:   Answer: 0 1 2 3 4 Correct: 0   3. 77, 12? is divisible by 2 if ? is:   Answer: 1 2 3 4 5 Correct: 2 and 4   4. 882, 22? is divisible by 2 if ? is:   Answer: 5 6 7 8 0 Correct: 6, 8 and 0   5. Is 6,769,260 divisible by 2?  Answer: T/F Correct: T   6. Is 900,121 divisible by 10?  Answer: T/F Correct: F   7. Is 85,110 divisible by 3?  Answer: T/F Correct: T     206   8. Type a single digit that makes this statement true.  445,17  is divisible by 2.   Correct: 0, 2, 4, 6, 8   9.  Type a single digit that makes this statement true.  113,65  is divisible by 5.   Correct: 0, 5   10. Type a single digit that makes this statement true.  759,28  is divisible by 10.   Correct: 0   11. Is 35 divisible by 3?  Answer T/F Correct: F   12. Is 63 divisible by 9?  Answer T/F Correct: T  Activity 2 Prime Numbers  13. Which number is prime?  21, 2, 4, 27, none  Answer: 2  14. Which number is prime?  -3, 7, 34, 49, none  Answer: 7  207   15. Which number is prime?  -1, -3, 1, -2, none  Answer: none    16. Which of the following numbers are prime?  91, 1, 67, 23  Answer: 67, 23  17. Which of the following numbers are prime?  2, 93, 29, 41  Answer: 2, 29, 41   Activity 3 Decomposition   18. Which number is a prime factor of 64?  3, 12, 32, 30, none  Answer: none  19. Which number a prime factor of 18?  3, 6, 9, 12, none  Answer: 3  20. Which number a prime factor of 24?  2, 6, 9, 12, none  Answer: 2    208  21. Which number a prime factor of 24?  4, 6, 9, 12, none  Answer: none  22. What is the prime factorization of 28?  2 x 14 2 x 2 x 7 4 x 7 2 x 3 x 7 none  Answer: 2 x 2 x 7    23. What is the prime factorization of 36?  2 x 2 x 3 2 x 2 x 3 x 3 6 x 6 2 x 3 x 6 none  Answer: 2 x 2 x 3 x 3      209  Appendix 6: Post Questionnaire   Student #:       Gender: Boy/Girl  Do you agree with the following statements about your game experience?    Disagree  Neutral  Agree 1 I wanted to have fun when I was playing the game. 1 2 3 4 5 2  1 2 3 4 5 3 The monkey was annoying. 1 2 3 4 5 4       5 The level of difficulty was right. 1 2 3 4 5 6 I wanted to learn when I played the game. 1 2 3 4 5 7 I felt upset when I did not answer correct.  1 2 3 4 5 8 I became curious about math (divisibility/prime numbers/factorization) by playing the game. 1 2 3 4 5 9 The monkey reacted correctly to my emotions.  1 2 3 4 5 10 I didn’t care whether I had fun or not.  1 2 3 4 5 11 The game was too easy or too hard for me. 1 2 3 4 5 12 I was proud to see the monkey happy.  1 2 3 4 5 13 I wanted to learn math by playing the game.  1 2 3 4 5 14       15 I felt shame when I made mistakes and the monkey was sad. 1 2 3 4 5 16 I didn’t want to think about math (divisibility / prime numbers / decomposition) when I was playing the game. 1 2 3 4 5 17 I didn’t mind to see that the monkey was sad.  1 2 3 4 5  210   Do you agree with the following statements about your game experience?   NO Neutral   YES   Didn’t Happen 1 I had fun when I chose my avatar. 1 2 3 4 5  N/A 2 I had fun when the monkey was happy. 1 2 3 4 5  N/A 3 I had fun when the monkey was sad. 1 2 3 4 5  N/A 4 I had fun when the monkey was confident. 1 2 3 4 5  N/A 5 I had fun when the monkey was frustrated. 1 2 3 4 5  N/A 6        N/A 7 I had fun when I finished the quest. 1 2 3 4 5  N/A 8 I had fun when I responded to math questions before play. 1 2 3 4 5  N/A 9 I had fun when I responded to math questions after play. 1 2 3 4 5  N/A 10  1 2 3 4 5  N/A 11 I had fun when the rocks turned into gold. 1 2 3 4 5  N/A 12 I had fun when prime number rocks were removed.  1 2 3 4 5  N/A 13 I had fun when I got hints. 1 2 3 4 5  N/A 14 I had fun when I got examples. 1 2 3 4 5  N/A 15  1 2 3 4 5  N/A 16 I had fun when I visited the castle. 1 2 3 4 5  N/A  Please tell us what emotions you experienced when you played the game:           Please tell us any other times you did not have fun         211   Do you agree with the following statements about your game experience?    NO  Neutral  YES  Didn’t Happen 1 I learned math when I played the game. 1 2 3 4 5  N/A 2 I learned math when I made mistakes. 1 2 3 4 5  N/A 3 I learned math when I got hints. 1 2 3 4 5  N/A 4 I learned math when I got examples. 1 2 3 4 5  N/A 5         6 I learned math when I helped the miners. 1 2 3 4 5  N/A 7         8         9 I learned math when I finished the quest. 1 2 3 4 5  N/A 10 I learned math when I had harder questions. 1 2 3 4 5  N/A 11         12         13          I was already good at divisibility before I played the game. 1 2 3 4 5 I was already good at prime numbers before I played the game. 1 2 3 4 5 I was already good at decomposition before I played the game. 1 2 3 4 5  I also learned math when:     I was stopped from learning math by:         212  Do you agree with the following statements about your game experience?    NO  Neutral  YES  Didn’t Happen 1 I played before video games. 1 2 3 4 5  N/A 2 I like playing video games. 1 2 3 4 5  N/A 3 I played before math games. 1 2 3 4 5  N/A 4 I like playing math games. 1 2 3 4 5  N/A 5 This game was boring. 1 2 3 4 5  N/A 6 This game was more interesting than video games that I played. 1 2 3 4 5  N/A 7         8 This game was more interesting than math games that I played. 1 2 3 4 5  N/A 9         10 I would like to play the game again 1 2 3 4 5  N/A 11 I believe that I can learn math better by playing this game. 1 2 3 4 5  N/A 12 I would like to see more quests. 1 2 3 4 5  N/A 13 I like math 1 2 3 4 5  N/A    Please tell us what you would like to see improved in this game:       213  Appendix 7: Interview Guide Students  Interview Questions  1. Did you notice the monkey? 2. Why was the monkey annoying? 3. How did you feel when the monkey reacted? 4. What did you like about the game? 5. How did you feel playing this game? 6. Would you study with this game? 7. Would you like to play again? 8. What improvements do you suggest?     214  Appendix 8: Observer Guide     1. Boredom – behaviors included slouching, resting the chin on his/her palm; statements such as “Can we do something else?” or “This is boring!”  2. Delight – similar with happiness; behaviors included clapping hands; laughing with pleasure; smiling; statements such as, “Yes!” or “I got it!”  3. Confusion – behaviors included scratching his/her head, repeatedly looking at the same interface elements; waiting a long time without knowing what to do next, consulting with a fellow student or the teacher; looking at another student’s work to determine what to do next; statements like, “I’m confused!” or “Why didn’t it work?”  4. Confidence - behaviours include solving problems fast, without hesitation, statements like “I know this” or “this is easy”; searching for help when needing. Body language of confidence.   5. Frustration – behaviors included banging on the keyboard or mouse; pulling his/her hair; deep sighing; statements such as, “What’s going on?!”  6. Pride - behaviors included smiling broadly, satisfied sense of attachment toward his/hers choices and actions; having a high opinion of oneself; statements like “I am good at this”  7. Shame / embarrassment – behaviors included blushing, downward cast eyes, slack posture, and lowered head, statements as “I should have known this”  8. Engaged concentration – behaviors included immersion, focus, and concentration on the system, with the appearance of positive engagement (as opposed to frustration); leaning towards the computer; mouthing solutions; pointing to parts of screen  9. Flow – behaviors included situations when the student did not appear to be displaying any of the other affective states or when the student’s affect could not be determined for certain  10.  Surprise – behaviors included sudden jerking or gasping; statement such as “Huh?” or “Oh, no!” Adapted and extended from Baker, D'Mello, Rodrigo and Graesser (2010).  215  Appendix 9: Protocol for Observing Affective States     Affective states will be recorded in an EXCEL spreadsheet.   Affective states will be recorded for each video portion of 20 seconds. The observer watches the video and stops it every 20 seconds to record what happened.   Several affective states can be reported (example: during the video portion 00:01:00 – 00:01:20 the student was engaged and surprised).  Intensity of emotion will not be recorded.   When an emotion is identified, report it as a ‘1’ on the spreadsheet.  Complete the observation on paper and transfer the data to the spreadsheet afterwards.  The observer can watch the same video sequence several times if he/she is concerned that the observed affective states were not correctly recorded.   The first entry in the EXCEL document is for the portion of video from 0:00:00 to 0:00:20.  If a student has 2 videos, the observer will use 2 different EXCEL files and should indicate the video number in the header and in the file name (remember that the second video starts also from time 0:00:00).   Naming conventions: the name of the EXCEL document should be: Observation_Study1_Student1_Video1_Observer’sName   Observers should fill the header of the EXCEL spreadsheet:     Student   Video    Observer   Date   If other affective states are observed, the observer should use the “Other” column (e.g., if a student shows less common affective states like “insight/realization”, “angry”, “fear”, “despair”, “burden”, “anxiety”, etc.)  In situations when you cannot see the student’s face, the student is taking a break, or if there are camera problems then report ‘N/A’.  Report any issues at the bottom of the EXCEL document. Note: Do not over-report: observers do not have to feel that they should report all affective states indicated in the spreadsheet: for example some affective states might not be relevant to this particular study. The observed affective states will be: Boredom, Shame, Frustration, Confusion/Hesitation, Displeasure /Disappointment, Surprise, Neutral, Curiosity, Engaged Concentration, Pleasure/Delight, Excitement, Confidence, Pride, Other  Observer’s guide for affective states: 1. Boredom – behaviors include slouching, resting the chin on his/her palm; statements such as “Can we do something else?” or “This is boring!” The student stopped playing the game, is leaning forward very far, sitting back in the chair.  216  2. Shame (or embarrassment) – behaviors include blushing, casting eyes  downward, slack posture, lowered head, statements such as “I should have known this”, tension, and appearance of feeling of “being put on the spot”. 3. Frustration – behaviors include banging on the keyboard or mouse; pulling his/her hair; deep sighing; statements such as, “What’s going on?!” 4. Confusion/Hesitation – behaviors include scratching his/her head, repeatedly looking at the same game elements; waiting a long time without knowing what to do next, consulting with the observers on what to do next or asking for help because he/she doesn’t know what to do; statements like, “I’m confused!” or “Why didn’t it work?”. The situation is novel and incomprehensible. Hesitation: behaviors indicate that the student is unsure; uncertain; not definite or positive; tentative.  5. Displeasure/Disappointment – behaviors include crying, the student is less outspoken, and less energetic. The student is sad, lowers his/her head and is looking down. 6. Surprise – behaviors include sudden jerking or gasping; statements such as “Huh?” or “Oh, no!” and widening of the eyes. 7. Neutral – report neutral in situations when the student did not appear to be displaying any affective states or when the student’s affective state could not be determined for certain. 8. Curious – behaviors include inquisitive comportment such as exploration and investigation. The situation is novel and comprehensible.  9. Engaged concentration – behaviors include immersion, focus, and concentration on the system, with the appearance of positive engagement (as opposed to frustration); leaning towards the computer; mouthing solutions; pointing to parts of screen. If the student is not playing the game and talks to the observers, report listening. 10. Delight / Pleasure –behaviors include laughing with pleasure and smiling; also smiling with relief. 11. Excitement – similar behaviors to those for delight/pleasure but of a higher intensity; behaviors indicate great pleasure, enthusiasm and eagerness, clapping hands and statements such as “Yes!” or “I got it!” 12. Confidence - behaviors include solving problems quickly, without hesitation, statements such as “I know this!” or “this is easy”; searching for help when needing. Body language of confidence: sitting tall, without slouching or fidgeting. Asking questions about the subject matter and knowing what to ask.  13. Pride - behaviors include smiling broadly, and the appearance of a satisfied sense of attachment toward his/hers choices and actions; having a high opinion of oneself; statements like “I am good at this”. Other states: 1. Anxiety –behaviors indicate that the student is nervous, unease, or concerned (anxiety is correlated with an event of uncertain outcome). Anxiety is different from fear: fear is a reaction to a present danger or threat, while anxiety is more a reaction intended to help the subject cope with future possible unpleasant events.   217  2. Anger – behaviors indicate aggression; the face is flushed, the brow muscles move inward and downward, fixing a hard stare on the target of the anger. 3. Insight/Realization – behaviors indicate a moment of realization, an “Ah ha!” moment, statements such as “Now I understand!” and backward head movement accompanied by smile/laughter. 4. Listening – behaviors include listening to what the observers are saying. Still engaged but, outside the game.  The observer guide is adapted and extended from Baker, D'Mello, Rodrigo and Graesser (2010).     218  Appendix 10: Instrument  Instrument used for affect analysis:      219  Appendix 11: Interview Guide Judges  1. How did you find the process of judging emotions? 2. To what extent were you comfortable judging emotions? 3. Did you find judging emotions difficult? Why or why not? 4. Was one of the students easier to judge than others? Why or why not? 5. Was an emotion easier to identify than others? What emotion? Why or why not? 6. What was the main affective state/emotion that you observed in students? 7. It was ok to report no emotion (neutral). Did you find many instances of neutral?  8. What about engaged concentration? Did you find many instances of that? 9. Can you elaborate more with respect to confusion, engaged concentration and confidence? 10. Can you elaborate more with respect to boredom and frustration? Do you believe that students learn during experiments? 11. Many students reported that they were happy during the game play; however, we did not find too many instances of pleasure? What is your opinion in this regard?  12. Did you find that reporting several emotions per 20 seconds interval was a good idea? Would have been a better idea to report only one emotion per interval? 13. Did you find the 20sec interval adequate? 14. How did you find the process of judging emotions in terms of task? Was it clear? To what extent was the system of labeling emotions (the forms) easy to follow and understand? To what extent was the form easy to use?  220  15. We run some group training sessions; were the training sessions helpful? Were you comfortable during the group training sessions? Were you nervous? Did you also have difficulties then or only home? 16. Can you estimate how long it took you to judge (i.e., a 30 min video) double, triple time? 17. Deciding what emotions and how to proceed was team work. What would you do differently with respect to this study?  18. To what extent do you believe that observing emotions is relevant to game design? 19. Remember that we observed: 12 emotions (plus neutral): boredom, confidence, confusion/hesitancy, curiosity, delight/pleasure, disappointment/displeasure, engaged concentration, excitement, frustration, pride, shame and surprise. What emotions were the most prominent during the game play? What was your overall perception?       221  Appendix 12: Sample of Raw Data Analysis: Aggregated Analysis    [DataSet1] C:\Thesis 2012\Data Analysis\SPSS\Students Updated March 31\SPSS Files Students\StudentTotal.sav    Case Processing Summary  Cases Valid Missing Total N Percent N Percent N Percent Emotion * Emotion 1082 78.1% 303 21.9% 1385 100.0%    Emotion * Emotion Crosstabulation Count  Emotion Total Bo Sh Fr Co/Hes Dis Su Ne Cu Eng Pl Confid Proud Emotion Bo 1 0 0 2 0 0 0 0 2 0 2 0 7 Fr 0 0 9 6 0 0 0 0 10 0 0 0 25 Co/Hes 1 0 0 116 0 1 0 0 34 0 19 0 171 Dis 3 0 0 4 1 0 0 0 7 0 3 0 18 Su 0 0 0 0 0 8 0 0 0 0 1 0 9 Ne 0 1 0 1 0 0 1 0 1 0 2 0 6 Cu 0 0 0 0 0 0 0 2 1 0 1 0 4 Eng 0 0 0 7 0 1 1 0 530 2 100 0 641 Pl 0 0 0 0 0 0 0 0 1 23 0 0 24 Confid 0 0 0 0 0 0 0 0 0 0 175 0 175 Proud 0 0 0 0 0 0 0 0 0 0 1 1 2 Total 5 1 9 136 1 10 2 2 586 25 304 1 1082  222      Symmetric Measures  Value Asymp. Std. Errora Approx. Tb Approx. Sig. Measure of Agreement Kappa .676 .019 35.527 .000 N of Valid Cases 1082    a. Not assuming the null hypothesis. b. Using the asymptotic standard error assuming the null hypothesis.        223  Appendix 13: Sample of Raw Data Analysis: Students 2, 6, 9, 13 and 21   [DataSet2] C:\Thesis 2012\Data Analysis\SPSS\Students (1)\SPSS Files Students\Student2.sav    Case Processing Summary  Cases Valid Missing Total N Percent N Percent N Percent Emotion * Emotion 100 98.0% 2 2.0% 102 100.0%   Emotion * Emotion Crosstabulation Count  Emotion Total Co/Hes Ne Eng Pl Confid Proud Emotion Co/Hes 9 0 0 0 2 0 11 Dis 0 0 1 0 0 0 1 Ne 0 1 0 0 0 0 1 Eng 1 0 64 0 10 0 75 Pl 0 0 0 1 0 0 1 Confid 0 0 0 0 10 0 10 Proud 0 0 0 0 0 1 1 Total 10 1 65 1 22 1 100   Symmetric Measures  Value Asymp. Std. Errora Approx. Tb Approx. Sig. Measure of Agreement Kappa .708 .070 10.717 .000 N of Valid Cases 100    a. Not assuming the null hypothesis. b. Using the asymptotic standard error assuming the null hypothesis.  224           225    [DataSet5] C:\Thesis 2012\Data Analysis\SPSS\Students (1)\SPSS Files Students\Student6.sav    Case Processing Summary  Cases Valid Missing Total N Percent N Percent N Percent Emotion * Emotion 113 90.4% 12 9.6% 125 100.0%   Emotion * Emotion Crosstabulation Count  Emotion Total Bo Co/Hes Eng Confid Emotion Bo 0 2 2 2 6 Co/Hes 1 14 3 0 18 Eng 0 1 59 18 78 Confid 0 0 0 10 10 Proud 0 0 0 1 1 Total 1 17 64 31 113   Symmetric Measures  Value Asymp. Std. Errora Approx. Tb Approx. Sig. Measure of Agreement Kappa .526 .067 8.850 .000 N of Valid Cases 113    a. Not assuming the null hypothesis. b. Using the asymptotic standard error assuming the null hypothesis.   226         227     [DataSet8] C:\Thesis 2012\Data Analysis\SPSS\Students (1)\SPSS Files Students\Student9.sav    Case Processing Summary  Cases Valid Missing Total N Percent N Percent N Percent Emotion * Emotion 64 91.4% 6 8.6% 70 100.0%   Emotion * Emotion Crosstabulation Count  Emotion Total Co/Hes Su Eng Pl Confid Emotion Co/Hes 9 0 0 0 0 9 Dis 1 0 0 0 0 1 Su 0 1 0 0 0 1 Eng 3 0 24 0 2 29 Pl 0 0 0 3 0 3 Confid 0 0 0 0 21 21 Total 13 1 24 3 23 64   Symmetric Measures  Value Asymp. Std. Errora Approx. Tb Approx. Sig. Measure of Agreement Kappa .862 .053 10.818 .000 N of Valid Cases 64    a. Not assuming the null hypothesis. b. Using the asymptotic standard error assuming the null hypothesis.   228            229  [DataSet12] C:\Thesis 2012\Data Analysis\SPSS\Students (1)\SPSS Files Students\Student13.sav    Case Processing Summary  Cases Valid Missing Total N Percent N Percent N Percent Emotion * Emotion 112 85.5% 19 14.5% 131 100.0%   Emotion * Emotion Crosstabulation Count  Emotion Total Co/Hes Su Eng Confid Emotion Fr 1 0 7 0 8 Co/Hes 26 0 11 0 37 Dis 1 0 2 0 3 Su 0 1 0 0 1 Ne 0 0 1 0 1 Eng 0 0 50 3 53 Pl 0 0 1 0 1 Confid 0 0 0 8 8 Total 28 1 72 11 112   Symmetric Measures  Value Asymp. Std. Errora Approx. Tb Approx. Sig. Measure of Agreement Kappa .602 .063 9.646 .000 N of Valid Cases 112    a. Not assuming the null hypothesis. b. Using the asymptotic standard error assuming the null hypothesis.   230           231    [DataSet15] C:\Thesis 2012\Data Analysis\SPSS\Students (1)\SPSS Files Students\Student21.sav    Case Processing Summary  Cases Valid Missing Total N Percent N Percent N Percent Emotion * Emotion 61 84.7% 11 15.3% 72 100.0%   Emotion * Emotion Crosstabulation Count  Emotion Total Co/Hes Eng Pl Confid Emotion Co/Hes 9 4 0 1 14 Eng 0 31 1 1 33 Pl 0 0 1 0 1 Confid 0 0 0 13 13 Total 9 35 2 15 61   Symmetric Measures  Value Asymp. Std. Errora Approx. Tb Approx. Sig. Measure of Agreement Kappa .810 .067 9.188 .000 N of Valid Cases 61    a. Not assuming the null hypothesis. b. Using the asymptotic standard error assuming the null hypothesis.   232         233  Appendix 14: Sample of Raw Data Analysis: Number of Emotions per Interval   [DataSet1] C:\Thesis 2012\Data Analysis\SPSS\NumberOfEmotions Update March 31\SPSS Files Number of Emotions\NumberOfEmotionsStudentsTotal.sav    Case Processing Summary  Cases Valid Missing Total N Percent N Percent N Percent Number of Emotion * Number of Emotion 1090 78.7% 295 21.3% 1385 100.0%   Number of Emotion * Number of Emotion Crosstabulation Count  Number of Emotion Total 1 2 3 4 Number of Emotion 1 249 233 21 1 504 2 238 197 29 2 466 3 38 58 9 0 105 4 3 6 6 0 15 Total 528 494 65 3 1090   Symmetric Measures  Value Asymp. Std. Errora Approx. Tb Approx. Sig. Measure of Agreement Kappa -.011 .025 -.429 .668 N of Valid Cases 1090    a. Not assuming the null hypothesis. b. Using the asymptotic standard error assuming the null hypothesis.   234        235  Appendix 15: T-Test and Descriptive Statistics      T-Test  Notes Output Created 06-APR-2013 20:48:32 Comments  Input Active Dataset DataSet0 Filter <none> Weight <none> Split File <none> N of Rows in Working Data File 15 Missing Value Handling Definition of Missing User defined missing values are treated as missing. Cases Used Statistics for each analysis are based on the cases with no missing or out-of-range data for any variable in the analysis. Syntax T-TEST PAIRS=pretest WITH posttest (PAIRED)  /CRITERIA=CI(.9500)  /MISSING=ANALYSIS. Resources Processor Time 00:00:00.02 Elapsed Time 00:00:00.01   [DataSet0]     Paired Samples Statistics  Mean N Std. Deviation Std. Error Mean Pair 1 pretest 77.7357 14 9.26271 2.47556 posttest 83.5500 14 8.73761 2.33522    236  Paired Samples Correlations  N Correlation Sig. Pair 1 pretest & posttest 14 .710 .004    Paired Samples Test  Paired Differences t df Sig. (2-tailed) Mean Std. Deviation Std. Error Mean 95% Confidence Interval of the Difference Lower Upper Pair 1 pretest - posttest -5.81429 6.87155 1.83650 -9.78180 -1.84677 -3.166 13 .007     Frequency Tables  IWantedToLearnWhenIWasPlaying  Frequency Percent Valid Percent Cumulative Percent Valid Disagree 1 6.7 7.1 7.1 Agree 6 40.0 42.9 50.0 Strongly Agree 7 46.7 50.0 100.0 Total 14 93.3 100.0  Missing System 1 6.7   Total 15 100.0     IWantedToLeranMathByPlayingTheGame  Frequency Percent Valid Percent Cumulative Percent Valid Disagree 1 6.7 6.7 6.7 Neutral 2 13.3 13.3 20.0 Agree 8 53.3 53.3 73.3 Strongly Agree 4 26.7 26.7 100.0 Total 15 100.0 100.0   237   LearnedMathWhenIPlayedTheGame  Frequency Percent Valid Percent Cumulative Percent Valid Neutral 1 6.7 6.7 6.7 Agree 6 40.0 40.0 46.7 Strongly Agree 8 53.3 53.3 100.0 Total 15 100.0 100.0    ILeranedMathWhenIGotHints  Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree 2 13.3 13.3 13.3 Disagree 3 20.0 20.0 33.3 Neutral 6 40.0 40.0 73.3 Agree 2 13.3 13.3 86.7 Strongly Agree 2 13.3 13.3 100.0 Total 15 100.0 100.0    ILearnedMathWhenIHelpedMiners  Frequency Percent Valid Percent Cumulative Percent Valid Neutral 4 26.7 26.7 26.7 Agree 8 53.3 53.3 80.0 Strongly Agree 3 20.0 20.0 100.0 Total 15 100.0 100.0    ILearnedMathWhenIMadeMistakes  Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree 1 6.7 6.7 6.7 Disagree 2 13.3 13.3 20.0 Neutral 4 26.7 26.7 46.7 Agree 5 33.3 33.3 80.0 Strongly Agree 3 20.0 20.0 100.0 Total 15 100.0 100.0   238    ILearnedMathWhenIFinishedTheQuest  Frequency Percent Valid Percent Cumulative Percent Valid Neutral 3 20.0 20.0 20.0 Agree 7 46.7 46.7 66.7 Strongly Agree 5 33.3 33.3 100.0 Total 15 100.0 100.0    ILearnedMathWhenIHadHarderQuestions  Frequency Percent Valid Percent Cumulative Percent Valid Strongly Disagree 1 6.7 6.7 6.7 Disagree 2 13.3 13.3 20.0 Neutral 1 6.7 6.7 26.7 Agree 5 33.3 33.3 60.0 Strongly Agree 6 40.0 40.0 100.0 Total 15 100.0 100.0  Histograms    239      240      241      242      243         244      245      246      

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0167291/manifest

Comment

Related Items