UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Playfulness in e-picturebooks : how the element of play manifests in transmediated and born-digital picturebook… Frederico, Aline 2014

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2014_september_frederico_aline.pdf [ 2.21MB ]
Metadata
JSON: 24-1.0167471.json
JSON-LD: 24-1.0167471-ld.json
RDF/XML (Pretty): 24-1.0167471-rdf.xml
RDF/JSON: 24-1.0167471-rdf.json
Turtle: 24-1.0167471-turtle.txt
N-Triples: 24-1.0167471-rdf-ntriples.txt
Original Record: 24-1.0167471-source.json
Full Text
24-1.0167471-fulltext.txt
Citation
24-1.0167471.ris

Full Text

PLAYFULNESS IN E-PICTUREBOOKS: HOW THE ELEMENT OF PLAY  MANIFESTS IN TRANSMEDIATED AND BORN-DIGITAL PICTUREBOOK APPS  by Aline Frederico  B.A., Universidade de São Paulo, 2005  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  MASTER OF ARTS in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Children’s Literature)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  May 2014  © Aline Frederico, 2014  ii Abstract This thesis analyzes how playfulness is expressed in eight picturebook apps available for the iPad, four of them being born-digital picturebook apps and the other four picturebooks being transmediated into apps from print counterparts. These works of digital literature, aimed at children between 3 and 8 years old, underwent a close reading through the lenses of social semiotics, as presented by Gunther Kress and Theo Van Leeuwen, and its manifestation in picturebook theory, as presented by David Lewis’ ecology of the picturebook. Playfulness was analyzed according to the three categories proposed by Nikolajeva: through the interanimation among modes, through metafiction and through performance–implicit and explicit. One born-digital and one transmediated app was selected among a sample of 100 picturebook apps as quintessential examples of that type of playfulness, although all apps manifested on different levels all kinds of playfulness.  The multimodal analysis of these picturebook apps revealed that each app is unique in the way modes both individually and in combination work as a multimodal text in expressing playfulness. The different modes may work in counterpoint to generate irony, or they may complement each other building signs that are ironic in contrast with other signs inside the narrative. The inclusion of interactivity makes possible new combinations of modes that integrate reader inputs and various forms of participation. Participation is an important element in the construction of metafiction since, as interactive narratives, most texts manifest an overt recognition of the reader as a participant. The differences between born-digital and transmediated apps are subtle, but this sample suggested that the counterpart among modes is used as a playful resource more significantly in transmediated apps, while the born-digital apps count more on interactivity and performance to   iii generate playfulness. Half of the transmediated apps manifested covert metafiction, while all of the born-digital texts manifested overt metafiction. Finally, in terms of performance, the born-digital apps showed highly theatricalized participation of the reader and also promoted reader participation in co-authoring, which was not seen in the transmediated apps.   iv Preface This dissertation is an original intellectual product of the author, A. Frederico. The selection of the primary texts was based on a collection of 100 picturebook apps, organized by E. Zaminpaima and myself as part of the research project The Future of Children’s Texts, coordinated by Dr. E. Meyers.   v Table of Contents Abstract ................................................................................................................................. ii	  Preface .................................................................................................................................. iv	  Table of Contents ...................................................................................................................v	  List of Tables ...................................................................................................................... viii	  List of Figures ...................................................................................................................... ix	  Acknowledgements ............................................................................................................. xii	  Dedication ........................................................................................................................... xiii	  Chapter 1: Introduction .................................................................................................. 1	  1.1	   Introduction .......................................................................................................... 1	  1.2	   Origins of my interest ........................................................................................... 1	  1.3	   Topic statement .................................................................................................... 2	  1.4	   Research questions ............................................................................................... 3	  1.5	   Relevance of this study ......................................................................................... 4	  1.6	   Constructs: from picturebook to picturebook app ................................................ 6	  1.7	   Summary ............................................................................................................ 13	  Chapter 2: Literature Review ...................................................................................... 15	  2.1	   Introduction ........................................................................................................ 15	  2.2	   Picturebook theory .............................................................................................. 15	  2.3	   Playfulness and the picturebook ......................................................................... 25	  2.4	   Multimodality and children’s literature .............................................................. 36	  2.5	   Summary ............................................................................................................ 56	  Chapter 3: Methodology ............................................................................................... 57	    vi 3.1	   Introduction ........................................................................................................ 57	  3.2	   Theoretical framework ....................................................................................... 57	  3.3	   Rationale for selection of primary texts ............................................................. 60	  3.4	   Close reading of digital texts .............................................................................. 64	  3.5	   Summary ............................................................................................................ 66	  Chapter 4: Findings: Close Readings of Eight Picturebook Apps ............................ 67	  4.1	   Introduction ........................................................................................................ 67	  4.2	   What Does My Teddy Bear Do All Day? by Bruno Hächler and Birte Müller .. 67	  4.3	   Lil’ Red by Bart Blocmen and Brian Main ......................................................... 75	  4.4	   The Monster at the End of This Book by Jon Stone and Michael Smollin ......... 82	  4.5	   Don’t Let the Pigeon Run This App! by Mo Willems and You .......................... 88	  4.6	   The Heart and the Bottle by Oliver Jeffers ........................................................ 94	  4.7	   The Three Little Pigs by Nosy Crow ................................................................ 101	  4.8	   The Trip - Little Critter Reading Adventure by Mercer Mayer ........................ 109	  4.9	   Little Red Riding Hood by Nosy Crow ............................................................ 114	  4.10	   Summary ........................................................................................................ 121	  Chapter 5: Discussion: Playfulness in Picturebook Apps ........................................ 122	  5.1	   Introduction ...................................................................................................... 122	  5.2	   Playfulness through the interanimation among modes ..................................... 122	  5.3	   Playfulness through metafiction ....................................................................... 138	  5.4	   Playfulness through performance ..................................................................... 150	  5.5	   How this discussion addresses my research questions ..................................... 169	  5.6	   Summary .......................................................................................................... 171	    vii Chapter 6: Conclusion ................................................................................................ 173	  6.1	   Introduction ...................................................................................................... 173	  6.2	   Research contributions ..................................................................................... 173	  6.3	   Contributions to the design of picturebook apps .............................................. 175	  6.4	   Implications for the mediators of children’s literature ..................................... 178	  6.5	   Limitations ........................................................................................................ 181	  6.6	   Recommendations for future research .............................................................. 182	  6.7	   Summary .......................................................................................................... 184	  References ..........................................................................................................................185	  Primary texts .............................................................................................................. 185	  Secondary texts .......................................................................................................... 186	  Children’s literature cited .......................................................................................... 195	  Online resources consulted for picturebook app selections ...................................... 195	  Appendix: Screen references ..............................................................................................197	       viii List of Tables Table 3.1  Initial app selection ................................................................................. 62 Table 3.2  Final app selection  .................................................................................. 63 Table 3.3 Method for close reading the picturebook apps ...................................... 65   ix List of Figures Figure 4.1 What Does My Teddy Bear do All Day?, screen 6  ..................... 71 Figures 4.2 (A) and (B)  What Does My Teddy Bear Do All Day?, screen 8 in two different  moments  ..................................................................................... 73	  Figure 4.3  Lil’ Red, screen 6  ........................................................................ 76	  Figures 4.4 (A) and (B) Lil’ Red, screen 9 in two different moments  .............................. 77	  Figure 4.5  Lil’ Red, transition between screens ............................................ 78 Figures 4.6 (A) and (B) Lil’ Red, screen 9 in two different moments ................................ 81 Figures 4.7 (A), (B) and (C)  The Heart and the Bottle, screen 13 in three different moments ... 95 Figure 4.8  The Three Little Pigs, screen 2 .................................................. 103	  Figure 4.9  The Three Little Pigs, screen 4 .................................................. 104	  Figures 4.10 (A), (B) and (C)  Little Red Riding Hood, screen 13 in three different moments  .. 119 Figure 5.1  What Does My Teddy Bear Do All Day?, screen 7 ................... 123 Figure 5.2  Little Red Riding Hood, navigation screen ................................ 146 Figure 5.3  The Heart and the Bottle, screen 7 ............................................ 157	  Figure A.1 Screen 0  .................................................................................... 198	  Figure A.2  Screen 1  .................................................................................... 198 Figure A.3  Screen 2  .................................................................................... 198	  Figure A.4  Screen 3  .................................................................................... 198	  Figure A.5  Screen 4  .................................................................................... 199	  Figure A.6  Screen 5  .................................................................................... 199	  Figure A.7  Screen 6a  .................................................................................. 200	  Figure A.8  Screen 6b  .................................................................................. 200	    x Figure A.9  Screen 7  .................................................................................... 200	  Figure A.10  Screen 8a  .................................................................................. 200	  Figure A.11  Screen 8b  .................................................................................. 200	  Figure A.12  Screen 8c  .................................................................................. 200	  Figure A.13  Screen 8d  .................................................................................. 200	  Figure A.14  Screen 9 . ................................................................................... 200	  Figure A.15  Screen 10a  ................................................................................ 201	  Figure A.16  Screen 10b  ................................................................................ 201	  Figure A.17  Screen 10c  ................................................................................ 201 Figure A.18  Screen 11  .................................................................................. 201	  Figure A.19  Screen 12  .................................................................................. 201	  Figure A.20  Screen 13  .................................................................................. 201	  Figure A.21  Screen 14  .................................................................................. 202 Figure A.22  Screen 15a  ................................................................................ 202 Figure A.23  Screen 15b  ................................................................................ 202 Figure A.24  Screen 16a  ................................................................................ 202 Figure A.25  Screen 16b  ................................................................................ 202 Figure A.26  Screen 16c  ................................................................................ 202 Figure A.27  Screen 16d  ................................................................................ 202 Figure A.28  Screen 17a  ................................................................................ 202 Figure A.29  Screen 17b  ................................................................................ 202 Figure A.30  Screen 17c  ................................................................................ 202 Figure A.31  Screen 18a  ................................................................................ 203	    xi Figure A.32  Screen 18b  ................................................................................ 203	  Figure A.33  Screen 18c  ................................................................................ 203	  Figure A.34  Screen 19  .................................................................................. 203	  Figure A.35  Screen 20  .................................................................................. 203     xii Acknowledgements I would like to thank my supervisor, Dr. Eric Meyers, and my committee member, Dr. Maureen Kendrick, for their guidance and support in the writing of this thesis; Dr. Teresa Colomer and Cristina Correro, who guided me during my research exchange at Universitat Autònoma de Barcelona; and Dr. Judith Saltman, a role model and inspiration on my academic path and a person who has always been available with words of wisdom, care and encouragement.  In terms of financial support, I would like to thank the Social Sciences and Humanities Research Council of Canada, which awarded me with a Joseph-Armand Bombardier Canada Graduate Scholarship, and the UBC Office of the Vice President Research & International, which awarded me with an UBC Graduate Student International Research Mobility Award in support of my research exchange to Universitat Autònoma de Barcelona. I would also like to thank my fellow colleague in the Master of Arts in Children’s Literature, Cynthia Nugent; Dr. Kathie Shoemaker from the Department of Language and Literacy Education, and the PhD students from Universidad Autònoma de Barcelona Celia Turrión; and Brenda Bellorin. I have immensely benefited from their knowledge and insights on children's literature and picturebook apps in the writing of this thesis. In addition, I would like to thank Auryn Inc., Brian Main, Nosy Crow and Oliver Jeffers for authorizing the reproduction of print screens from their picturebook apps in this thesis. Finally, thanks to my family and friends for being so far but so close, giving me the support I need in the various stages of this degree.   xiii Dedication     Aos meus pais,  que me ensinaram o poder do conhecimento.  To my parents, who taught me the power of knowledge.    1 Chapter 1: Introduction 1.1 Introduction In this chapter, I introduce the aims of the present thesis and the area of picturebook app scholarship, its context and relevance. First, I explore the origins of my interest in picturebook apps as a professional editor and designer. I then present the topic statement and research questions that guide my work and aim to discuss the forms through which playfulness manifest in picturebook apps and the ways different modes participate in creating the playful features. I also state the importance of the present study in starting to fill the gap in picturebook scholarship with regards to the digital manifestation of picturebooks as apps and the understanding of how this emergent form of literature for children works. Finally, I examine a historical perspective of the scholarship regarding digital picturebooks for children and define the term “picturebook app” according to the framework that guides this study.  1.2 Origins of my interest  As an editor and designer in the educational publishing industry for the past 8 years, I have followed how technology is dramatically affecting the way we produce and receive texts. The publishing industry is being completely reformulated at the present moment to adapt to and accommodate such changes, but the answers are not clear or definitive. I was part of the team that created the digital contents department in a big multinational publishing house in Brazil in 2010, and since then I have been trying to make sense of these transformations with the objective of producing the best possible books for children in print and electronic format. I have always been fascinated with children’s literature, especially in the picturebook format and for this reason I decided to specialize in the area by taking this master’s degree. With   2 the release of the iPad in 2010 and the first interactive picturebook apps being released, I was immediately delighted by the new affordances of this technology and how they turn the picturebook format into an even more complex multimodal text. Even the quite simple (for today’s eyes) first picturebook apps had a huge impact on me and I believe this format will allow picturebook creators to take their creativity and artistry to a whole new level. The understanding of this new form of literature and how children receive it has been for the past year and a half the focus of my academic curiosity.   1.3 Topic statement As a very complex literary form, it was difficult to select and focus on one aspect of picturebook apps for this study. One characteristic that has always drawn my attention in picturebooks is the power to convey deep and complex meanings in a delightful and playful way that speaks to children. Playfulness in picturebooks can manifest in so many different ways, from the simple relationship between images and texts–that some authors call interplay–to works that explore the materiality of books, such as baby books or pop-up books. Nonetheless, the first study addressing children’s engagement with picturebook apps (Chiong, Ree, Takeuchi, & Erickson, 2012) considered that the playfulness promoted by the interactions with the screen was distracting and detrimental to children’s understanding of the story.  I believe that the playful potential of picturebooks changes significantly in the transition from print to digital media. New modes of meaning making are integrated in this “interplay” and the materiality of the picturebook that is related to many of its playful aspects is drastically changed. With this research I aim to find out how playfulness in picturebook apps that were created based on a print picturebook counterpart–which I call transmediated apps based on   3 Suhor’s (1984) definition: “the translation of content from one sign system into another” (p. 250)–differs from playfulness manifested in born-digital picturebook apps–which are books “first created for use only in a digital form, having no previous version existing on paper” (Armstrong, 2008, p. 197). My hypothesis is that the latter tend to explore the affordances of this new media in more novel and innovative ways. This study focuses on analysing fictional picturebook apps aimed at children from 3 to 8 years old and available for the iPad.   1.4 Research questions In this study, I seek to understand the various forms through which playfulness manifests in picturebook apps and also to understand how the multimodal text of these narratives interanimates different modes to construct its playful features. Thus, the study is guided by the following research questions:  • How does playfulness manifest in picturebook apps transmediated from print books and in born-digital picturebook apps? • How do the different modes–writing, images, speech, sound, animation and touch–contribute to the playful aspects of picturebook apps? In order to answer these questions, I closely read eight picturebook apps through the lens of multimodality focusing on how each of the modes works in the construction of playfulness and how they interanimate to manifest playfulness in three ways: through the interanimation among modes, through metafiction and through performance, both implicit and explicit.    4 1.5 Relevance of this study Reading on small, medium and large screens have become part of the reading habits of a great part of the North American population, including children at very early ages. When talking about reading habits for young children, the picturebook is one of the most significant reading experiences, promoting language, visual and aesthetic development since birth (Kiefer, 1995). The complexity of picturebook texts and how they create meaning for children have been the focus of discussion among picturebook scholarship for the past three decades (Agosto, 1999; Lewis, 2001; Nikolajeva, & Scott, 2000, 2001; Nodelman, 1988; Salisbury, & Styles, 2012; Schwarcz, 1982; Sipe, 1998; Van der Linden, 2011). With the pervasiveness of technology throughout most aspects of modern life, the transmediation of the picturebook format from print to digital media happened as soon as portable tablet computers with bright multitouch screens became available. As is also the case with print picturebooks, most of the scarce studies discussing digital picturebooks–usually referring to CD-ROM interactive picturebooks–come from education and literacy studies (Chiong, et al., 2012; de Jong, & Bus, 2003; Hempstead, 2008; Kim, & Hall, 2002; Korat, 2009, 2010; Korat, & Shamir, 2007, 2008; Mol, Bus, & de Jong, 2009; Parish‐Morris, Mahajan, Hirsh‐Pasek, Golinkoff, & Collins, 2013; Robb, 2010; Smeets, & Bus, 2012; Underwood, & Underwood, 1998; Wasik, & Bond, 2001; Zucker, Moody, & McKenna, 2009). Although it is important to understand how these texts influence children’s language and multiliteracy development, most studies discuss the picturebook from a very instrumental point of view and with a very traditional perspective on what it means to be literate–basically decoding the written text or gaining vocabulary. During the first years of picturebook app development, most scholars have focused on trying to establish some criteria for evaluating these apps, again with the focus on educational and library uses of picturebook apps (Bircher,   5 2012; Bird, 2011; Cahill, & McGill‐Franzen, 2013; Ishizuka, 2011; Yokota, & Teale, in press). The agendas of children’s literature conferences that took place in 2013 show that digital children’s literature and picturebook apps have been of interest to many scholars. The graduate conference The Child and The Book had as theme “Children’s Literature, Technology and Imagination” (Zago, Lambello, Merlo, Goga, & Campagnaro, 2013), while the International Research Society for Children’s Literature conference discussed “Children’s Literature and Media Cultures” (Ghonem-Woets, Joosen, Dinh, Van Lierop, Wenz, & Wolters, 2013) and both included several presentations discussing and analyzing picturebook apps. Nonetheless, probably due to the long process involved in the publication in peer review journals, little has been published, especially from an aesthetic point of view. Al-Yaqout (2011) was the first to comment on the changes and challenges picturebook apps pose to picturebook scholarship. Although her paper was published at a very early stage of picturebook app development, it has valuable insights on the differences and similarities between print and app picturebooks. It is urgent, then, to start filling the gaps in our understanding of how these digital texts work in order to inform the decision of creators, publishers, readers and mediators–such as parents, teachers and librarians. Al-Yaqout stated in 2011, “with regard to picturebook scholarship, a more widespread ownership of the iPad might possibly even lead to a reformulating of picturebook poetics” (p. 67). Only two years later, we seem already to be living part of what she foresaw. For Penelas (2013), “even though the exercise of revisiting previous works and noticing possible influences is extremely important to the understanding of book-apps within our existing literary system, we should enhance the analysis by going forward, in order to identify these products for what they   6 are, apart from these influences”1 (p. 11). By analysing picturebooks that were transmediated from print media and born-digital picturebooks, I expect to be mapping the emergence of the multimedia picturebook and pointing towards what the characteristics this new format may assume in the near future.   1.6 Constructs: from picturebook to picturebook app 1.6.1 Defining picturebook This study draws on the definition that picturebooks are “books intended for young children which communicate information or tell stories through a series of many pictures combined with relatively slight texts or no texts at all” (Nodelman, 1988, p. vii). They are different from illustrated books, in which the narrative is carried mostly or exclusively by the words, while the images represent visually certain passages and/or decorate the book without being necessary for the understanding of the story. As in most recent scholarly works in the field, I use the term picturebook as a single word, since it reflects this interdependence of words and pictures in the creation of meaning in picturebook narratives. As Lewis (2001) posits, the picturebook is a “perpetual developing form” (p. 61). He attributes the creation and early development of the picturebook with the technological changes that allowed printing images and texts on the same page; social and cultural changes that promoted the “rise of the image” and the importance of visual literacy; and also experimentation                                                 1 My translation from the original in Spanish: “aunque el ejercicio de revisitar obras anteriores y apuntar posibles influencias se considera de enorme importancia para la comprensión de las aplicaciones-libro dentro de nuestro sistema literario existente, debemos enriquecer el análisis yendo más allá, para identificar estos nuevos productos por lo que son, al margen de dichas influencias.”   7 in arts and literature that happened from the 1960s onward. Similarly, we have been seeing in the past decade technological, social and cultural changes that made possible the development of the picturebook into a multimedia text. Due to the novelty of the field, definitions of the digital possibilities of picturebooks are still not clear. Digital picturebooks can appear in different forms with different affordances, depending on the will of the creators, the format in which they are created and the device on which they are played, all of which have changed considerably in the past decades.  1.6.2 Origins of digital picturebooks During the 1990s and 2000s, digital picturebooks were present mostly as CDs and DVDs for desktop computers, and myriad terms were used to describe them: interactive books (Mol et al., 2009; Robb, 2010), electronic books (Zucker et al., 2009), CD-ROM storybooks (Korat, 2009; Mackey, 2002), interactive electronic storybooks (Smeets, & Bus, 2012). These studies analysed digital picturebooks from an educational point of view, with clear concern for the impact on children’s language and literacy learning. Such emphasis reflects the fact that, in general, these books seemed to be created exclusively with the objective of promoting these skills; they were not created as literary texts. Although it is hard to find numbers regarding the popularity of such texts, it is known that they were not consistently part of the reading habits of the population in general and the number of titles available was extremely limited in comparison to traditional print picturebook publishing.  Online libraries have also provided access to digital picturebooks, usually through public or school library websites. Tumblebooks is one example of a digital library for children being used by many library systems across Canada and includes more than 200 picturebooks in English, and other titles in French and Spanish. It includes classics and bestsellers such as the   8 work of Robert Munch, Paulette Bourgeois, and Melanie Watt. All picturebooks available are e-book versions of print picturebooks with added narration, word highlighting and simple animation, but no interactivity. In the auto mode the reader does not control the turn of the pages and the narrative is presented in a fashion similar to an animation, except for the fact that the text is also on the screen as it is being narrated. It can be viewed on a tablet computer, but then the book is available as video, distancing it even more from the book format. The service has been available since January 2011 at the Vancouver Public Library System and has reached more than 30,000 book views in almost 3 years (Vancouver Public Library, n.d. a). It may seem a big number but in fact it is but still insignificant in comparison to the 9 million items borrowed annually through the VPL system (Vancouver Public Library, n.d. b). 1.6.3 Tablet computers and the emergence of picturebook apps The publication of digital picturebooks changed significantly in 2010 with the release of the iPad. In this light, portable, multitouch, high-resolution device, picturebooks seem to have found their ideal platform in their transmediation into digital media. Its format, weight and resolution were much closer to that of print picturebooks and allowed children and parents to reproduce reading habits already associated with print picturebooks, especially that of bedtime storytelling. According to Al-Yaqout (2011), “the iPad is the closest thing that technology has produced to mimic not only the physical aspect of the picturebook, but the book reading experience as well” (p. 64). As of October 2013, Apple has sold 170 million iPads and more than 475,000 apps are available for the device (Guglielmo, 2013). The iPad allows the creation of digital picturebooks including text, images, animation, video, audio effects, soundtrack, narration, and interactions. Interactions can take on myriad forms. The iPad’s multitouch screen allows up to 30 interactions   9 at a time and measures 9.7 in (250 mm) diagonal. The later versions of the device also include built-in speakers, two cameras, a microphone and an accelerometer, a three-axis gyro, ambient light sensor, a digital compass and GPS locators; the device is sensitive to movement in any direction, as well as to its location in space and its geographic coordinates. All these features allow for a truly multimedia experience. As seen in the apps analyzed in this study, these affordances can be utilized to generate incredibly creative interactions. There are two main formats in which picturebooks appear today in portable devices: as epub or app. The epub format is significantly more limited in terms of interactivity and does not allow the inclusion of all the digital features available for tablet computers and smartphones. Apps are software and allow the explorations of all the device’s affordances, although the cost and difficulties in programming some of the more complicated features may make them economically unfeasible. Thus, the fact that the app format allows for the creation of such multimedia experiences does not mean that all apps take advantage of them, and there are many examples of apps that show much more restricted interactivity than most books in the e-pub format. One significant difference between the two formats appears to be how you access these texts. In Apple devices, picturebooks in app format are available for sale and download at the app store, while epubs are available at the iBook store, with a similar but separate process. Most digital picturebooks available today for the iPad are in app format, and picturebook apps have been so far the focus of studies addressing picturebooks on tablet computers. The terminology being used varies significantly: “book app” (Bircher, 2012; Bird, 2011; Cahill, & McGill‐Franzen, 2013; Ishizuka, 2011; Leverkus, 2011; Penelas, 2013; Stichnothe, 2014), “e-book app” (Parish‐Morris et al., 2013) and “picturebook app” (Al-Yaqout, 2011) are some of the terms used for picturebooks for the iPad in the recent literature. The main limitation of using   10 terms that involve the concept of an app is that the app format is probably transitory. As technology evolves, it is very likely that new formats will appear or that the epub format will develop and acquire the same affordances that the app format currently possesses. Nonetheless, for now, these terms adequately distinguish the formats of digital picturebooks being discussed.  A “quick study” from The Joan Ganz Cooney Center makes reference to “basic” and “enhanced e-books.” They define “enhanced e-books” as e-books that “support highly interactive, multimedia experiences” (Chiong et al., 2012) available for the iPad. It is very likely that “enhanced e-book” in this study was used to refer to picturebook apps, while the “basic e-books” would be in the epub format, but the paper does not clarify this information. As mentioned before, this differentiation is problematic because both formats can vary significantly. A more general term being used is “digital picturebook” (Yokota, & Teale, 2014), which was referred to in the same paper also as “e-picture book.” Although the term is not as precise, being necessary to specify if dealing with book apps, epubs, or CD-ROMs, “digital picturebook” is certainly a longer-lasting term. I believe that in the future, as digital picturebooks integrate in a more meaningful way the different modes of telling stories, it will be necessary to formulate a new term that does not privilege the image to the detriment of other modes of meaning-making.  1.6.4 Is it still a picturebook? Al-Yaqout (2011) considers the iPad as the electronic device most similar to the format of a codex picturebook. She has compared the formal aspects of the picturebook in both mediums and asserts that the classical definition of a picturebook is applicable to picturebooks in app format. She calls our attention to the fact that the iPad is much easier to use than a computer. Its instructions can be both written and spoken, and as a result young children enter the category of potential users of tablets. In terms of format, the iPad is the closest in form, shape, weight and   11 coloured image reproduction to a codex picturebook: “it appears as if its primary aim is to come as close as possible to the experiences had with the conventional codex picturebook” (p. 65). In terms of the relationship between image and text that is the essence of the picturebook form, Al-Yaqout asserts that “the iPad has remained true to such amalgamation that is essential to the makeup of the form [but] in the iPad, the level of interaction actually surpasses that of its codex counterpart” (p. 66). According to the Oxford Advanced Learner’s Dictionary (n.d.), in this realm, a book has two main definitions. The first refers to the materiality of the book, “a set of printed pages that are fastened inside a cover so that you can turn them and read them”; the second refers to the content, “a written work published in printed or electronic form,” although this definition would exclude, for instance, wordless picturebooks from this category. Barbara Bader’s (1976) definition of the picturebook is the most noted in children’s literature scholarship:  A picturebook is a text, illustrations, total design; an item of manufacture and commercial product; a social, cultural, historical document; and foremost an experience for a child. As an art form, it hinges on the interdependence of pictures and words, on the simultaneous display of two facing pages and on the drama of the turning page. (p. 1)  This definition is very encompassing, and considers the textual, material, aesthetical and cultural aspects of the picturebook. Al-Yaqout compared every aspect of Bader’s definition in both codex and app format. Contrasting this definition with the dictionary’s definition of a book, the similarities between both media listed by Al-Yaqout lay mostly in the aspects of a picturebook related to the second   12 definition: app and codex are similar in the relationship between image and text; both are documents and an experience for the child reader. Yet, some of these aspects refer to the materiality of the codex: both are manufactured products and, although there aren’t real pages in picturebook apps, their narratives are also built upon the “drama of turning the pages.” The differences Al-Yaqout lists between codex and app lie on the materiality of the devices, which influence its durability, price and how the content itself is accessed: the iPad is more fragile and more expensive than a codex. Also, the support–the tablet computer–is dissociated from the content; it requires readers to download picturebook narratives in an action that is independent of buying the tablet itself. When purchasing an iPad, users are not acquiring a picturebook, but the possibility of numerous picturebooks, among other types of content such as music, movies, news articles, novels, etc. In summary, it seems that picturebook apps have more resemblances than disparities from their codex counterparts. Nonetheless, Al-Yaqout’s examples refer to the first generation of picturebook apps, which were certainly more attached to the conventions of codex books and did not explore the boundaries between picturebooks and other forms of sequential narrative, especially animation and videogames. For example, although most picturebook apps have the system of “turning the pages,” there are already a few examples of picturebook apps that challenged the notion of “page,” which migrated from the codex book, with scrolling structures that are typical of reading on the screen. The picturebook apps analyzed in this study represent some of the more innovative apps produced so far, and challenge Bader’s definition of picturebook in ways not considered by Al-Yaqout. Yet, while we may question one or another aspect of what we usually understand as a picturebook, the presence of a great number of the other aspects make the form recognizable.   13 1.6.5 Definition of picturebook apps In this study, I am exclusively discussing digital picturebooks in the format of apps, and I call them simply picturebook apps. The working definition, based on Nodelman’s definition of the print picturebook, is: picturebook apps are digital books that communicate information or tell stories through a multimodal text. In addition to images and usually writing, the picturebook app designed to touchscreen devices often combines at least one other interrelated mode, such as speech, movement, music, or interactivity.  Nikolajeva and Scott (2001, p. 6) use the term “iconotext,” coined by Kristin Hallberg (1982), to refer to the multimodal text of picturebooks. The use of this notion in the analysis of digital picturebooks would privilege the verbal and visual system to the detriment of other modes. In the lack of proper terminology, the term “text” in this study then refers to a specific combination of two or more modes that together generate meaning in picturebook apps. The verbal system as expressed by the written text on the screen is called “word” or “writing,” while the verbal system as expressed by speech is called “narration.”  1.7 Summary This chapter covered the objectives of this study and the background information necessary to understand the field of picturebook apps scholarship and its significance. First, I presented my motivation as well as the topic statement and research questions that guide this study. Guided by my passion for picturebooks and background in publishing and graphic design, this thesis discussed the forms through which playfulness manifests in picturebook apps by comparing four transmediated and four born-digital picturebook apps and their multimodal texts. In order to define some key constructs in this study I also presented a panoramic view of the scholarship   14 around digital picturebook apps, from the first manifestation as CD-ROMs for desktop computers to the most recent manifestation as apps for portable multitouch devices. Lastly, I posit the definition of “picturebook app” that guides this study.   15 Chapter 2: Literature Review 2.1 Introduction In this chapter, I review the three areas of the scholarship that I believe are fundamental for the understanding of the picturebook text and the understanding of how playfulness manifests in this literary form. The first area is generally called picturebook theory, and it reviews various authors who analyze the print picturebook as a literary form, its multimodal text and how this text constructs meaning in the relationship between writing and visual modes. The second area reviews authors who discuss the concept of play from various disciplines, and scholars who analyze how playfulness manifests in children’s texts, particularly in picturebooks in the context of postmodernity. The last area reviews the various sources that discuss how writing, visual, colour, typography, sound and interactivity work to construct meanings from a social semiotics perspective. This part provides the present study with a metalanguage to analyze the multimodal text of picturebook apps.   2.2 Picturebook theory One of the key debates in picturebook theory has concentrated on the multimodal aspects of the form, especially on how meaning is created through the complex interactions between words and images (Agosto, 1999; Lewis, 2001; Nikolajeva, & Scott, 2000, 2001; Nodelman, 1988; Schwarcz, 1982; Sipe, 1998; Van der Linden, 2011). As pointed out by Lewis (2001), “there has been a broad consensus about the basic characteristics of the form, its combining of two distinct modes of representation–pictures and words–into a composite text, but it is precisely this doubleness, this two-sided quality, which has led to much confusion and disagreement” (p. xiii).    16 Joseph Schwarcz (1982) considers that one of the main pleasures of the picturebook lies in the fact that both modes are read together. Nonetheless, he does not seem to consider that they form one multimodal text; he shows a word-centered perspective, in which illustrations come after the writing, to modify its meaning in some way. Schwarcz posits that reading the verbal text occurs as a “linear progression,” while reading the image happens all at once, simultaneously. The author proposes two ways images and words relate to each other in picturebooks, which he calls the “functions of the illustration.” When in congruency, words and images agree with each other. There is never complete redundancy between images and words because the images are more concrete than words and provide details that words do not. The illustrator then “adds to, elaborates the text” (p.14). Within this category also lie narratives in which images and writing alternate in telling the story, which he calls “alternate progress” (p. 15). On the other hand, images and words can relate to each other in deviation, in which “illustrations which are initiated by the textual framework … veer away from it due to the illustrator’s own associations and ideas” (p.16). This relationship can happen when the images oppose or alienate the words, in other words, when they counterpoint each other.  Nodelman (1988) focuses on the semantic qualities of verbal and visual texts and how the differences between them contribute to the meaning-making process in picturebooks. He explores how images affect words and vice versa with the support of concepts from semiology and neuroscience.  To attest the differences in the essential characteristics of images and words in terms of reading and meaning-making, Nodelman uses Robert Ornstein's (1975) studies of the differences in consciousness between left and right brain hemispheres. The left hemisphere consciousness is “lineal, sequential, casual, focal, explicit, verbal,” while the right hemisphere is   17 “nonlineal, simultaneous, acasual, diffuse, tacit, spatial” (p. 83). Nevertheless, both parts of the brain work together in the process of reading and codifying any kind of information.  Analyzing the natures of verbal and visual language gives us important insights into their roles in the construction of the picturebook narrative. The differences between the two forms are what create the richness of their interactions: “words make pictures into rich narrative resources–but only because they communicate so differently from pictures that they change the meanings of pictures ... for the same reason, ... pictures can change the narrative thrust of words” (Nodelman, 1988, p. 196). For example, in an image without any words, it is difficult to know if it occurs in the present, past or future, while writing could express it very precisely with a few words. On the other hand, an image can show details, textures and styles that would never be as clear if they were only expressed in writing, even with a long and detailed description.  Bill Nichols (1981) compares the nature of words and images in terms of digital as opposed to analogical codification. For him, writing can be understood as digital codes that allow the expression of complex semantically precise meanings; in contrast, images have an analogical nature, rich in meaning but imprecise in signification. Drawing from Nichols, Nodelman (1988) states “placing [words and pictures] into relationship with each other inevitably changes the meaning of both, so that good picture books as a whole are a richer experience than just the simple sum of their parts” (p. 199). The differences between word and image and how they are received and interpreted by the reader occur in highly specialized interesting ways and their combination is what enables picturebooks to convey the story in a complex and unique way. Nodelman’s understanding of picturebooks is also informed by Roland Barthes’ (1985) effect of “relaying,” in which words and images are only part of a general “syntagm” and “the message’s unity occurs on a higher level, that of the story” (Barthes, 1985, p. 30). Although   18 Nodelman's theories are substantially focused on the analysis of the image rather than the words, in a text comprising of both words and images he suggests that there is a predominance of the verbal in the reading and interpretation of images. Words suggest cause and effect relationships, temporal assertions and make us give little or no attention to other possible interpretations of an image. He attributes this phenomenon to a concept Barthes (1985) called “anchoring”: “language helps identify purely and simply the elements of the scene and the scene itself… [for] the text directs the reader among various signifieds of the image, causes him to avoid and to accept others; through an often subtle dispatching, it teleguides him toward a meaning selected in advance” (pp. 28-29). An image provides diffuse information–many elements may have equal importance and important information may not stand out from the whole–so it is up to the reader to interpret its meaning. The written information can then direct us to certain elements that are intended to be noticed, creating hierarchy that helps in the construction of the narrative. In sum, Nodelman proposes three effects of words upon pictures (p. 215):  • Express emotions or stress the importance for the narrative of gestures presented in the images. • Connect or stress relationships between images or elements in a image, such as cause and effect or passage of time. • Point out the elements that matter in the complexity of visual information in an image. Meaning and communication occur through the ‘interplay’ of word and image, one changing the meaning of the other. When looking at a page, we have a “general understanding of what pictures communicate”; then, “words correct or particularize our understanding,” while, simultaneously or almost simultaneously “pictures provide   19 information that causes us to reinterpret and particularize the meaning of the words” (p. 217). As a conclusion, Nodelman posits this interaction acts in a sense of constraint or limitation: the picture limits the words as the writing limits the image, and it is in this mutual “control” of information that lays down the meaning in the picturebook, establishing an ironic relationship: “each speaks about matters on which the other is silent” (p. 221). Although claiming images and words each change the meaning of the other, Nodelman tends toward an image-centered perspective, in which he considers the function of words in changing the meaning of the pictures, without discussing further the final multimodal text that is generated by the combination and interaction of both codes. Golden (1990) proposes five possible relationships between writing and image, depending on the level of participation of one mode or the other: 1) Writing and pictures are symmetrical; they present redundant information; 2) Writing depends on picture for clarification; words need the image to “carry the narrative”; 3) Illustration enhances writing; writing is the fundamental narrative but the image “extends and elaborates the [words]” by providing details; 4) Writing carries the primary narrative; illustration is selective; 5) Illustration carries the primary narrative; writing is selective. As mentioned by Sipe (1998), one of the key issues with Golden’s proposition is that she focuses on the role of each mode independently and not on the transactions between them. Agosto (1999) discussed what she calls picture storybooks in terms of “twice-told tales,” in which parallel stories are told by images and text, and “interdependent storytelling,” in which there is a clear difference between the visual and the verbal discourse and both must work in unison to convey meaning. Under the latter, she proposes two main categories: augmentation   20 (subdivided into irony, humor, intimation, fantastic representation and transformation); and contradiction (subdivided into irony, humor and disclosure). Sipe (1998) uses the term synergy to illustrate that the meaning created by the picturebook as a whole is more than the sum of its elements. Images and words have a “synergistic relationship, in which the total effect depends not only on the union of the text and illustrations but also on the perceived interactions or transactions between these two parts” (pp. 98-99). The distinct nature of verbal–which is predominantly sequential–and visual–which is predominantly spatial–modes generates a complex process of meaning-making which has multiple paths and invites re-readings, generating multiple stories in one. Sipe discusses this process in semiotic terms using Suhor’s (1984) concept of transmediation. Images and words present distinct semiotic triads of object, representamen and interpretant that combine in the multimodal text. We interpret images in terms of the words and vice-versa, resulting in the creation of new circles, different and more complex than the meaning conveyed by words or images alone. The process is always shifting, generating endless possibilities of meaning. Sipe’s analysis is interesting because he explains in semiotic terms the transactions between one mode and the other–that Nodelman (1988) calls ‘interplay’ and Lewis (2001) calls ‘interanimation’–and how, depending on whether readers read the images as a primary text and the words as modifying it, or vice-versa, the meaning they extract from the text might be different, producing multiple interpretations of the same story. He does not go further, however, in how the form and the content of a specific text influences this relationship, suggesting a certain position to readers. These aspects are discussed extensively by Nikolajeva and Scott (2000, 2001). They base their understanding of the picturebook on the Hallberg’s (1982) concept of iconotext, “an   21 inseparable entity of word and image, which cooperate to convey a message” (Nikolajeva, & Scott, 2001, p. 6), and analyze in depth how this complex cooperation establishes setting, characterization, point of view, spatiality and temporality in these multimodal narratives. They categorize five types of relationship between images and words in picturebooks. The most common, which they also consider the least sophisticated, is symmetry. Symmetric picturebook narratives occur when words and pictures tell the same story; they are equivalent. Nikolajeva and Scott understand that in this concept of picturebook, the illustrations don’t go beyond the information conveyed by the verbal text, they only repeat it, being then unnecessary for the understanding of the narrative. Enhancement occurs when one of the modes expands or amplifies the other but this interaction can happen in various degrees. In some cases there is little difference between the information conveyed by the text and the one conveyed by the image, while in other cases this expansion is extreme, which the authors call then a complementary relationship. On the other hand, in picturebooks the multimodal narrative may be told by the contrast of images and words, each of which tell distinct stories; their relationship is then one of counterpoint, in which they “collaborate to communicate meanings beyond the scope of either one alone” (p. 226). Counterpoint can be expressed in different degrees. When this relationship is extreme, the authors call it contradictory. Yet, a picturebook may have parallel stories that overlap. They may be told only by the images or by the interaction between image/text. These picturebooks are called sylleptic. Nikolajeva and Scott recognize that the iconotext is complex and many distinct relationships can be used to build a narrative. They are especially interested in the relationship of counterpoint, which they posit can manifest in different ways: counterpoint in address (promotes distinct reading by the dual audience of picturebooks: children and adults), in style (e.g., one text   22 is humorous and the other not), in genre (one is fantasy and the other realism), in perspective (the words and images represent the POV of different characters), in characterization (the information about the characters is not the same in each mode or one mode represents characters that the other does not), in time and space (they always contradict in this case, because images show while text tells the narrative). Finally, there can be counterpoint by juxtaposition, when words and images tell parallel stories, and counterpoint of a metafictive nature, which can happen when one cannot represent notions present by the other or words and images utilize the paratexts to build contradiction. As pointed out by Nodelman (2003), there seems to be a bit of an inconsistency between Nikolajeva and Scott’s categories and the analysis that they make of the picturebooks in their book. As discussed by Schwarcz (1982), Nodelman (1988), and Lewis (2001) due to the distinct qualities of image and text as modes of meaning-making, it is simply impossible that they mirror each other completely. Although Nikolajeva and Scott (2001) have a category where hypothetically this could happen, in their analysis of picturebooks they take into account the characteristics of each mode and the impossibility of complete symmetry. For example, when discussing how characterization can be built in the multimodal text, they mention that physical characterization is usually expressed by images, while emotional, psychological or philosophical characterization is usually embedded in the verbal text.  Besides this inconsistency, Nikolajeva and Scott’s work provides extremely valuable insights into how meaning is created in picturebooks through these complex image-words interactions. They suggest a metalanguage for the analysis of picturebooks based on these relationships, which if used with attention and avoidance of generalizations, is extremely useful in the analysis of the different aspects of the picturebook narrative.    23 Lewis (2001), on the other hand, discusses the picturebook through a more holistic perspective.  He discusses not only the image-text relationships but also the key characteristics that constitute the picturebook form as a whole in contemporaneity. He establishes a distinct theoretical framework that takes into account the picturebook as an ecosystem in which complex relationships take place and each element has a specific and fundamental role in the total meaning of the narrative.  Throughout the book, Lewis ideas on the relationship between text and image emerge, but he problematizes this discussion in the chapter entitled “The interaction of word and image in picturebooks: a critical survey” (pp. 31-45), where he revises the metaphors and taxonomies presented by different authors.  Based on Nodelman’s (1988) proposition that “although pictures and words in close proximity in the picturebook influence each other, the relationship is never entirely symmetrical” (p. 196), Lewis criticizes the taxonomies proposed by other scholars. He understands the creation of taxonomies is problematic in general because they cannot always be applied to the book as a whole; they represent types of relationships that can happen in picturebooks, but in most cases many of them are combined within the same story to generate original and sophisticated narratives. Lewis is highly influenced by Nodelman’s work, which he synthesizes in the following passage: “the words in the picturebook tend to draw attention to the parts of the pictures that we should attend to, whereas the pictures provide the words with a specificity–colour, shape and form–that they would otherwise lack” (p. 35). The interanimation between words and pictures and between pictures and words, or the transformation one affects on the other, is what constructs the narrative in a picturebook.    24 The variety of possible relationships is a result of the combination of flexible forms of texts and diverse image possibilities. He proposes, then, a phenomenological approach he calls “the ecology of the picturebook” (p. 46). Employing ecology as a metaphor makes possible the understanding of the diverse and complex elements and relationships within picturebooks: “In claiming that picturebooks possess an internal ecology we are not claiming the exact same relationship of word and image for each and every picturebook” (p. 47); the connections and the interanimations between these multiple elements are what bring the stories to life. Lewis claims that interanimation also occurs directly between the reader and the text (p. 57): the story is generated by the flow from text-to-reader but also–and at the same time–from reader-to-text. The interanimation is made by “an active, meaning-seeking reader” (p. 55); in the multiple possibilities of texts and images, conception depends on the reader’s context and active participation. These concepts have been extensively discussed in transactional or reader-response theory (Rosenblatt, 1978) and are in accordance with multimodality’s notion of design (Kress, 2010; New London Group, 2000), which will be discussed in another section of the literature review in more depth.   As seen from the authors presented above, in the discussion of image and word relationships and how they create meaning in picturebooks, there are two main streams in picturebook scholarship. The first proposes taxonomies that describe the different ways and degrees in which image and text are in consonance or contradiction (Agosto, 1999; Golden, 1990; Nikolajeva, & Scott, 2001; Schwarcz, 1982). The second group focus on the characteristics of images and texts and how they transact to create meaning (Lewis, 2001; Nodelman, 1988; Sipe, 1998). Both models propose interesting insights on how picturebooks   25 work, but it seems to me that, when applying them to the understanding of picturebook apps, whose texts are even more complex with the inclusion of other modes, a more holistic approach allows us to have a better sense of these kinds of texts, while applying the taxonomies would be exhaustive and probably inconclusive. Nonetheless, the different types of relationships described by the first group of authors may be useful in describing how the different modes contribute to the generation of playfulness in the picturebook.   2.3 Playfulness and the picturebook The notion of playfulness is a rather complex one that has been explored by different fields through different perspectives. This literature review is not exhaustive, but tries to bring a multidisciplinary approach to this study combining a sociocultural perspective (Huizinga, 1949), a psychology/child development/sociocultural perspective (Vygostzky, 1978), an educational/literacy perspective (Mackey, 2002) and a literary perspective, using some authors that discuss playfulness and the picturebook (Kanatsouli, 2012; Lewis, 2001; Nikolajeva, 2008). J. Huizinga’s Hommo Luddens (1949) is a seminal text in the field of ludology, the study of play. One of the main contributions of his ideas to this work is in the proposing a definition of play–which none of the other works in this review actually do; play is “a voluntary activity or occupation executed within certain fixed limits of time and space, according to rules freely accepted but absolutely binding, having its aim in itself and accompanied by a feeling of tension, joy, and the consciousness that it is ‘different’ from ‘ordinary life’” (p. 28). They key aspects of play for Huizinga, then, are: • Play is voluntary, and can be suspended at any time. • Play is not real life, and is disinterested.   26 • Play involves tension and/or joy. • Play has limits in time and space, and it can be repeated. Taking an aesthetic perspective towards children’s literature Huizinga’s definition of play could even embrace the very definition of children’s literature. Nonetheless, it is arguable that the voluntary and disinterested aspects are not present in the way many children have access to children’s literature from parents and teachers, who often show a very didactic and utilitarian approach to children’s fiction. Huizinga himself makes this connection between art and play. He claims that due to the “profound aesthetic quality” (p. 2) of play, play and art are very connected, “for although the attribute of beauty does not attach to play as such, play nevertheless tends to assume marked elements of beauty” (p. 7). In literature, metaphors are a form of play, because through them a secondary world is generated. Also, “the rhythmical or symmetrical arrangement of language, the hitting of the mark by rhyme or assonance, the deliberate disguising of the sense, the artificial and artful construction of phrases–all might be so many utterances of the play spirit” (p. 132). Lev Vygotsky (1978) understands play from the psychology and child development perspective. He posits that play is not necessarily related to pleasure, since children can have sources of pleasure other than play, and not all forms of play lead to pleasure. For him, play and imagination can work as synonyms. A child begins to play in order to realize, through imagination, desires that are unrealizable in real life. While playing, children adopt a set of rules that will regulate their actions, and these actions rely on the meaning they are trying to convey. In real life, on the other hand, actions come first, and meaning is created based on these actions.  Considering Vygotsky’s perspective, it is possible to conclude that while reading and navigating a narrative through imagination, children are already in the territory of play, since   27 they have assimilated the conventions of books and understood the distinctions between fiction and real life. Although Vygotsky understands play in psychology terms, he sets up some aspects that are central to any discussion about play: imagination, actions and rules. Margaret Mackey (2002) has mapped the ecology of literacy by studying children from grades 5-6 and 8-9 and their engagement with texts in different formats and media, such as picturebooks in print and DVD format, videogames, novels in print and e-book formats, illustrated or not, movies in DVD format including its extras. In the ecology of texts, she considers “to play” as a term that puts the reader in a participatory position and better expresses “the liveliest connections with our contemporary textual ecology” (p. 181); “to play” is an action verb that can be used in establishing readers’ engagement with any kind of text.  As the conclusion of her study, Mackey discusses each of the expressions of play and how all these forms are in some ways present while reading any kind of text. Many of these aspects are entangled, influencing one another. • Play as imagination is the most basic definition of play. It involves “stepping inside the fictional universe” (p. 182), assuming the fictional world as real during the duration of the “game.” From this perspective, every work of children’s literature is playful. While reading, no matter whether the narrative is fictional or based in real life, players enter this imagined world, accepting it as a momentary truth, moving toward a future that, although is (usually) already written and concluded, is lived as unknown. This element seems to be central in any definition of play, being crucial also in the ideas of Huizinga (1949) and Vygotsky (1978). • Play as performance “involves some kind of bodily immersion in the activity” (p. 183). Engaging with any kind of text involves performance. Nevertheless, in many   28 cases such as while reading a book, we take our body’s engagement for granted, ignoring how we engage our hands, eyes, balance and breath in the process. In videogames, the performatic element is more obviously present, since the player is the character in the narrative and his/her movements derive from real movements performed by the player’s hands, in a more subtle way with a computer keyboard, a bit more explicitly with videogame controllers, and completely performatic in the case of some Xbox or Nintendo Wii games, for example. In Vygotskian (1978) terms, performance can be understood as children’s actions while playing, which are realized with the intention of creating meaning. • Play as engagement with rules considers that rules are one of the key elements of play–as also discussed by Vygostky (1978): “Engagement with a text of any kind entails accepting and working the rules and conventions in some way or another” (p. 183). This engagement is closely associated with the imaginative aspect of play, since for immersion into the imagined world, it is necessary to understand and accept the rules and conventions of that specific text entirely. As Nodelman (1988) claims, by being exposed to books from birth, much before being able to read, children learn the conventions of books and fiction. • Play as strategy refers to reflecting and making choices while engaging with a text, especially towards the performative aspect of players’ engagement in this imaginary world. Referencing Steven Poole (2000), Mackey explains that “imagining how” is a prerequisite of “imagining into” (p. 185). Again, this aspect of play is more explicit in some texts such as videogames, in which players are required by the text to make somewhat conscious choices and plan their journey through the narrative. But in fact,   29 playing any kind of text involves strategy. In picturebooks, going back and forth when noticing an element in the illustration that was considered secondary in the first read is one less-explicit example.  • Play as orchestration refers to our capacity to combine different skills and literacies while interpreting any kind of text: “we orchestrate many activities for even the simplest form of text interpretation, and we must manage our affairs so that many of these activities can be conducted automatically” (p. 185). In multimodal texts, we have to integrate signs from distinct meaning-making systems into one final multimodal text. As discussed above, this process has been extensively studied by picturebook scholarship through the lenses of both semiotics and reader-response theory. Nevertheless, even a simple text only composed of words is multimodal and involves multiple literacies to be interpreted. • Play as interpretation–Mackey refers to interpretation in the sense of making one’s own personal retelling of a text, which is very connected to the performatic aspect of play, as it is in the interpretation of a song or a dramatic text. • Play as fooling around, which I prefer to call play as experimentation, is Mackey’s reference to the trial and error aspect of play, in which new possibilities are played out without worries about its possible consequences: “the capacity to experiment with texts, to fail sometimes, to try again without recrimination or penalty, and/or to abandon them, is an important part of mastering new media” (p. 187)–similar to Huizinga’s (1949) claim that play is free, voluntary, and can be abandoned at any time. While this process is present in the engagement with any kind of text, especially for kids, who, at one time or another, face a new kind of text for the first time, it is   30 particularly relevant in the engagement with texts in emerging media formats, in which experimentation is part of mastering the rules and conventions of the medium. • Play as not working, which I prefer to call play as leisure. Finally, play “demands a consequence-reduced zone” (p.187). This last aspect of play is connected to the previous one, as it is a condition for experimenting. We can only experiment with texts because there isn’t a direct need for results. Although play is in fact taken very seriously by the players, since, as claimed by Huizinga (1949), just this seriousness permits the imaginary game effectively to take place, the freedom and flexibility inherent in children’s play relies on the fact that play is not work, or that it does not require that something be achieved at the end. Play at leisure is also associated with free time and the choices that we make to fulfill this time, the kinds of texts that we choose to engage with among the numerous possibilities available, and the level of attention that we put into those texts, which can vary significantly to the level of attention we put into texts that are work (and thus not play). Mackey then connects the different elements of play discussed by previous authors with children’s engagement with texts in a non-academic context. This synthesis explored by Mackey is especially useful in this work because the texts being analyzed here are a new format that has been integrated in the ecology of children’s texts. They manifest sometimes similarly to and sometimes differently from the examples she discusses, but having a point for comparison is extremely useful for the analysis. According to Lewis (2001), playfulness is an intrinsic characteristic of most picturebooks. They are playful, but “not in the sense that they are necessarily fun or funny–although of course they often are–but in the sense that they do not take settled forms of proceeding, such as genre   31 conventions and the usual strategies for structuring narrative, as wholly binding” (p. 81). The ludic experience in picturebooks has also huge educational potential, as children “[familiarize with] the world through laughter” (Bakhtin, 1981, p. 23). Lewis, on the other hand, also says, “the breaking of rules and the flouting of conventions [in picturebooks] … suggest a strong connection with forms of play” (p. 81). Thus, both the assimilation of rules, as its subversion in picturebooks are forms of play. The first may imply pleasure in itself but it does not necessarily imply amusement, while the breaking of conventions seems more often to be associated with amusement and fun. For Nikolajeva (2008), play and playfulness manifest in two aspects of the postmodern picturebook: in its materiality and in its word and image relationship. For the first aspect, she posits, “In postmodern picturebooks, playfulness is often expressed through their materiality, their quality as an artifact” (p. 57). Materiality refers not only to the more tactile aspects of a book, such as the paper or other material in which it is printed, size, format, if it is in colour or black and white, etc., but also to its conventions as an object, which includes its paratextual elements. These elements are fundamental in creating meaning in the postmodern picturebook, and its manipulation by picturebook creators is fundamental in the building of metafictional texts. In the postmodern picturebook, as she exemplifies with different books from Danish literature, the paratexts, such as half and full title pages, dedication and CIP data can all be used as part of the narrative to discuss the book as an artifact and discuss the distinction between fictional and real worlds. While examining the playful aspects made possible by the exploration of the materiality of the picturebook, Nikolajeva also discusses interactivity and how it enhances the metafictive and playful aspects of picturebooks. In the case of pop-up books, paper-  32 engineering techniques “add to the playful dynamics and demand a certain degree of interaction to engage the viewers and make them co-creators” (p. 67). Another aspect of playfulness in postmodern picturebooks relies on the relationship between words and images. The dynamic meaning-making process necessary for interpreting the picturebook would be by itself playful, especially if their meanings contradict each other: “the counterpoint of word and image inherent to postmodern picturebooks, is in itself, a playful element” (p. 55). This active process would also make any picturebook interactive, since “interactivity of a more subtle nature lies in decoding complex iconotexts that require the viewer’s full attention to appreciate the visual signs as well as the word/image interplay” (p. 67).  Nevertheless, in postmodern picturebooks, words and images do not only interact one with the other but also can reference other texts and images from art, literature and culture in general. Nikolajeva points out that these references also contribute to the playful aspects of postmodern picturebooks: “intertextuality, or rather, intervisuality as a more appropriate term in connection with images, references to well-known works of art, naturally creates a playful atmosphere” (p. 67). For Nikolajeva, then, playfulness is an essential quality of picturebooks–postmodern in particular–that can manifest in different ways, more than one of which may be present in the same book. Although she never really defines playfulness in her study nor goes deeply into other scholars that have discussed the subject, with her deep understanding of picturebook theory, she has rich insights into how a picturebook can be playful. Meni Kanatsouli (2012) claims “all children’s books inevitably involve play” (p. 39). Based on Sutton-Smith’s (1997) four ways in which literature and play interact with each other, the author discusses a few contemporary Greek picturebooks.    33 Literature as play manifests in the transition from reality to fantasy or from literal to metaphorical. As also debated by Huizinga (1949), Vygotsky (1978) and Mackey (2002), considering that both literature and play involve the creation of an imaginary world, “as a broad generalization as it is to say that all art, in this case, all literature, is play … there is no doubt that the end result of many children’s books is transformed into play” (p. 35).  Considering “whether as a metamorphosis or a paradox, the meta-communicative nature of play denotes that play is not merely play, but also a message about itself (a meta-message)” (p.36), metafictive narratives enter the category of literature with playful content. In this case, the author highlights that readers must be aware of the conventions of literature as an art form and of the book as a support for literature. When these conventions are subverted, the game starts being effective. In the case of fractured fairy-tales, for example, readers must have the knowledge of the original text for the game to exist. Other elements typical of metafictional texts, such as intertextuality and parody, lie on this premise: “inter-textual play’s aim is to turn the attention of the reader to the play of texts, how they interact, and what the inter-textual relationship is” (p. 36), while parody is one key element of child’s play, since children parody adult behaviour in their imaginary games. Finally, Kanatsouli remarks that books that represent the shape of other objects have their function transformed, since they “may not only be read, but also played” (p. 37). Some literary genres are based on language play and its capacity for generating humor, such as non-sense verse, tongue-twisters, jokes and other language games. They are part of what Sutton-Smith (1997) call play that is literature. They accompany children at every stage of language development, becoming more sophisticated on the way, such as, on her example, in literary works that explore neologisms, relying on children’s capacity for combining the meaning   34 of different words to produce a new, often humorous, one. Sutton-Smith was probably discussing purely verbal literature, but since Kanatsouli is trying to apply their theory to picturebooks, she failed in discussing the visual aspects of the picturebook text, and does not consider the how an image can be a game in itself, for example when there are elements hidden throughout the narrative or when surreal images question readers’ sense of fiction and reality. From a Vygostkian perspective (1978), play contributes to child development because it allows children to imagine and live situations that are not possible in real life: “This type of play is a transference … of a child’s need to explore the world through play” (Kanatsouli, 2012, p. 38). As mentioned, literature is also an imaginary space in which children can live different lives. Through surreal or metaphorical narratives, children are able to “live” and cope with realities and emotions above their level of development. Literary metaphors as play is then the last category mentioned by Sutton-Smith.   From the authors mentioned above, it is clear that children’s literature in general and the picturebook in particular have deep connections with play, and reading a picturebook can even be considered a form of play. While considering picturebook apps in comparison to print picturebooks, these elements remain, but they are affected by the changes in the materiality of the media in which they are conveyed, as well as in the addition of modes, which transform the ecology of the picturebook into an even more complex narrative.  2.3.1 Postmodernism and metafiction as playfulness Postmodern theories have been largely used to understand contemporary literary production, especially concerning books that reflect more explicitly some social and cultural   35 values understood as unique to the historical moment in which we live. Lawrence R. Sipe and Sylvia Pantaleo (2008) define postmodernism as “a general term to describe the changes, tendencies and/or developments that have occurred in philosophy, literature, art, architecture, and music during the last half of the twentieth century” (p. 1), and they comment on the lack of consensus about this definition, with some authors–such as Hutcheon (1988)–understanding it as a reaction to modernism and others–such as Watson (2004)–as its continuity. With regards to children’s literature, postmodern theory has been used to explain some of the transformations and innovations of picturebooks, especially the ones published from 1980s on. Many authors have enumerated the features of postmodern picturebooks, as Sipe and Pantaleo map in their introduction to Postmodern Picturebooks: Play, Parody and Self-referentiality, among them: Anstey, & Bull, 2004; Coles, & Hall, 2001; Goldstone, 2004; Grieve, 1993; Lewis, 2001; McCallum, 1996; Nikolajeva, 1988; Watson, 2004. Sipe and Pantaleo also synthetize different theorists and propose six groups of characteristics of postmodern picturebooks: 1) Unclear boundaries between pop and high culture, among literary genres, and among author, narrator and reader; 2) Subversion of literary traditions and conventions, and of the distinction between fiction and reality; 3) Explicit intertextuality, including pastiche and layering of texts from various origins; 4) Multiple meanings, ambiguity and open-endedness; 5) Playfulness; 6) Self-referentiality and metafiction. Technology lays at the origin of many of the postmodern features listed above, and the exploration of such connections is especially significant in the analysis of digital children’s literature. Dresang (1999, 2008) with her Radical Change Theory working parallel to postmodern theory in the analysis of young people’s texts, pays careful attention to three principles emerging from technology that pervade other aspects of contemporary culture and society: interactivity,   36 connectivity, and access. Interactivity can correspond in postmodern theory to the concepts of non-linearity, user participation or performance, boundary breaking, fragmentation, and playfulness. Connectivity can be associated with the idea of indeterminacy; the gaps in the narrative make possible an endless number of connections between children and the book and between children “in completely different times and spaces” (p. 47). Access can be associated with ideas of decanonization, intertextuality, parody, pastiche, and again, indeterminacy. Lewis (2001) disagrees with the term ‘postmodern picturebooks’, arguing that children’s literature does not manifest postmodernism as adult literature does: “The [“postmodern”] picturebooks in question rarely unhook themselves totally from mainstream literary norms and none possesses the apocalyptic, endgame quality … that is found in most postmodern art” (p. 99). Also, for Lewis, the metafictional characteristics of such books are not exclusive to postmodernity, and he proposes that we call such works metafictional instead of postmodern.  Lewis (2001) defines a metafictive picturebook as a book that can “comment upon, or direct attention to, the nature of fiction in the process of creating it” (p. 93). Although metafiction is an “a-historical notion” (p. 94), metafiction and self-referentiality are certainly considered by most postmodern theorists one of the most characteristic features of postmodern picturebooks.  2.4 Multimodality and children’s literature Although scholars in picturebook theory (e.g, Lewis, 2001; Nikolajeva, & Scott, 2001) rarely mention the influence of social semiotics and multimodality in their understanding of the picturebook, nor do they systematically use this framework on the analysis of children’s texts, their understanding of the picturebook as a multimodal text is in accordance with the principles of this theory.    37 Lewis (2001) believes that “the picturebook is a particularly flexible form of text and that pictures in general are extraordinarily diverse” (p. 44). Within one picturebook, different sorts of effects derived from the “interanimation” between words and pictures may be combined to create meaning. He proposes, then, the study of the “ecology of the picturebook” (p. 46), making it possible to consider the picturebook in its “flexibility and complexity.” In picturebooks, as in ecosystems, the elements are diverse and are all connected; any change in one of these elements results in changes in the ecosystem as a whole. When analysing picturebook apps, the addition of modes produces new challenges in the analysis of the picturebook. How do we make sense of a multimodal text that combines various semiotic resources that work so distinctively to generate meaning?  Lewis (2001) presents a broad understanding of the contemporary picturebook and his propositions suggest that most of what is considered innovative in picturebook apps is already characteristic of good print picturebooks. His concepts only need to be loosened a bit to encompass the digital features possible in the electronic format. His notion of interanimation can be applied not only to words and images but also to any other mode that is used to create meaning in a picturebook. So, it is possible to apply the ecological perspective proposed by Lewis to understand the interanimation of words and audio effects, images and animation, animation and interactivity, and so on, covering all the relationships present in the enhanced e-book. Another important characteristic pointed out by Lewis is the double orientation of the print picturebook; in other words, in the print picturebook, two directions are seen at once: the one of “words-as-influenced-by-pictures,” and the one of “pictures-as-influenced-by-words.” Similarly, we can consider the multiple orientations of the picturebook app when analyzing all the elements that help to build meaning in the electronic format.   38 Lewis’s (2001) ecology of the picturebook is an extremely useful general framework in the understanding of picturebook apps but does not take into account the analysis of the modes per se and how their similarities and differences build the picturebook narrative. Nonetheless, for a more detailed understanding of the possibilities and restrictions of the different modes in the construction of the picturebook app narrative, it is necessary to go deeper into multimodal theory. 2.4.1 General aspects of multimodal theory Multimodality as a theory of communication based on social semiotics considers that we express meaning through a variety of modes or semiotic resources. Jewitt (2009) states that multimodal theory is based on four key theoretical assumptions: language is part of a multimodal ensemble, it is not the only nor the most important mode; each mode realizes different communicative work, in other words, enables different forms of cognition; people orchestrate meaning through their selection and configuration of modes; the meanings of signs are social, they vary in different settings, periods and according to the social group of the individuals.  Theo Van Leeuwen (2005) posits “social semiotics compares and contrasts semiotic modes, exploring what they have in common as well as how they differ, and investigating how they can be integrated in multimodal artefacts and events” (p. xi). According to Kress (2009), a “mode is a socially shaped and culturally given resource for meaning making” (p. 54). Each mode has different meaning potentials, based on its history of use; it has a set of affordances, which indicate its possible uses; and these meanings are actualized in the social context, so modes are in constant change. Some aspects of a mode vary from one culture to another (for example, intonation in speech) while others are universal (for example, linearity in language).    39 In this context, sign-makers are active agents, who design the signs they create and read according to their interests and motivations. Readers are also sign-makers because they recreate these signs while reading according to their own interests: “reading is sign-making, in which the object which is being read forms the substance of the new sign-making by the reader” (Kress, 1997, p. 46). This is one of they key differences between social semiotics and traditional semiotics, which considers the system of signs as closed and the reader as a passive recipient of the signs. Interest is described by Kress (1997, p. 90) as  “a composite of [the sign-makers’] experience; but it is also a reflection of [their] present place, and an assessment of [their] present environment”; interest reflects the sign maker and his or her history, and his or her consideration of the communication environment, to which one adjusts the signs one is creating. Lewis’ (2001) theory is in accordance with multimodality as he also puts the reader at the center of the meaning-making process: “in ecological terms, the text–a specific combination of words and pictures–can only function … in the supporting context of a reader’s engaged and active attention” (p. 59).  While, in linguistics, form (syntax) and meaning (semantics) are analysed separately, in social semiotics they are considered together in the study of the sign as a whole. Kress (1997) states, “Form is the best possible guide to meaning (because makers of signs seek to find the most apt form for expression of meaning); the internal organization of the sign is the best possible clue to the organization of the meanings of the sign-maker” (p. 102), so analyzing form in each of the modes at work in modal ensembles is fundamental to understanding their meaning.  Derivative of M.A.K. Halliday’s (1978) systemic functional grammar of language, Kress and Van Leeuwen (1996) state that all semiotic modes present three major metafunctions: the ideational or representational, with the “function of representing ‘the world around us and inside   40 us’,” or that aspect of this world which is represented in the text (Kress, & Van Leeuwen, 1996, p.13); the interpersonal or interactive, with the “function of enacting social interactions as social relations,” or the interaction of this text with an external participant–the reader (Kress, & Van Leeuwen, 1996, p.13); and a textual or compositional metafunction, which is the capacity to form texts that are coherent both internally and within the context of production. Kress (2009) describes some general characteristics of various modes. Writing is linear and sequential; it is a very structured mode, with lexis, syntax and grammar and which uses such resources as punctuation, spaces and layout to frame and arrange its entities of meaning. Speech is quite similar to writing in linearity and sequentiality, as well as in grammar, but it makes use of different semiotic resources, audial instead of visual, such as rhythm, intonation, and silence. The visual mode, on the other hand, is spatial and simultaneous. Its units are framed through resources of size, colour, line and shape. Van Leeuwen (2007) compares the general qualities of sound, including speech, and of image. Sound, like writing, is linear, but–differently from modes that have visual representation–it is immaterial, and, although, in the case of picturebook apps, sound is recorded and can be replayed, it is also evanescent. It is to be experienced and remembered rather than possessed. It envelops the listener, and the “reading” of audio is collective and invites participation, while the “reading” of writing and images is individual and requires focus. The modes of gesture or the moving image combine time and space and, like audio, are transient: once they are over, no trace remains. Kress (1997) also discusses the concept of synaesthesia, which refers to the capacity to change meanings from one mode to others to communicate in a complex integration of all modes. Depending on the intention and motivation of sign-makers, certain mode or modes may be selected to convey meaning. When facing a modal ensemble, each mode carries different   41 ‘information weight’ and performs different functions in the text as a whole, according to its affordances. As modal ensembles, different picturebook apps may attribute more weight to one mode or modes over another. It is certain, nonetheless, that the additional modes, in comparison with the codex format, change the weight traditionally attributed to writing and visual modes. Social semiotics, then, provides useful tools to analyzing how the different modes work in the modal ensembles which are picturebook apps to construct meaning and, in the case of this study, how they contribute to the their playful aspects. Although a lot of discussion has taken place regarding how to analyze the images in a picturebook, social semiotics is especially useful because its principles can be applied to any mode of meaning-making, in turn permitting the analysis of each mode of picturebook apps through the same framework and principles.  2.4.2 Writing  In Western societies over the last few centuries, the writing mode has had supremacy both regarding its social and historical importance in consideration of the various theoretical frameworks that were developed to explain its functioning.  With regards to the metafunctions, in language the ideational metafunction is realized through the system of transitivity, which includes lexis–identifying the represented participants (nouns) and their characteristics (adjectives), the processes (verbs), and circumstances of place and time (adverbs)–and grammar–how these elements are organized in the sentences to affects the meaning. The interpersonal is realized through the system of person, which can be inclusive (we) or exclusive (I) and attributes a role of demand or gives to the speaker at the same time as suggesting the role to the recipient of the message, and a system of mood, which establishes whether the message is a statement, a question or a command. Modality, or how a text expresses   42 truth or reliability, is also included in the interpersonal metafunction. Finally, the textual is realized though the resources in the text which promote cohesion and the flow of information, such as articles and pronouns (Kress; 2010; Kress and Van Leeuwen, 2002; Downing, & Locke, 2006). 2.4.3 Visual  In the last decades, we have been facing a dramatic change in the way we articulate the different semiotic systems available for communication. The visual has been progressively regaining importance and being used to communicate certain types of meanings that used to be conveyed mostly through writing. The emergence of the picturebook format in the 1960s is part of this process of “the rise of the image.” Kress and Van Leeuwen (1996) have proposed a grammar of visual design, in which they discuss how images fulfill the ideational, interpersonal and textual metafunctions. In visual analysis, the ideational is understood in terms of its representation, while the textual, in terms of its composition.   2.4.3.1 Representation and interaction Kress and Van Leeuwen (1996) discuss how the producer of an image and the reader communicate through the representation of an image. They acknowledge the existence of two kinds of participants in images, the represented participants (depicted in the images) and the interactive participants (people who communicates with each other through images – the producer and the viewer). While the producers adhere to values and conventions of the audience and the institutions that circulate their work and they are active participants, readers bring with them their own values and conventions but are passive.   43 Interactive meanings have three simultaneous occurrences that can confirm or contradict one another: contact, which corresponds to the presence of gaze (demand) or absence of gaze (offer) from the represented participants toward the reader; social distance, which corresponds to how through the use of different shots (close shot, medium shot and long shot) the represented participants can enact different social relations with the reader (respectively, intimate and personal, social, or impersonal); and finally the attitude the represented participants may have toward readers, which can show subjectivity or objectivity. In subjectivity, this interaction can be of involvement, if the image was taken from a frontal angle, or detachment, if from an oblique angle. The level of the camera can also imply power relations between the represented participants and the reader. If the camera shows a high angle, or looking from top to bottom, the reader is in a position of power, while with the opposite, the represented takes this position. If the image shows an eye-level angle, both share the same power, they are equal. Objective images refer to scientific and technical images and infographics, and they can show action orientation or knowledge orientation. In picturebooks, as in most sequential texts, the relationship between the depicted participants and reader may shift considerably, usually at every page. In most picturebooks the reader is watching the story unfold, so medium and long shots are commonly used. However, to the complete understanding of the interactions, it is necessary to consider the other instances, and in most cases the use of a frontal angle usually contrasts with the social distance, creating involvement of the reader with the story. Although in many cases using medium or long shots, there is an intimate interaction between the reader and the represented. In some metafictional stories, in which characters or narrator talk to the reader directly, or stories that require the participation of the reader in some way, the demand contact is usually used.    44 2.4.3.2 Modality Modality is part of the interpersonal metafunction. “Truth is a construct of semiosis” (Kress, & Van Leeuwen, 1996, p. 159), so ‘truth’ varies according to the person and his/her social and cultural backgrounds. Nevertheless, although we know photographs can be manipulated, in western culture they represent the highest modality, or the closest we can get to the ‘real’. When we see a photograph, we tend to believe it is the most accurate representation of the truth possible, and other images vary in their capacity to express the truth usually as they distance themselves from what we believe to be the realist colour, perspective, sharpness, etc. of a photograph. Modality varies based on the scales of modality markers: three are related to colour: colour saturation (full saturation to black and white), colour differentiation (diverse range of colours to monochrome), and colour modulation (diverse shades of a colour to unmodulated colour). The photograph, or highest modality possible, is not at the extreme, but somewhere in the continuum, so an image can be more or less saturated than a photograph, have more or less colour differentiation than a photograph, and have more or less colour modulation than a photograph. In all cases, it will present lower modality than a photograph. Other modality markers are contextualization (from white background to very detailed background), representation (from abstraction to pictorial detail), depth (from flat to maximally depth perspective), illumination (from the fullest representation of light and shadow to none), and brightness (from various degrees of brightness to black and white). All these markers work as independent scales, and one image can be of high modality in one marker and low in another. 2.4.3.3 Composition For the analysis of the spatial composition, Kress and Van Leeuwen (1996) propose three interrelated systems. The first, the information value, refers to the distinct value zones (left and   45 right, top and bottom, or centred) of the composition. An element of the multimodal text has its value and meaning changed depending on the where in the composition it is placed and how it relates to the other elements. In Western culture, most images can be divided in four quadrants, upper left, upper right, bottom left, bottom right and the relationship between each of these segments is very complex. Considering the huge influence our reading system has in the way we read any text, we tend to move from top left and end at bottom right, but how the elements are depicted may alter our reading path.  In some images, the left and right dichotomy is the strongest or most significant, this usually being the case with texts in the landscape format, which privileges the exploration of the horizontality of the composition. In these cases, the left tends to be understood as something given, already established, evident, commonsense, while the right would be understood as something new, problematic, contestable. This understanding evolves from the sense of sequence embedded in the movement from left to right that is also present in language. In sequential texts, what was new in one moment becomes given in the following moment.  In other texts, usually in portrait format, the value of the composition is more strongly established by the dichotomy between top and bottom.  The top section is usually idealized, generalized and prominent, while the bottom refers to the real, the action, and the practical. The top and bottom structure also interferes in the relationship between texts and images. The element appearing at the top section is usually taken first, as more important, while the element at the bottom is read in relation to the element at the top. This affects quite significantly the relationship between text and image in vertical compositions. So in a multimodal text with one image and one block of text, if the text appears at the top and the image at the bottom, the image   46 will have a propensity to work as an evidence or specification of the text. On the other hand, if they occupy inversed positions, the text would work as an elaboration of the image.  In picturebooks, however, the relationship between texts and images seems to be more complex than that, since on each page the text may be positioned at a different place and in many stories the text is embedded in the image. In picturebooks, both image and text work very intimately, and usually it does not make sense to analyze their position as independent, polarized instances. In many cases, when texts and images constitute two distinct elements in the composition, they are positioned side by side, then becoming subject to the tendencies described in the left/right composition. When the image is at the top and the text at the bottom, the latter may work almost as a caption. In such cases, the idea of the text as an elaboration of the image seems to make some sense. Finally, the informational value of composition can be analyzed in terms of center and margin. Center usually means the most important part, the nucleus, of the information, while the margin shows the secondary elements, dependent on the center. The values of center/margin can be combined with those of left/right and top/bottom. In these cases, the center usually works as a mediator between the polarized sections, and the center may not be the most important element. A significant change regarding the transmediation of a picturebook from codex to screen is that the strong division between left and right from print books disappears. Now, the center of the page, which had to be avoided before due to the gutter space, gains a great deal of importance and is widely explored to generate salience. In addition, most picturebooks, even if they have portrait orientation when closed, show landscape orientation when open; the double spread of a picturebook is very often used as one single landscape image. In the case of picturebook apps, nonetheless, the portrait orientation is most common. Also, the flexibility of using images in both   47 orientations within the same book does not seem to work naturally in picturebooks and is uncommon. However, the screen offers new possibilities of expanding the composition, and some books are already exploring these alternatives by using the scroll option and by letting the reader navigate through the image through interactivity or through the movement of an “imaginary camera.”  Salience is another important system in the analysis of composition. The elements in a composition may present different levels of salience. The difference in salience between them creates a visual hierarchy. Salience is important in sequential texts because it creates rhythm through the “alternation between successive sensations of salience” (Kress, & Van Leeuwen, 1996, p. 212). In the picturebook, then, salience is an important aspect of the narrative and contributes significantly toward creating a coherent and cohesive story. Salience depends on the complex relationships amongst size, sharpness of focus, tonal contrast, colour contrast, placement, perspective and cultural factors. Its perception is, then, subjective. The balance of the different levels of salience between the elements in a composition, together with the rhythm created in the sequence of images, directly affects the aesthetic pleasure generated on the reader by the text. Framing alters the relationship amongst the elements in a composition by connecting or disconnecting them visually. Framing can be achieved by isolating the elements using lines, changing colour or shape, or even by having an empty space or a gap between elements. On the other hand, it is possible to stress the connection between elements by employing vectors or repetition in colours and shapes. Framing is also related to the notion of rhythm, speeding or slowing the flow of a narrative. If a framing is very dramatic, it may cause a break in the narrative.    48 Still analyzing composition, Kress and Van Leeuwen (1996) discuss its linearity and non-linearity. For them, the “[readings] shape themselves to become sources of meaning” (p. 219) and convey a cultural message. Today, linear compositions are losing ground to an increasing number of non-linear compositions. In non-linear compositions, the reader is free to decide his/her own reading path; they are ‘interactive’. In picturebooks, if on the one hand the narratives are, most of the time, linear, the reading path inside every spread may vary a lot. Usually salience is used to guide the reader in the reading of the image, but since image and text are usually very connected, whether the reader reads the image first, text first, or has a general impression of both first depends in many cases on the reader. When reading the images, in some cases, the reader’s attention may be drawn directly to the protagonist, and there are few or no details or secondary information. The character may even be in the center of a page on a white background. In other cases, the image may be diffuse with many elements with similar visual weight, and the reader is free to explore them as he likes; every reading may be quite a new experience.   Kress and Van Leeuwen (1996) present numerous concepts that help to elucidate how images in picturebooks create meaning. Some of the limitations of their theory are that all the examples analysed constitute individual static images and/or compositions. Even when discussing linearity and non-linearity, their example is one single image, and when analyzing a movie, they stick to one shot, which certainly is not significant to the movie composition as a whole. They do recognize rhythm when discussing framing and salience, but they do not explore how sequences of images may configure very complex articulations of these systems.    49 Rose (2001) proposes a method for visual analysis in social sciences that considers three sites for meaning-making of images: the production site, the image itself and the audience site. Each of these can be understood considering its technological modality, the compositional modality and the social modality–including economical and political aspects as well. Kress and Van Leeuwen (1996) do not comment in their analysis how the technological aspects, both in production as in audience, interfere in the meaning of a text and how they limit certain forms of composition. The transposition of a picturebook from codex to screen results in profound transformations in its meaning, even without considering the additional modes that are usually present in these transmediations. Even on a screen, the numerous kinds of screens available and the different uses and relationships readers have with them may significantly change the meaning of an image. For example, when an interactive book is presented on a computer screen as opposed to an iPad’s touchscreen and as opposed to a cellphone screen, the great difference in size may change the way readers interact with the narrative.   2.4.4 Colour Kress and Van Leeuwen (2002) claim that colour can function as a semiotic mode since it can present the three metafunctions: colour can identify people, places and things –for instance colours used to identify a brand or a country–or represent ideas–for instance white used to express purity–(both ideational); colour can also serve as a resource used by a participant to affect another participant, for example in colour therapy, in which certain colours are used to relax or to energize people (interpersonal); finally colour can be used to create coherence, for example in a system of signs, in which each colour refers to a certain kind of information (textual). In some cases the three functions can be achieved at the same time, while in other uses   50 of colours the three functions may not all be realized. Colour as a mode, however, differs from other modes because it cannot exist on its own. It depends on the combination with other modes to convey meaning, as Kress and Van Leeuwen (2002) state: “it [colour as a mode] can survive only in a multimodal environment” (p. 351). Also, colour can be said to be a less articulated resource; while the concept of grammar usually assumes a set of rules accepted by all in a group, the meanings associated with colour are limited to small groups or even individuals.  The authors describe two affordances of colour: the first is association since the meanings of colour depend completely on its context of production and reception, including its cultural and historical use; the second affordance is called “the distinctive features of colour” (p. 355), namely value (grey scale from white to black), saturation, purity (whether a colour is pure or ‘mixed’), modulation (textured to flat colours), differentiation (from monochrome to a varied palate), and hue (scale from blue to red). “Any specific instance of a colour can be analyzed as a combination of specific values on each of these scales – and hence also as a complex and composite meaning potential, as we now demonstrate” (p. 355). 2.4.5 Typography Van Leeuwen (2006) problematized the lack of a systematic approach to the analysis of typography as a mode of communication. He claimed typography too can fulfill the three metafunctions: the ideational can be realized when fonts try to represent in their design a certain idea or sensation, and gives an example in which the text “dead man” comes in a font whose letter shapes are constructed with bones, and typography also “might ‘interpret’ or, you might say, ‘perform’ texts” (p. 143), attributing certain characteristics to the text, namely transmit a classic or modern feeling; typography can realize the interpersonal metafunction, for example, through the use of emoticons, that combine letters, numbers and especially punctuation and other   51 kinds of characters to connect the participants of a text; the textual is probably the most obvious metafunction realized by typography, since it can organize the text, attribute hierarchy and give different values according to the variations of font, weight, etc.  As with colour, typography is usually associated with other modes in modal ensembles. In the first place, typography is the visual manifestation of language and is usually associated with writing – and writing, too, can be said to depend on typography. Typography, nonetheless, is also regularly associated with colour and texture, and can be associated with volume, three-dimensionality, or movement (for example, in movie titles). In different types of texts, typography is commonly used to create salience –for instance a title in bold font– and framing –for instance, different articles or different parts of an article in a magazine may appear in different fonts. Finally, Van Leeuwen (2006) lists the distinctive features that are combined to express different meanings through typography. They are weight (fonts usually come in light, medium, bold weight, etc.), expansion (fonts can be condensed or expanded), slope (the differences between fonts that are more cursive-like, and fonts that are upright), curvature (if a font is more rounded or more angular, including the shape of its ascenders and descenders), connectivity (if the letters in a word are connected to each other or independent), orientation (if the letter forms orient towards a more vertical or horizontal line), and regularity (if the forms of the letters present some regularity or if each letter has distinct features, such as different kinds of serif or different ascenders). These features described by Van Leeuwen are limited to typography at the level of the character or the word, but he does not discuss typography at the level of a block of text or a page, which, in most cases, can be analyzed according to the framework he and Kress propose in the grammar of visual design.   52 2.4.6 Sound Van Leeuwen (1999) discusses the semiotics of the mode of sound, which he understood to include speech, music and sound in general. Differently from his work on other modes discussed above, the author does not describe the semiotics of sound based on ideational, interpersonal and textual metafunctions because, with sound, the resources can simultaneously realize more than one metafunction; they are not specialized. Another aspect of sound that differs from other modes is that “sounds can both present their source (identify who or what their source is and does), and convey what it means to be that source, or be like that source” (p. 35). The description of the semiotic potentials of sound was then based on its material aspects. Van Leeuwen proposes a very complex system that involves numerous variables. I will briefly summarize some of the key variables that are especially useful for the analysis of narration, soundscape and sound effects in picturebook apps. In sound, the distance between the represented participants and the recipients of that sign is realized through two variables: perspective and social distance. Perspective organizes the sounds in a hierarchical order and it is usually realized through loudness; louder elements are closer, and more important in the message. In this realm, there are three categories in which sound can be placed: figure is related to sounds that are in the foreground and are the center of attention for the listener, and that carry the most important meaning in the message; ground encompasses the sounds in the middle ground, and represent the setting or the context of a message–important in meaning, but carrying less weight than the sounds in the foreground; finally, in the field category, there is the soundscape, or sounds that are part of the environment but do not add meaning to the message.  In picturebook apps, as expected, the narration is usually in the foreground, and when the sound effects or the soundtrack is too loud, there is the sensation that it   53 is impeding the more important message: the narrative. Nonetheless, the elements may move from one position to the other during the narrative. For example, the music or the sound effects can be louder at the beginning of a story, for a few seconds, to create a mood or give a sense of the story’s setting, and then it is lowered when the narrator starts the story. Some sound effects activated through interaction, as a response to the reader’s command, are usually played in the foreground, and given maximal importance, sometimes interfering with the listening of the narrative. The scale of social distance is “able to create imaginary social relations between what is presented or represented by a sound and the listener” (Van Leeuwen, 1999, p. 27). Voice is usually the most meaningful element in indicating social distance, but it can also be realized through sounds that represent places or things. On one extreme of the scale, there is intimate distance, usually manifested by whispering or very soft voices. The scale progresses through personal distance, informal distance, formal distance to public distance, usually demonstrated by a very loud and shouting voice. As pointed out by Nugent (2013), some picturebook apps’ narrations show intimate to personal distance, which would be the equivalent to an adult reading to a close child, while others are closer to the other extreme of the scale, delivered in a more shouting voice, similarly to an actor performing to an audience, or to a storyteller in the public reading of a picturebook.  Sound quality can convey meaning based on the combination of a series of variables.  These meanings are associated through connotation, and they depend on the context, the participants and the associations they can make according to their education and culture. The variables listed by Van Leeuwen (1999) are as follows: the scale from tension to laxity usually represents different levels of alertness and stress; roughness to smoothness, friction and   54 irregularity; loudness to softness, different levels of power and distance; breathiness is usually associated with intimacy; nasality usually has negative connotations; vibrato can be understood as emotional or as lack of control; finally, pitch register, in the scale from high to low sounds, is a rather complex variable, because it can represent size (high sounds represent small instruments of elements, and low sounds represent bigger ones), but also has social and historical meanings, as high being associated with child and female voices and low to the male voice, and all the implications of this association can result in a sound message. Modality is a concept relevant to various different modes, namely writing, visuals, and sound. In this realm, sound can be abstract-sensory, through which abstract meanings are conveyed with emotive effect. Music fits in this category, although some types of music may have a stronger stress in the abstract aspect, as does classical music, than others do; the more impact it causes, the higher the modality. Music in picturebook apps is usually in this category; it tends to express a certain mood or feeling, but usually maintains a certain level of abstraction. Nonetheless, sound can be solely sensory. This happens, when a sound is created to convey a certain emotion, such as in advertising or the sound design in good suspense movies. The goal is to provoke a certain emotion from the listener, not to represent it. This kind of sound can and should be used in picturebook apps to expand the drama of the multimodal narrative. However, developers do not seem, in most cases, to be investing enough in the creation of a sound design that complements other modes in conveying certain emotions of the narrative. Sound can also be naturalistic, representing what one would hear in real life if in a certain setting and situation. These categories are not closed, and one sound can express more than one form of modality. Time is another important element in sound. Most sounds are measured, although there are sounds that are unmeasured. A measured sound has a pulse that marks its rhythm. Most Western   55 music has regular timing. Speech has rhythm but is less regular since each sentence has a different number of syllables and the pulses do not exactly follow a certain beat. In certain situations, regularity can be created in speech through the stressing of certain words and syllables, as in certain radio broadcasts, but the effect is usually of a monotonous and unattractive sound. In book apps, the narrator is usually expected to stress certain words according to the meaning in a rhythmic and engaging narration. 2.4.7 Interactivity As discussed in the sections above, interactivity between text and reader happens in all modes through the interpersonal metafiction. Interactivity, thus, is not itself a mode. In the realm of game theory, Ritterfeld, Shen, Wang, Nocera, & Wong (2009) define interactivity as “a property of serious games that allows for communication between an individual player and the digital gaming system through different forms of activities” (p. 691). In this thesis, interactivity will be considered from this perspective, which takes into account the active involvement of the reader in an activity, and not only the communication which is established between them according to the textual elements of a message (for instance, the gaze in the visual mode or the addressing of the reader through the pronoun ‘you’ in the verbal language). Most activities in picturebook apps involve bodily engagement from the reader in controlling the narrative through touch or other methods of activating a response from the digital system.  Finally, Stichnothe (2014) lists some of the functions of interaction in picturebook apps, which involve creating tension between the various modes of the narrative, moving the narrative forward, generating humour, adding information and proposing activities, making choices within the narrative, including reader’s audio recordings, and connecting through social network. Here, humour is considered as an effect of the contrast between the modes or a characteristic of the bits   56 of the story that are revealed through certain interactions, and not as one of interaction’s functions. In the apps examined for this study, due to the young audience they address, there is no integration with social media.   2.5 Summary This literature review covers three areas of study that are essential to the understanding of playfulness in picturebook apps: picturebook theory, and the authors who have proposed models for the understanding of picturebooks and how they work; ludology, covering scholars who discuss the concept of playfulness from various disciplines and authors who analyse the connections between playfulness and picturebooks, especially in the context of postmodernity; finally, the chapter covers some essential texts in multimodal theory and presents the terms central to the description of how the different modes create meaning and of their application in the analysis of picturebooks.   57 Chapter 3: Methodology 3.1 Introduction I start the methodology chapter discussing the theoretical framework that guides the analysis of how playfulness manifests in picturebook apps. The proposed framework comes to life in the combination of Nikolajeva’s (2008) and Mackey’s (2002) works and suggests categories through which playfulness manifests in these texts: through the interanimation among modes, through metafiction, and through performance, which can be explicit or implicit. The criteria for the selection of the eight picturebook apps analyzed in this study are also presented: out of 100 picturebook apps, one born-digital and one transmediated fictional apps were selected as quintessential examples of each of the categories. Lastly, I present the parameters established for the close reading of the texts, which involve describing the most significant meaning-making features of each of the modes separately as well as analyzing the ecosystem of each book as a whole in the light of the various categories of playfulness.  3.2 Theoretical framework Among the theories previously mentioned, it seems to me that Lewis (2001) has succeeded the most in taking into account the complex, playful, ever changing nature of the print picturebook. His ecology of the picturebook seems the most adaptable theory while encompassing the multimedia features available in picturebook apps. This study analyses electronic picturebooks through the lenses of multimodality, more specifically through Lewis’ ecology of the picturebook, analyzing how the different modes participate and interact in the playful aspects of picturebook apps.    58  As Mackey (2002) has discussed, any kind of text involves a myriad of forms in which play is involved: imagination, performance, engagement with rules, strategy, orchestration, interpretation, experimentation and leisure. Although all of them are always present, some are dominant and explicit in some types of texts. Nikolajeva (2008) points out that playfulness can be related to the materiality of the picturebook or to its image/text relationship. These aspects change significantly in the picturebook app because the size, the format and the material are fixed and there is not the possibility of using paper engineering to create real three-dimensional structures. Nonetheless, the technology permits the creation of virtual 3D images, flaps and bends that respond to the user’s touch virtually, but yet in the same fashion as in pop-up books. It also permits animated images and characters that can be controlled by readers, making possible a new form of embodiment that was already typical in videogames but is more rare in books.  Combining Nikolajeva and Mackey’s discussions on playfulness, I propose for this study four different ways through which picturebook apps manifest their playfulness, according to the dominant aspects of play they manifest: 1. Playfulness through the interanimation among modes. Although all multimodal texts involve in some way this kind of playfulness, some narratives rely more strongly than others on readers’ participation in interpreting the different modes and their interanimations to extract meaning and, sometimes, produce joy. While engaging in this aspect of play in picturebook apps, imagination, orchestration and interpretation are the dominant aspects. A classic print picturebook example of this kind of playfulness would be Jon Klassen’s I Want My Hat Back (2011). 2. Playfulness through metafiction: playing the rules of fiction. Literature is in itself play in the sense that it involves entering a fictional world and “living” it according to certain   59 pre-established rules. Nevertheless, the subversion of these rules in metafictional texts, usually with humorous effect, also constitutes a form of play. Here, imagination, engagement with rules and experimentation are dominant aspects. A classic example of this form of playfulness is the widely discussed print picturebook The Stinky Cheese Man and Other Fairly Stupid Tales, by Jon Sciezka and Lane Smith (1992). Playfulness through performance refers to the participation of the body while engaging with a book, in print or digital format. In picturebook apps, the engagement of the body is mandatory, since all apps include some form of touch-activated interactivity, but here we are considering narratives in which this embodiment is significant to the telling of the story. There are two main possibilities for performance in digital picturebooks: 3. Playfulness through explicit performance: readers should engage their bodies as if they are the characters and represent the character’s role. This happens when, for example, readers are required to make gestures that are exactly the same as performed by the character(s). Imagination and performance are dominant aspects in this type of play and the book here presents many characteristics that closely approximate it to a toy. The Jolly Postman or Other People's Letters, by Janet and Allan Ahlberg (1986), is the quintessential example of print picturebook that manifests this kind of interactivity, where readers must open envelopes and act as the characters receiving those letters.  4. Playfulness through implicit performance: the readers’ body is still engaged, not in executing the action, but in controlling the action performed by the character, as in a videogame. Here, besides imagination and performance, strategy seems to be one dominant aspect. Also, the book may approximate itself significantly to a game. In the   60 print media, this is the case of most pop-up books, for example, The Incredible Book Eating Boy: Pop-up Book, by Oliver Jeffers (2009).  3.3 Rationale for selection of primary texts The number of picturebook apps available today is enormous, and although there are no statistics, it is humanly impossible to be aware of everything that is available, especially because the number of apps reviewed by the specialized media is very limited. It is common in children’s literature to select awarded books by reputable institutions, but the fact is that the picturebook app phenomenon is very recent, few awards have been established and the apps awarded would not form a big enough corpus for analysis.  As part of the research project The Future of Children’s Texts, supervised by Dr. Eric Meyers, for whom I work as a research assistant, I have read more than 100 picturebook apps aimed at children from 3 to 8 years old. They were played on an iPad 4th generation retina display. Although Android devices have increased their participation in tablet’s market share, the iPad is still the most popular device, with a greater number of book apps. Also, this is the device available to the researcher at the moment.  The process of selecting the apps involved navigating through the most significant online picturebook app review databases and compiling a list of 200 picturebook apps recommended by such sources. The sources used were: Appysmarts, Best Apps for Kids, Children’s Technology Review, CM Magazine, Common Sense Media, Digital Media Diet, Digital Story Time, Kindertown, Kirkus Review, Mac News World, Parents’ Choice Awards, School Library Journal, Smart Apps for Kids, Teachers with Apps, The Horn Book, and Wired. These sources and the process through which they select the apps to be reviewed varied, but most of them   61 reviewed the apps by the request of the developers for a fixed-value financial compensation, and they stated this compensation did not interfere in any way with their evaluation of the app. It was acknowledged during the process that such sources represent distinct stakeholders, with varied levels of education and criteria for what constitutes a good app. Nevertheless, this diversity was considered beneficial for the study because the selection represents the various stakeholders involved in the ecology of children’s texts, such as publishers, parents, teachers and librarians, making our sample more representative of the apps that are being read and commented on by the population in general.  The 200 pre-selected apps were then filtered to100 apps based on the criteria that the app had to have been recommended by at least two–but ideally three–of the review sources. From these 100 apps, I have defined some parameters to select the final eight picturebook apps that are analyzed in this study: 1. They should be fiction storybooks. As mentioned in our definition, picturebooks can contain information or a story. For this study, those being considered are only texts that have a narrative. Despite certain information books making use of narratives to facilitate the reader’s understanding of the content, they were not included for consideration. The objective of making such a limitation is to maintain as a sample a corpus of works that have similar characteristics and thus allow comparison in similar terms. Also, playfulness can manifest differently in fictional and non-fictional texts since they have different objectives. All genres of fiction were equally considered.  2. Since the objective of this work is to compare how playfulness manifests in transmediated and born-digital apps, half of the apps should have a print picturebook as a precedent. In other words, they should have the same story, with the same pair of writing and illustrations,   62 in a previously published print picturebook. The second half of the sample should consist of books created specifically for digital media. It is interesting to notice, however, that it is very hard to find born-digital apps that are not based on a traditional tale–in most cases, fairy-tales. The apps selected for this study, then, reflect this phenomenon, two of them being retellings of the story of Little Red Riding Hood and another a retelling of The Three Little Pigs. Although they retell a classic tale, their images and writing (except for Lil’ Red, which is wordless) are original and created exclusively for this edition as a picturebook app. 3. Although, as discussed in the literature review, all children’s picturebooks can be considered playful in some way, for this study I was looking for examples that clearly demonstrated the four different types of playfulness available in picturebooks. Thus, I have purposefully selected for each type of playfulness one picturebook app that was transmediated from print media and one born-digital picturebook app. For this reason, the final sample of this work comprises eight picturebook apps. Although each book fits strongly into one category, all of them manifest different kinds of playfulness, and all these instances are being considered in this study. Thus, it will be possible to discuss each form of playfulness based on the analysis of the eight picturebook apps. After considering the criteria above, out of the 108 apps, I initially selected the following: Table 3.1 Initial app selection  Type of playfulness Transmediated App Born-digital App Interaction in decoding the multimodal text What Does My Teddy Bear Do All Day? Lil’ Red Metafiction Another Monster at the End of This Book - Explicit Performance The Heart and the Bottle The Three Little Pigs Implicit Performance The Trip: Little Critter’s Reading Adventure Little Red Riding Hood    63 The first change in this selection was to exchange the app Another Monster at the End of This Book (2011) for its antecendent, The Monster at the End of This Book, since they were based on the same principles of metafiction. Because in the selection of the 108 apps the reviews were our criteria and Another Monster at the End of This Book had received more reviews than its prequel, it ended up being selected. Nevertheless, for this study it did not make sense to analyze the sequel since it is very similar in content and form to the first book. Second, in the 108 selected apps, I could not find a born-digital app that fit the category of metafiction. As a researcher in book apps for the past year and a half, I knew the picturebook app Don’t Let the Pigeon Run This App!, by Mo Willems, had the characteristics I was looking for. Although this app follows a series of print picturebooks and follows the same plot structure as the print books, it consists of one original narrative (in fact, multiple narratives) created exclusively for the digital media. At first I was conflicted as to whether this book would be a born-digital or transmediated app, because it is very similar to its counterparts in the Pigeon series, especially Don’t Let the Pigeon Drive the Bus! (2003), yet it did fit the criteria of having original writing and illustrations created for digital media. Also, it is the first metafictional app to consider the format of an app as an equivalent to that of a book in the role of telling a narrative. Thus, the final selection of primary texts for this study is: Table 3.2 Final app selection  Type of playfulness Transmediated App Born-digital App Interaction in decoding the multimodal text What Does My Teddy Bear Do All Day? (2010) Lil’ Red (2012) Metafiction The Monster at The End of This Book (2010) Don’t Let the Pigeon Run This App! (2011) Explicit Performance The Heart and the Bottle (2010) The Three Little Pigs (2010) Implicit Performance The Trip: Little Critter’s Reading Adventure (2013) Little Red Riding Hood (2013)    64 3.4 Close reading of digital texts Close reading is a qualitative method largely used in the study of literature and other types of narratives. In the case of reading digital narratives, especially narratives that present multiple paths such as videogames, close reading can be extremely challenging. Bizzocchi and Tanembaum (2011) problematize the application of close reading techniques to new media. Close reading enhanced e-books is an intermediate situation between the close reading of traditional literature and that of video games. In some situations, it is unclear if the action performed by the “reader” is that of reading or that of playing, or if the reader should be called user or player. Since in this study picturebook apps are considered digital books, the term ‘reader’ is used to refer to the person decoding the diverse forms of text that compose enhanced e-books. Although the extension and complexity of interaction with e-books is not as problematic as with videogames, in some apps, the random appearance of certain texts and other elements occur, and the reader may have more than one path through the story, which require multiple readings and an extreme attention to details.  The close reading of the apps is structured then in a way that permits crossing the multimodal aspects with the playful aspects present in the apps. The modes that are part of picturebook apps’ multimodal text are writing, image (including illustration, typography and layout), speech (here present as narration), sound (including soundscape and sound effects), movement (here as animation) and gesture (here as interactivity).  Taking into account the ecology of the picturebook and that meaning only happens through the interanimation between modes, each of these modes is analyzed in distinct topics in order to facilitate its understanding but always in light of how they interanimate with the others modes in generating the playful aspects of such texts.   65 Also, although each book was selected because it consists of a quintessential example of one of the four types of playfulness that dominate the picturebook format, all of them present elements of at least one other type, and the richness and complexity of its playful aspects cannot be missed by this analysis. Thus, the close reading presents a section for each mode, which is analyzed in light of the four possible ways that its playfulness manifests, if that is the case, as shown in the grid below. Table 3.3 Method for close reading the picturebook apps Playfulness / Modes Interanimation Metafiction Explicit performance Implicit performance Writing     Visual     Speech     Sound     Movement     Gesture      Given that a text is “a gestalt of medium and message” ” (Bizzocchi, & Tanembaum, 2011, Medium, message, and poetics, para. 10) and its close reading may reveal the “poetics of a medium writ large” (Bizzocchi, & Tanembaum, 2011, Theories of reading, para. 14), it is expected that, by close reading these picturebook apps, some insights emerge as to how playfulness manifests in this new form of literature and how they differ in transmediated and born-digital apps.  The technical aspects of the apps were not considered in this analysis, although they might be mentioned in cases where they interfere in the understanding of the narrative.    66 3.5 Summary In this chapter, I have defined the theoretical framework for this study, which emerged in the crossing of Mackey’s (2002) proposition of how texts for children can be read/played, which positions the reader as an active participant, with the categories proposed by Nikolajeva (2008) of how playfulness manifests in postmodern picturebooks. In this chapter I have also described the process and criteria for selecting the sample for this thesis, which involved the analysis of more than 100 picturebook apps and the purposeful assignment of one born-digital and one transmediated app per category of playfulness. Finally, I discussed the difficulties of close-reading digital texts and established parameters on how these apps would be close read in order to answer the research questions, which involved the analysis of each of the modes present in each text.    67 Chapter 4: Findings: Close Readings of Eight Picturebook Apps 4.1 Introduction In this chapter, each of the picturebook apps from the sample of this thesis are given a close reading from a social semiotic perspective, considering each of the modes that build the modal ensemble constituting each narrative. The analysis of each book contains a synopsis of the story and a section for each mode. In cases where the connection between modes is very intimate in constructing meaning, these modes will be analysed together. The synopsis will briefly describe the plot and, in some cases, the reading modes available in the app since they interfere in how the presentation of the modes are presented for the reader.  4.2 What Does My Teddy Bear Do All Day? by Bruno Hächler and Birte Müller 4.2.1 Synopsis In What Does My Teddy Bear Do All Day?, a little girl wonders if her teddy bear comes to life when she is not looking. Intrigued by the possibility that Teddy has a secret life, she decides to spy on him and try to “catch him” in the act: she pretends to be asleep, looks through the keyhole, stares with binoculars through the window, hides inside the wardrobe; she teases him by putting his favourite show on the TV, but yet he does not move; her final strategy is to leave the honey jar open. As it turns out, the bear cannot resist and she finally catches him with a full belly as he moves to clean his mouth after having eaten the whole jar of honey. Parallel to the central narrative, there are other narratives, conducted by her other toys, who are alive and move without her noticing them.   68 4.2.2 Writing The writing in this narrative is simple, short and rhymed. On most pages, the story is conveyed in four verses, using a rhyme scheme of abcb, often accompanied by one or two extra lines. The verbal narrative is in first person, showing the girl’s point of view. The language is informal with marks from oral communication, for example, interjections such as “Oh, no!” (screen 12).  The dubious relationship between the girl and the bear, manifested through the contrast between the sentences “I love my teddy bear” (screen 3) and “he seems really sweet / but maybe I’ve been fooled” (screen 2), is the premise of the narrative, which will develop upon her desire to know “the truth.” In the first part (screens 1 to 4), this problem is being established; the girl is wondering what activities her teddy might do, asking questions to herself as well as to the reader, since there are no markers to solve the dubious addressee of the questions. She is counting on the reader–an intimacy created through the use of informal language–to help her solve the mystery. In this part, words and images mirror each other, and Teddy is shown representing the actions mentioned by the verbal text. On screen 5, the central part of the narrative begins as the girl establishes the game with Teddy, which is marked by the sentence “You wait. I’m gonna catch you!,” repeated from screens 5 to 11. This game is constructed in the counterpoint between what she is saying and seeing and what readers are seeing. As the game progresses, the verbal text alternates between telling the story to the reader–the poem–and addressing teddy directly–“You wait. I’m gonna catch you!.” The shift to the conclusion (screen 11), is marked in the written text by a change in the pattern to “And now I’m gonna catch you!” [my emphasis]. At this point, the girl is certain that she can catch Teddy, until finally, on screen 13, the game is solved: “See… I   69 caught you.” The contrast between verbal and visual ends as the text shows Teddy with a big belly, information mirrored in the verbal text. 4.2.3 Narration The narrator in this app has the voice of a little girl, probably around seven years old. The voice matches the writing, emphasising the first person point of view of the verbal text; it also promotes the empathy of child readers. The narrator’s voice shows personal distance: it has a natural, average tone and volume, as when friends talk to each other, matching the writing’s informal and oral language register. The intonation, however, shifts from one part of the text to the other. The poems in every scene are spoken in a neutral tone, which denotes a slightly more impartial and distant communication in relation to the other parts of the written text; yet, these lines are musical since they have regular timing and rhymes. In these verses the girl is telling the story, mostly using past tense, which matches this slightly more distanced tone of the narration. The extra lines that accompany the poem in almost every scene have a more emphatic and expressive tone, which denotes a very playful and personal communication since through these lines the girl is teasing Teddy as part of their hide-and-seek game. 4.2.4 Illustrations The illustrations from this book are bright and colourful, built with simple forms and shapes that are not realistic, yet easily recognizable. The visual narrative includes various elements, including characters not mentioned in the verbal text, but that are necessary to the progress and concatenation of the multimodal narrative. The doll and the mice, which are not mentioned in the writing, are important to the narrative as a whole because their existence and movements indicate that the toys in this story indeed have a secret life. The presence of the mice, in various instances with little salience or even hidden, creates a playful parallel narrative, which   70 readers may want to follow. The doll is a central character in creating the irony between verbal and visual narratives. She has a privileged position, usually portrayed close to teddy or to the girl, contrasting with the mice, which are often “thrown” into some part of the scene. Yet, the doll does not receive the same attention as the bear, and this is essential in the construction of the playful narrative. Since the girl’s focus is on Teddy, the doll can circulate with certain freedom without being noticed, in fact putting herself in situations in which she could easily be “caught”–opening the door on screen 9 and lurking right behind the girl on screen 7–generating tension in the narrative. Through the combination of the doll’s actions and the mice’s movements, readers become accomplices of the toys’ secret lives; this combination of verbal, visual, and kinetic elements allows the reader to confirm that the toys have a life behind the girl’s back. 4.2.5 Animation Most animations in this app are restricted to certain small areas of the screen, such as the eyes of a character that move or blink, or a character that moves after being activated through interaction, while the rest of the image remains static. Movement in this app reproduces mostly gestures and body language from the characters; it is critical to the narrative because movement expresses the differences between being alive or coming to life and being static, lifeless toys. In the scenes where the girl is spying on Teddy, he is always static, but the other characters, towards which she is not looking, show movement. Movement may start automatically or must depend on reader activation through interaction.  One small movement that carries a lot of meaning in the narrative is found in the eyes of the toy characters. They blink in most scenes and, in one case, alter their gaze without ever moving their heads or bodies. In this case (screen 5), the movement of Teddy’s gaze conveys a meaning that is not conveyed in any other mode, showing that yes, Teddy is alive. It is,   71 nonetheless, a very subtle movement that may pass unnoticed. In most scenes, Teddy blinks, but that does not necessarily mean he’s alive. Nonetheless, on screen 9, the words contradict this information. The verbal text says the bear “didn’t even blink an eye” suggesting that, for the girl, if teddy blinked, he would be considered to be alive. In order to build a coherent narrative, it would be expected that, in this scene, the images showed teddy with static eyes. Regardless, both teddy and the doll are blinking, promoting inconsistence in the narrative.  Figure 4.1. What Does My Teddy Bear do All Day?, screen 6. Teddy looks at the girl, revealing to readers he is alive. In red, the outline indicates readers can interact with a certain object. At the wall, drawings made by readers are incorporated. Copyright Auryn Inc., artwork by Birte Müller.  In this last scene, when the girl finally “catches” Teddy (screen 13), movement complements the verbal text in various ways. At first, Teddy’s body moves slightly, as if   72 breathing heavily, indicating he is unaware of the girl spying on him. As she enters the room and “catches” him, he turns instantly into a completely static, lifeless state, but it is too late: the girl has already seen him moving. Also, the girl’s jumping inside the room for Teddy’s surprise is a gesture that matches the sentence “I caught you!” in the writing, generating a modal ensemble similar to that which happens when an adult and a child play peekaboo–unfortunately not completely realized here due to the lack of synchronization between narration and movement.  4.2.6 Sound  The sound effects in this app are realistic sounds associated with certain objects and gestures that are activated through reader interactions; nevertheless, not all interactions are accompanied by a sound. The sounds do not promote readers’ immersion and their realism is limited, since they are isolated and do not create a complete soundscape. Also, the sounds are not distributed in foreground, middle ground and background. Both speech and sounds have the same volume and seem to be in the foreground. When readers touch some interactions while the narration is being played, for instance on screens 3 and 4, in which the sound effects are especially loud, these sounds overlap the narration, which consequently cannot be understood. None of the toys produce sound in any of the scenes. The only character with a voice, thus, is the girl, who expresses herself through speech, narrating the text, and through some interactions.  Sound effects are playful in this app in the sense that they come as a response to reader interaction in communion with the animations, but in most cases they are not playful in themselves. An exception in which sound directly contributes to the playful aspects in the app is on screen 4, when Teddy and the doll dance. The music being played by the mice is the classic folkloric Mexican song Jarabe Tapatío, an amusing and unexpected song that might produce   73 delight. Sound in this app is not a mode on its own, but it is supporting the visual/kinetic narrative, and its meaning depends on the association with these modes.  4.2.7 Interaction     Figures 4.2 (A) and (B). What Does My Teddy Bear Do All Day?, screen 8 in two different moments.  Interaction changes the point of view of the reader: the narrative is mostly represented visually through an external point of view (A), but when readers look through the binoculars they assume the role of the girl (B). Copyright Auryn Inc., artwork by Birte Müller.  This app has two or three interactions per scene. The hotspots, where readers should tap to activate the interactions, are highlighted with a coloured outline that blinks on the screen, and the interactions have different levels of importance to the flow of the narrative. There are three types of interactions. The first and most common are replayable interactions, in which readers touch the screen and a movement, sometimes accompanied by sound, is played, usually in a limited section of the scene. The scene does not significantly change; the characters perform certain actions for a few seconds and then return to their initial positions, such as most interactions with the mice. In the second kind, as readers touch the screen, the scene shifts significantly and does not return automatically to the initial position. These interactions are linear   74 and move the story forward. In the third kind, readers assume the role of the girl and become part of the narrative. For instance, readers can assemble the puzzle or make drawings (both on screen 5), which, although extremely ludic, is not necessary for understanding the story. Differently, on screens 7 and 8, readers position themselves as the girl, spying on Teddy–first, through the keyhole, in which they can “move their gaze” and observe the entire room, and second, through the binoculars, in which readers can adjust the focus and the zoom. In both cases, these interactions take readers further into the narrative, shifting the visual text’s point of view from external to first person.  4.2.8 Summary of What Does My Teddy Bear Do All Day? This picturebook app relies on the counterpoint between modes to build its narrative. The verbal text, including writing and narration, shows the girl’s limited point of view in a tentative dialogue with an unidentified participant who is possibly the reader. The visual text, which is animated in some areas and accompanied by sound affects, shows an external point of view, which reveals to readers that her toys are alive, although she, while trying to catch Teddy, misses that they move behind her back. Interactions allow readers to activate the movements and sounds of certain elements in the scenes and in some cases they allow readers to assume the girl’s point of view and spy on Teddy. Playfulness relies strongly in this counterpoint, but some of the modes have exclusive playful features: the text is rhymed and reproduces features typical to children’s play, the visual reveals the parallel lives of the toys, and interactions engage readers in the hide-and-seek game between the girl and Teddy. Sound is not used with the same emphasis as other modes in constructing the narrative and its playfulness.   75 4.3 Lil’ Red by Bart Blocmen and Brian Main 4.3.1 Synopsis Lil’ Red is a wordless retelling of the classic fairy tale of Little Red Riding Hood told in a multimodal text composed of images, animations and sounds, whose progression requires interactions from the readers to unfold. In this version, Little Red Riding Hood is a naïve and fragile little girl. On her way to grandma’s house, she picks flowers and apples, greets the woodcutter, and when face-to-face with the wolf, she sees him as a cute dog, unaware of the danger she is in. The wolf arrives at grandma’s house and after locking her in the wardrobe, he fools Little Red and eats her. Grandma manages to get out of the wardrobe and calls the woodcutter, who saves the little girl.  4.3.2 Illustrations The illustration represents the narrative in this app, as in most picturebooks, giving an external point of view. The images are very expressive, although they show only two colour tones: red and grey, in a variation of shades. Most of the images are in grey, and red highlights some details, giving them salience. Little Red although small is the most salient element, in her bright red cape, while the wolf, usually the darker element in the scene, also gains salience with cold red eyes and its massive body. The fact that the colour red is always associated with Little Red and the bird makes it a cohesive resource promoting associations throughout the narrative. The illustrations are very cartoonlike, simple in shape and details, but very expressive, with exaggerated features: the wolf has a very scary appearance, with sharp teeth and red eyes. Little Red has big, manga-style eyes, and a small body, which transmits fragility and naivety. The contrast in size is obvious, showing the girl’s and grandma’s vulnerability, while the woodcutter is also massive and has a sharp axe by his side (screen 5), which parallels the wolf’s sharp teeth.   76  Figure 4.3. Lil’ Red, screen 6. The colour red is used to create salience and cohesion and to identify the interactive elements. The visual representation uses elements such as scale and the use of sharp and round forms to convey meaning. The girl is small, round and fragile; she sees the wolf as a cute little dog. He has his red angry eyes fixed on her. The dead tree in the background has a cadaveric face, expressing the danger Little Red is in as she meets the wolf. The red bird parallels the girl’s features (small, round, fragile), and as she contrasts with the wolf (big, red eyes, pointy forms), it contrasts with the crows. Copyright Brian Main.  Although the verbal text is absent in this app, the visual mode, in partnership with sound, simulate the verbal in creating dialogs between the characters (screens 6, 8 and 9). Speech bubbles are used, but instead of words they contain icons, and the sounds that correspond to each character reproduces tunes that simulate the intonation typical of questions and answers. This technique leaves many gaps for the reader to fill in, but this app counts on their previous knowledge of the tale, probably the most retold fairy tale in the Western world, so even young readers are likely to have the repertoire to make the connections and understand the story.   77     Figures 4.4 (A) and (B). Lil’ Red, screen 9 in two different moments. Dialogue is created through the use of speech bubbles and pictograms. The girl makes a question (A), and the wolf answers (B). Copyright Brian Main. There are several elements “hidden in the scenes,” and there are usually cues to the reader on where to tap to reveal these elements, generating a playful visual game. The cues, nonetheless, are not as obvious as in most apps. The salience of the colour red, for example, works as a cue calling the attention of the reader to certain hotspots. Animations, which are naturally integrated into the scenes, also generate salience and lead the reader’s eyes towards certain objects. A red bird, a character that gives continuity to the narrative, is present in all of the scenes interweaving them. In the transitions between scenes, all the elements of the scene slide out of the screen, but the bird remains and is integrated as the new scene slides in. The bird is never hidden as other secondary characters are; it constitutes the connection between scenes, and the first element to be seen on each screen. His permanence in the totality of the book, while the scenes change, visually marks the linearity of the narrative. His bright red colour contrasts with the majority of grey elements in the setting, and simultaneously retains the connection between him and Little Red. The bird represents a parallel narrative, which mirrors the central story. The   78 home screen (screen 1), instead of portraying Little Red, shows a black cat trying to capture the bird in vain, as a metaphor to the wolf capturing the girl.        Figure 4.5. Lil’ Red, transition between screens. The red bird remains on the screen as the scenes shift, creating continuity. Copyright Brian Main. 4.3.3 Sound Sound in this app is extremely meaningful. In every scene there is an original and complex soundscape. Sounds of birds are part of the soundscape in the outdoor scenes, but other elements also have sounds associated with them; for instance, on screen 2, you can hear the sound of the swing moving, of Little Red’s steps, and of the opening door. The superposition of sounds is realistic, generating an immersive environment that makes readers feel as if they are part of that scenario.  In the foreground, there are leitmotivs, or recurring musical themes associated to the characters. Each character is represented by a different musical instrument, which participates, in addition to the visual representation, in their characterization. For instance, Little Red is represented by a light and amusing melody played by a xylophone, and the wolf by a mysterious and frightening bass tune. The originality of this approach is playful in itself. The use of music instead of speech is joyful and unexpected.   79 This version substitutes the music for verbal language and shows how, besides the different intrinsic characteristics of both modes, music can present some narrative qualities similar to verbal language. In fact, this app calls attention to the melodic characteristics of spoken language and the importance of rhythm and intonation in the creation of meaning in speech, which here are reproduced through music without the use of words.  4.3.4 Interaction Interaction is fundamental for the construction of the narrative, because every action and “speech” realized by the characters depends on the readers to set them into play. Readers can keep paging through the screens without interacting with the elements in the scene, but this would fail to reveal the totality of the narrative and render the text quite incomprehensible. Although most readers may be able to fill in the gaps for this well known text, it is unlikely they will be tempted to skip the interactions, since reading the wordless narrative means engaging in, getting responses, and interpreting the multimodal text. Also, the navigation of the app is well designed and guides readers: as the key interactions have already been activated and readers can move on to the next scene, the arrow that leads forward becomes animated. Reader participation is fundamental to the narrative progression as they are required to tap the characters to start their performances and allow the actions that constitute the plot to develop. Touching the screen promotes two main responses from the narrative: activating a movement or activating a sound, or in the case of a character, “speech.” The cues that indicate the location of the key interactions are subtle, so readers may feel like exploring numerous elements through touch; in the great majority of cases, they receive some form of response. If they tap secondary elements in the narrative, such as objects in the setting, these elements usually produce a realistic movement or sound.    80 The fact that readers get a response in almost every interaction is very satisfactory, and in the case of this app, since “reading” the narrative in fact involves interacting with it, cannot be considered distractive. In these interactions with setting, readers are exploring the spacious qualities of the narrative, expanding these meanings in a way that is not possible in print books. Many of these interactions have purely ludic effects. For instance, on screen 3, when the butterfly is activated, Little Red jumps and laughs in delight. The narrative of Little Red Riding Hood is then transformed into a series of playful scenarios, in which the readers are invited to participate, to enjoy the path and all the secrets and beauty it can reveal. Although readers’ interactions are fundamental to the narrative, the story is linearly fixed; they do not have the power to decide what a character does or does not do, or to change the order of the scenes, although there is some flexibility in choosing the order of the interactions.  Readers participate as characters themselves in some passages, for instance on screen 4, in which they wake up the wolf by throwing a cone on his head. In this example, readers are given the responsibility for the forthcoming events of the narrative since they are the ones to wake up the wolf.  4.3.5 Animation In the absence of verbal text as the lineal element that conducts the narrative, movement plays an important role in this app; the movements realize the actions that are usually described in the verbal text, and move the narrative forward. Each screen works as a chapter, a section of the narrative inside which events unfold, limited by a more dramatic break which is the turn of screen pages.  The images are never completely static, even before reader interactions. The bird that makes the connections between scenes is constantly moving, and usually characters come and go   81 from the scenario as the scenes start and end. The central actions for the development of the narrative, nonetheless, depend on readers to unfold. Small movements and sounds are used to call the attention of the reader and invite interactions. The connection between characters and readers develops mostly through the characters’ eye movements. Their gaze towards readers indicates that they are expected to participate in the story, and character gaze may also express their dissatisfaction with the readers’ lack of participation–for instance, when Little Red’s eyes become heavy and sleepy (screen 6). Later on, in the same scene, both Little Red and the wolf, while facing each other, simultaneously move their gaze toward readers, prompting them to participate. Also, some elements in the scene operate as cues indicating where the central interactions with the screen are. The colour red may work as one of these elements, since its salience claims the attention of the reader. Also, secondary animated characters may draw readers’ gaze to a certain area–for instance, on screen 3, a fly arrives in the scene, and both its movement and sound conducts the readers’ eyes to the flower it lands on, leading readers to tap this area, initiating then the central action of this passage: picking up the flowers on the path.    Figures 4.6 (A) and (B). Lil’ Red, screen 9 in two different moments. If readers do not interact with the screen, Little Red moves her gaze from the character she is interacting with (A) towards readers (B), inviting their participation. Copyright Brian Main.   82  4.3.6 Summary of Lil’Red This app is a wordless narrative that combines the visual, sonic, kinetic modes and interactivity to generate a very complex and sophisticated narrative. The modes complement each other in this narrative, filling each other’s gaps according to their exclusive affordances. The visual text is unique in its use of colour and a cartoony representation that generates contrast between the elements in the scene, often with amusing and playful effects. Sound is used as a very playful resource with the use of leitmotivs and interactions that allow readers to engage with the sound elements of the app. The narrative progression completely depends on readers, since most of the actions performed by the characters only start with reader activation.   4.4 The Monster at the End of This Book by Jon Stone and Michael Smollin  4.4.1 Synopsis In The Monster at the End of This Book app, Grover, the famous Sesame Street character, is reading the book called The Monster at the End of This Book. At the beginning he teaches readers how to “turn the pages” of the book shown on the screen. However, when Grover realizes that by doing so he will eventually have to face the monster, he starts demanding the readers not turn the pages anymore, which of course, they always do. Grover starts to build barriers to block readers’ from turning the pages, but, by interacting with the screen, readers can dismantle these protections and keep turning the pages. At the end of the book, Grover realizes that he is actually the monster at the end of the book, but pretends he always knew it and had never been scared.    83 4.4.2 Writing The Monster at the End of This Book’s writing is very concise and directly addresses the reader as a participant in the communication: the text is a dialog between Grover and the reader, who is addressed by the pronoun “you.” As typical of this form of communication, the language is informal, containing many interjections, indicating proximity between Grover and the reader. In this dialog, Grover is commenting on and trying to restrain the reader’s action of turning the page in a very dramatic manner. The seriousness and acuteness of his reactions, as someone who is really scared about the possibility of facing the monster, play to a hilarious effect.  Writing is integrated with the illustrations. They are not in the traditional structure of a dialog in prose, but appear as intraiconic texts in speech bubbles. Different from most of the book apps analyzed in this study, typography is a strong element that connects visual, writing and speech modes. The use of different fonts, colours, sizes and weights expresses visually the intonation and emphasis the text has as the actor is interpreting it. This use of typography works in the print version to simulate speech, but with the actual integration of speech in the app format, the superposition of both generates an effect of exaggeration, which is comic and playful. Movement also takes part in in this connection, since the words appear on the page one by one, as they are spoken.  4.4.3 Narration The narration in this app is extremely expressive. Grover is not reading the words on the page; rather, a conversation is taking place in between fiction and reality, between character and reader, through the medium of a book, and as such, words are shown on the screen. The narration is more extensive than the writing on this app; there are several instances in which Grover talks to the reader but the writing on the screen is not seen. The first of these instances happens when   84 Grover teaches the reader how to “turn the page” on the app, instructions that are unnecessary in the print version. Most of the following instances refer to Grover insisting that the reader do not turn the page. Ironically, he keeps repeating the instructions as actions “readers should not perform,” an irony which contributes to the playfulness of this text.  On the first screen, as Grover starts reading aloud the title of the book hanging on the wall, he does so with fluency similar to that of a young child who does not yet show complete reading fluency, putting himself and the reader (presumably a young child) at the same level of power. The volume and intonation varies a lot from whispering to shouting. In most of the narration, Grover is shouting at the reader. Here, nonetheless, shouting does not mean public distance, but the opposite. Grover’s shouting indicates an extreme level of intimacy. He is discussing, almost fighting with readers about the progression of the narrative, playing with readers when he teases them not to turn the pages while at the same time giving them the advice on how to do so. The result is a metafictional game that is incredibly fun.  4.4.4 Illustration The illustrations are cartoony as are all Sesame Streets books, since they come directly from the TV screen aesthetic. Yet, there is a slight difference between the style used on Grover and the one used on the rest of the images. Grover is brighter and does not have an outline, which somewhat gives the impression that he is outside of the world of fiction–represented by the other pencil drawn elements in the scenes–and in the world of the reader. The first screen, which would be equivalent of the cover of the book, does not make any reference to the book form: Grover is walking along the street and sees a poster on the wall displaying the title of the book. On the next page, the format of an open print book becomes visible: the gutter space and the volume of the pages passed and the pages still to come. In this   85 way, readers can have a notion of how “close” they are to the monster at the end of the book, a notion that is usually lost in digital publications, in which this topographical aspect is absent. Grover, nonetheless, stands outside of the boundaries that define the book; he is positioned by the illustration as being part of the “real world” outside of fiction, from which he establishes the dialog with readers. 4.4.5 Interaction The key interaction in this app is tapping the bottom right hand corner of the screen and turning the pages. This action is done at every page, as a transgression of the recommendations given by Grover, but also as a confirmation that we are reading a book, and thus cannot continue without this movement. As mentioned, Grover is consistently asking the readers not to perform this movement while simultaneously taunting them to do so, by mentioning the spot with which they must interact. Other central interactions to the development of the narrative happen when readers dismantle the barriers created by Grover. These interactions are reinforced by a yellow highlight, which appears around the hotspots, and, again ironically, Grover teaches readers how to interact with them–and destroy what he has built. Finally, on some screens, readers can interact with the typography. Tapping certain words, in which the font is salient from the rest of the writing, may cause this section of the text to be repeated or simply animate some of these typographical elements, reinforcing typographically Grover’s expression of fear and apprehensiveness. 4.4.6 Animation Movement works alongside interaction in the gesture of “turning the page,” which is essential to this narrative. This movement, which is physically realized in the print book and virtually realized in the app, is the leading concept in the narrative and the basis for the   86 construction of its metafictional nature. This story, both in print and in app versions, proposes, then, the discussion that gesture and movement are already important elements in the meaning-making processes of reading a print book. As explored in the literature review, Mackey (2002) posits that this movement is usually overlooked in the analysis of the print reading processes. In addition, here, movement contributes to the visual mode in restabilising the boundaries between fiction and reality. While the elements inside the pencil drawn book are static–the title page and “THE END” on the last page–Grover, who is positioned outside the book and in the “real world,” can move, as can the print words he is producing as he speaks. So the fictional world in this app is represented by the lack of movement, while the “real world,” in which Grover and reader interact, is alive and in motion. The pencil drawn book, an object which, according to the new boundaries set by the narrative, is part of the real world, also has the corner of its pages move slightly, as if wafted by a soft wind, inciting readers to touch and turn the page. In the “real” world, movement is responsible for creating irony in contradiction to the verbal text when Grover says he is building a “heavy, thick, solid, strong” wall (screen 9), but as he knocks this same wall, a crack starts to form, which denotes the wall is in fact fragile. Furthermore, movement in the book hearkens to the origins of Sesame Street content as a TV show. Although the app references the book world in its content and form, the inclusion of animation and speech approximates the reading experience slightly to that of watching an animated show. This app, through its claim of a being a book and through its metafictional content, proposes then a discussion of the future conceptualizations of what reading means in the digital environment, a new form of reading which includes animation and speech. In addition, the fact that Grover has to teach readers how to “turn the page,” which does not happen in the   87 print version, reflects that there is a need to make reader wary of the shift in the conventions of the book taking place in the transition from print to digital media. Finally, movement is used with didactic intent when the words appear on the screen in synchronicity with narration, an effect usually called “word highlight,” which intends to help children associating the form of a word to its sound. 4.4.7 Sound This narrative starts with a lighthearted jazzy track that establishes a cheerful mood and an urban landscape–in accordance with the setting of the home screen and Grover’s mood at the beginning of the story. As we enter the story, the sounds effects combine musical effects, which participate in creating Grover’s different moods throughout the story (accompanying the meanings transmitted by other modes) and realistic sound effects, which work to generate believable animations. The sounds, then, are original and distinct in every scene. The musical effects use a great variety of tunes and instruments to transmit, in most scenes, suspense and tension as readers turn the pages and get closer to the “monster.”  Sound also contributes in establishing the drawn book as an object which is part of the real world; the sound of the page turning contributes to the virtual creation of the gestural and movement of reading a print book. Thus, it is fundamental to the metafictional character of the narrative. This sound is also repeated when readers are expected to interact with the screen by “turning” the digital pages. The sound of the page turning, along with its movement, serves as a cue, alerting readers to tap that area and turn the page.  4.4.8 Summary of The Monster at the End of This Book This classic metafictive narrative is built upon the convention of turning the page to move the narrative forward. This gesture, which is virtualized in the app version, puts the modes of   88 interaction and movement as central in the multimodal text. The verbal mode establishes the communication between Grover and readers who are active participants of the narrative. The visual text recreates the boundaries between the fictional and real worlds, and repositions Grover out of the fictional world. Sound effects contribute in creating a realistic animation but also in increasing the tension as readers approach “the end of the book”. Playfulness in this app then is centered in the possibility that the reader has to challenge the protagonist using the power associated to turning the pages and moving the story forward, attributed to him as conventions of reading and of the book form.  4.5 Don’t Let the Pigeon Run This App! by Mo Willems and You 4.5.1 Synopsis Don’t Let the Pigeon Run This App! is a picturebook app that is part of the Pigeon print picturebook series that started with  Don’t Let the Pigeon Drive the Bus! (2003).The book and its sequels were already considered books with high levels of interactivity, and Don’t Let the Pigeon Run This App! maintains the structure of the previous print books with the addition of new forms of interactivity made possible in the digital format. Essentially, the Pigeon picturebook series has a template that is repeated in every book. The digital version is based on this same template and many different versions of the story can be read inside a single app. Some elements, such as the main action the pigeon wishes to perform–“drive the bus” in the print book, but here “rule the world,” “kiss the cat,” etc.–and some of the elements he uses in his arguments to convince the reader, change every time the story is read. In general, the story can be summarized as: the pigeon wants to do something that readers should not allow. He makes promises and argues that other people would allow him and that other pigeons have already done it; he offers things in   89 return, and as readers keep repeating “NO,” “NO,” ‘NO,” the pigeon gets angrier and more desperate, he cries and flounces, until he realizes his wish is not going to be fulfilled and gives up, blaming the readers for his lack of success. 4.5.2 Writing and narration The writing in this app is only present in the “Egg” option. In the other two modes, the narrative is presented as an animated movie, and the verbal text emitted by the pigeon is not shown on the screen. Nonetheless, some words appear on the screen on all modes, as intraiconic texts, for instance the responses given by readers–“NO!” and some interjections. The narrative starts with the bus driver, which grants the reader with the power of creating a story by “shaking the pigeon.” He treats readers as friends, equals: his tone is informal–“I’ve got to go visit some pals.” Differently from the antecedent print books, the driver does not clearly transfer to readers the duty of not allowing the pigeon to do whatever he wants to do, which could mean that the author assumes readers are already familiar with the series and its structure, in an indirect intertextual reference. As the pigeon appears, a dialog between him and readers starts: the language is informal and friendly. The sentences alternate between questions (the pigeon asking the readers to do something) and statements (the pigeon trying to convince the readers to allow him to perform a certain action). The content is ironic and amusing because the arguments given by the pigeon are not convincing and somewhat nonsensical; it is clear that the pigeon is saying anything that comes to his mind, no matter what it is, trying to cajole and flatter readers. The content of his arguments is so meaningless, that it changes repeatedly as readers create new stories, without affecting the comical text.  The written text in this book appears as optional. As the narrative starts, it is hidden, and readers must tap the screen to reveal it. When revealed, its format is similar to that of a subtitle in   90 a movie, white on a transparent grey box, including even descriptions of sound effects, as if only the ones who cannot listen to the narration would read it, which confirms the extreme proximity between written text and speech in this app. Thus, speech is the dominant mode for the transmission of the verbal text, and it is connected to the high level of theatricality present in this app; the exaggeration and the dramatization that the pigeon uses in order to try to convince the reader is expressed significantly through speech. The pigeon’s voice is narrated by the author Mo Willems himself and is extremely dramatic, with pauses, emphasis and changes in tone–all of which express the various moods and feelings lived by the pigeon throughout the narrative. The irony of the arguments and the feeling that the pigeon is trying to fool readers are transmitted at a great level through the narration–and strongly accompanied by the pigeon’s eye movements and facial expressions. Differently from the book series, but similarly to comic TV shows and cartoons, this narrative incorporates readers expected reactions to the pigeon’s pleadings. In the print book Pigeon series, the negative response from readers to Pigeon’s demands is not part of the narrative. Through the continuation of the text and Pigeon’s insistence, it is implicit that readers always deny his demands. The app’s narrative, nonetheless, incorporates readers’ denials, both in the visual mode, through the typographic expression of the word “NO,” in big, capital letters, at the center of the screen, a ‘no’ that is shouted aloud by a group of children in the narration, which simultaneously causes the whole “page” to tremble. 4.5.3 Illustration The illustrations in this app are extremely simple and almost childlike. The drawings are minimalist, with a heavy outline filled with one flat colour, without any modulation. Its simplicity enhances the evocativeness of the facial expressions made by the pigeon, which are   91 central in conveying his feelings. The characters appear floating on the screen, looking directly at the reader and establishing the communication; the point of view of the visual text puts readers in the place of participants in the narrative, looking directly at the central character. There isn’t any setting or indication of place and time, which centers the focus of this narrative in the dialog between pigeon and reader; the simplicity of the drawings and the lack of backdrop cause readers to focus on the theatricality of pigeon’s communication. The colours are pastel and the background shows a flat colour, which changes according to the rhythm of the speech and to the mood it seeks to transmit: the neutral colour is beige, and it turns to warmer pinkish and purplish shades when readers shake the pigeon and to bluish and greenish shades when the pigeon is trying to convince readers. The frantic vibration of colours in the background expresses visually the annoyance of the pigeon as his wishes are not fulfilled. 4.5.4 Animation Movement in this app is constant and also extremely expressive. The elements on the screen are never completely static. Even the title of the story appears shaking on the screen and the fact that the pigeon is “trembling” all the time generates a constant sensation of anxiety and stress. The pigeon’s movements are bold and expressive, depending on his mood and tone in each situation, which shift dramatically from one moment to the other. He gesticulates constantly, his body language in coordination with the words he speaks. The pigeon tries to make the readers sensitive to his anger, desperation and pain. His eye movements and expressions are central to transmitting these feelings. His actions, such as stomping his feet, protesting, grumbling–which are central to building his character–can only be realized through the animations.   92 4.5.5 Sound The use of sound in this app is very similar to that of The Monster at the End of This Book, but as a minimalist text, the sound effects are not numerous and are usually intended to enhance the expressiveness of Pigeon’s emotions. Sounds can be organized in three categories. First, realistic sound effects, which accompany the movements of the characters, are limited to steps and the flap of the pigeon’s wings, especially when he is shaken. They are a bit exaggerated, as is the overall narrative. The second category is that of musical effects, which are intended to transmit though short pieces of melody the pigeon’s strong emotions, and these effects get more frequent, sharper and lower in tone towards the end of the story, accompanying the progression of the pigeon’s frustration. At the end, the sound effects transmit his resignation. Finally, the pre-recorded expected responses from the readers can be considered a third form of sound effect, which include a group of children shouting ‘no’ and laughing, suggesting to readers moments in which their interaction must take place. At the very end of the narrative, confetti is thrown across the screen and the children cry in happiness and contentment and triumph, as the pigeon is defeated and gives up. All sounds in this app work in close partnership with the animation and speech to characterize the pigeon and transmit the increase of his discontent.  4.5.6 Interaction This app is significantly different from most apps in terms of interaction because the possibilities of interaction precede the narrative itself. Once the narrative starts, it is shown as an animated movie, without any hotspots and only one instance in which readers must interact during the story. The interaction previous to the narrative takes place as readers co-author the story, which can happen in three different modes. The first, called “Egg,” is the closest to reading a picturebook. As readers “shake the pigeon,” a story is randomly set and presented like an   93 animation. Every time readers shake the pigeon, a different story is automatically built, but the reader does not have the power to customize it. The second option, “Chick,” is partially customizable. Before starting the story, the bus driver asks questions referent to each scene of the book and offers three possible choices; readers also get to record their names. When the story is told, the elements chosen by readers are incorporated into the narrative and their recorded names are integrated into the narration; they are proclaimed co-authors of the story. For the final option, “Big Pigeon,” the level of co-authoring is increased. The bus driver asks the same questions, but readers are free to voice-record their own answers, and their voices are incorporated into the story during the storytelling. This way, readers not only contribute with content but also become themselves narrators. The only instance in which readers participate during the story is when they are expected to shake the pigeon. In the first mode, as the readers shake the pigeon, a new story is created, so shaking the device becomes a “magical” gesture with generative power. The level of participation and co-authoring is further expressed, especially in the “Big Pigeon” mode, through the inclusion of the readers’ voices in the narration, which gives readers another way to manifest their power. Outside of the narrative, readers can learn with Willems how to draw the pigeon, and the drawing is also incorporated into the narrative, as the pigeon offers a “piece of priceless art” to convince readers to allow him to do what he wants. 4.5.7 Summary of Don’t Let the Pigeon Run This App! This metafictive app allows readers to participate as co-authors, with different levels of involvement at each reading mode, generating numerous stories that follow the same structure: readers deny Pigeon something that it wants, and it uses every possible argument, even if nonsensical, to try to convince them otherwise. Much of the playfulness here relies on the power   94 play between character and readers. All the modes work in coordination and create a very cinematic experience, where all the modes contribute to the construction of this unreliable, comic, and extremely expressive character. The interactions occur mostly before the narrative starts; the reader can add their inputs to the narrative through voice recordings, choosing among options offered by the app, or simply allowing the app to create a seemingly random narrative.  4.6 The Heart and the Bottle by Oliver Jeffers 4.6.1 Synopsis The Heart and the Bottle tells the story of a little girl who refuses to grieve when her grandfather passes away. The story starts with girl and grandfather–or possibly father, the story does not specify–exploring the world and appreciating stories, books, nature, living together the beauty of life. When Grandfather suddenly dies, the girl puts her heart in a bottle, which eases the pain but also suppresses her memories and curiosity about the world. She grows up in this awkward but safe state, until one day she is questioned by a little girl, joyful and curious as she used to be, but is unable to find the answer without her heart. Deciding to take her heart out, she tries every possible way to open the bottle, but is unable to release it, until, with the help of her new little friend, she is finally able to release her heart and start appreciating life and the joys of the world again. 4.6.2 Writing The writing in this app has the structure of traditional tales for children: it starts with “Once there was a girl,” has a third person narrator, and is told in past tense. The girl is not named and described as “much like any other,” which attributes universality to the story. As a narrative about death and grief, a situation that affects all of us equally at some point of our lives,   95 this universal tone provided by the text is profoundly touching, in a cross over narrative capable of stirring emotions of children and adults alike. The narrative is circular and shows the changes suffered by this girl when her grandfather dies and her journey back to joy, with the help of a little girl, after many years of denial. The text is brief and synthetic, and it focuses on showing the changes in the girl’s state, from curious and joyful to empty, then back to her original state. Much of the narrative can only be conveyed in its relation to the other modes, especially to the visual mode–although the other modes add significant meaning in the app version, as will be discussed below. The simplicity and vagueness of the writing contributes to making the narrative universal, allowing readers to associate the text with their own experiences, feelings and ideas.   Figures 4.7 (A), (B), and (C). The Heart and the Bottle, screen 13 in three different moments. Visual text and interaction are used to express Grandpa’s absence and the impact of his absence in the protagonist’s life, as her world shifts from light (A) into darkness (C). Copyright: Oliver Jeffers.  The narrative talks about death and grief, recovery and hope, without ever mentioning these specific words, in a text built around metaphors. The grandfather is not even mentioned in words, but in a silence that echoes his absence in the girl’s life. The death of the grandfather is represented by a moment of muteness in the narrative, in which the other modes, especially visual and kinetic (in this case, the lack of movement), but also interaction, carry the meaning:   96 his absence is symbolized visually by an empty chair, and the interaction in which readers must reveal the image of the girl staring at the empty chair, and transform the scene from light into darkness.  This action requires a few seconds to complete, and the pause it causes in the narrative contributes in transmitting without words the deep sorrow the girl may be feeling when having to face the death of her beloved grandfather.  4.6.3 Narration This app is narrated by the award-winning British actress Helena Bonham Carter. The tone of her voice is soft, almost a whisper, indicating an extremely personal distance and intimacy with readers. As Nugent (2013) mentions, the narration in this app reproduces the intimacy of a parent reading for a child on his/her lap. The layout of this app is organized in a way that one sentence may, in some scenes, be broken into parts and spread in small blocks of text in different areas of the illustration. The page layout suggests pauses and a specific reading flow, which are respected by the narrator. The tone of the reading is soft and yet serious and solemn, which helps building the mood of the narrative and is in accordance with the theme and the development of the story, resulting in a deeply moving narration. 4.6.4 Illustrations The illustrations in this narrative utilize mixed techniques, using traditional and digital tools. Most of the narrative is rendered in Oliver Jeffers’ characteristic style, with simple shapes, a varied colour palette and the use of rich textures–all of which generate vibrant and yet simple compositions. Some scenes include collages of elements with distinct representational styles, which we do not know if created by the author or taken from other publications–the only artwork credited is the one on page 12. On screens 6 and 27, which mirror each other, a collage is used to represent the distinct world to which the girl has access through books. The beach setting, which   97 is repeated in many scenes, always appears with variations, and in some cases, the texture of the sea or of the sky seem to be collages, an effect that suggests the passage of time. The characters are very simply depicted, with little facial expression. Pain or sadness are not mentioned by the words nor expressed notably by the images; they must be implied through the interpretation of the multimodal text. The images significantly expand the content of the words, specifying many elements in the narrative, which are general or not even mentioned in the verbal text. For instance, the relationship between the girl and the grandfather, which is central to the narrative, is transmitted exclusively through the visual text.  At the beginning of the narrative the scenes represent girl and grandfather in specific settings–at home or in nature. After he dies, the settings fade out–literally, as readers wipe them out through interaction. We are transported to the internal world of the girl; she does not notice the world around her and is focused on her sorrow, while the backgrounds become a flat colour– usually white, but in some instances, red and grey. When she meets the little girl, the representation of the setting is resumed, but not completely–the chair, the nightstand, the books and a carpet appear but not the rest of the room–which may indicate the beginning of the protagonist’s recovery, but not yet a complete moment of ecstasy or joy, which would seem artificial after such a long period of sorrow. 4.6.5 Sound  The sound effects in this app are very realistic and immersive. Every scene has its own original soundscape, transporting readers to the environment where the characters are. The soundscapes are complex, composed by the juxtaposition of different sounds that create a specific environment–in many scenes, a natural environment. The sounds created by the characters are in a higher volume, which works accordingly in representing their proximity to the   98 reader and their importance in the narrative. The sounds are especially meaningful in the central part of the narrative, after the grandfather dies. While the words and images do not touch directly upon the theme of pain and grief, the sound of a heart beating is extremely emotional, a level of emotion that is not explicit though the words or the images. In other instances, the sound effects add another layer of meaning, complementing the other modes. For example, the awkwardness of the bottle hanging on the girl’s neck mentioned by the others and visualized on the images, gains another dimension with the sound of the glass swinging, which is extremely uncomfortable. Similarly, the sounds produced when the girl is trying to take the heart out of the bottle, knocking on the glass with various materials, are intense and cause great discomfort. According to Nugent (2013), children in a shared reading assignment had a sudden reaction to the sound of the bottle falling from the high wall on screen 23, confirming that these sophisticated and subtle meanings are accessible to the child reader. 4.6.6 Animation Movements in this app are limited to certain areas of the screen. Some start automatically as readers enter the scene, but mostly they are activated by reader interaction. The most common action in most interactions is to move an object on the screen, a movement that in most instances is not necessary for the understanding of the narrative, but one that animates the scene with ludic effects, bringing a little joy in this sad and moving narrative. These ludic movements have meaning especially in the first part of the narrative, in which the girl is being described as joyful, and her movements as she plays with the grandfather contribute in transmitting the feeling of well-being. For instance, in the first scene (screen 4), movement is the mode responsible for showing the girl playing hide-and-seek with grandpa, an episode that in fact is not present in the print version of the narrative. Movement here is participating in characterization. The sudden   99 contrast between movement and lack of movement is especially meaningful on screen 12, when the girl is happily running with a drawing in her hand, contrasting with screen 13, when she is paralyzed looking at the empty chair. Here, stillness works as a metaphor for the sudden death of the grandfather. Also, the movement of the bottle hanging on her neck as she does other activities, and especially the movement of her heart twitching inside it, contributes to the feeling of awkwardness of the situation (screens 18 and 20). In addition with the sound effects, these movements contribute in causing discomfort to the reader and transmits the sensation of her own heart being a burden to her.  4.6.7 Interaction This app has a few (usually around two) interactions per page, which can be indicated by tapping the button “hint.” In most cases, after interacting with a certain character or area of the screen, a movement or sound is activated. These interactions may be initiated by tapping, dragging or sliding one’s fingers across the screen, or by shaking the device as a whole. The reactions might be more or less related to the movement performed by the reader, for example sliding may result in the girl’s bicycle moving, as if Grandpa were pushing her, and shaking the iPad may cause the bottle on her neck to shake, while in another scene it causes the snow storm to get heavier. As already mentioned, the interaction of wiping the screen and turning it from light into darkness (screen 13) causes a break in the narrative that has a profound meaning. It is a case in which action actually signifies pause and reflection. On screen 17, the interaction of sliding the finger and as a result having the girl growing up transmits a meaning that is not conveyed by any other modes, and in fact is only implied in the print version of the book. It shows the passage of time and the girl forgetting her grandfather and the joys of the world as she grows older without her heart. Similarly, interactions on screens 15 and 16 that permit “erasing”   100 the thoughts of the stars and of the sea, also are not present in the print book, and transmit the meanings of oblivion and emptiness in a more effective manner than the static illustrations of the girl not looking at the stars present in the print version. In the last scene, the girl is reading, and the revealing of the content depends on the readers. As they tap the screen, a new object, usually accompanied by the pronunciation of its name, appears on the screen in an extremely playful activity, which is random, for at every reading the order of the objects changes. This playfulness and the joy of this little “game” has the effect of producing in readers the same joy the girl is feeling as she finally overcomes her grief. In the print book, nonetheless, the static illustration shows a collage of numerous elements that are not named or identified, transmitting an idea of her imagination going through infinite possibilities. Thus, in the digital version, although playfulness was enhanced, the addition of interaction represented to a certain extent a limitation in the meaning in comparison to what is represented in the print book, since the sense of infinitum was limited to a certain set of possibilities and objects present in the app. 4.6.8 Summary of The Heart and the Bottle This complex and subtle narrative combines different modes to construct a text that conveys complex and abstract meanings related to the theme of death, loss and grief. The individual modes are very brief and indirect and the narrative only makes sense through the interanimation between them. The verbal text is short and metaphorical, never mentioning directly what the girl is going through. In many instances, the exact lack of verbal information, in other words, the moments of silence, is what in fact creates meaning. The visual text carries details about the narrative not mentioned in the verbal text. Sound and interaction participate in very meaningful ways transmitting discomfort, loss and emptiness. This is an example of playful narrative that does not convey amusement or comic content.   101  4.7 The Three Little Pigs by Nosy Crow 4.7.1 Synopsis In this version of The Three Little Pigs, Mom and Dad decide that it is time for their children to live independently, so off they go to the forest to build their houses. The oldest pig, tired of walking, is the first to stop and quickly builds a house of straw. The second pig is a girl, who then builds a house of sticks. The third pig has an elaborate project and, with a lot of hard work, builds a three-storey house of bricks. They live happily until the big bad wolf arrives. Without being allowed to come in, the wolf blows down the houses of the first two pigs, who run to their third brother. Unable to blow down the house of bricks, the wolf climbs the chimney, but is surprised by a pan of boiling water waiting for him at the bottom. Victorious, the little pigs live happily ever after in their new brick houses.  This book presents three ways of reading the narrative, which significantly changes how the modes are integrated, how much (if any) interaction is part of the narrative, and the pace in which the narrative is read. In “Read and Play,” the verbal text is narrated and readers must be active in certain essential interactions for the story to continue; they are responsible for moving from one screen to the other. In “Read to Me,” the verbal text is narrated and the story continues even without any interaction by the readers, functioning almost like an animation, yet the hotspots can be activated if users decide they want to explore the scenes further. The “Read by Myself” option is similar to the “Read and Play” option but without narration and allowing readers to choose how long the text is shown on the screen, depending on their level of reading fluency. The comments that follow will focus on the “Read and Play” mode, which activates all the features of the app, including all forms of interaction.   102 4.7.2 Writing The writing in this app is organized into two kinds of texts: the central narrative in prose, told by the narrator, and the intraiconic dialogues, spoken by the characters. The main narrative is short and synthetic, usually less than two sentences per page, and very general, focusing on moving the narrative forward. Its structure is that of traditional fairy tales, starting with “Once upon a time” and ending with “They lived happily ever after,” in past tense, with a third person narrator. The dialogues, on the other hand, appear in speech bubbles; some are necessary for the completion of the narrative–e.g., the famous dialog “Little pig, little pig, may I come in? / Not by the hair of my chinny chin chin. / Then I’ll huff, and I’ll puff, and I’ll blow your house down”–and others not, although they reveal more details about the characters and the situation they are in. In these dialogues, the characters not only interact among themselves, but may also engage with the readers directly, which may transpire through character gestures and not solely through the verbal text. These dialogs are all in present tense and many of them constitute questions towards the readers, establishing a communication between readers and the fictional world. The combination of the two forms of texts generates a different reading experience than focusing exclusively on the main narrative. The dialogs are central, for example, in characterization, because it is through these texts that the pigs express their personalities.    103  Figure 4.8. The Three Little Pigs, screen 2. The writing in this image is divided into the narrator's text and intraiconic dialogue from the characters. Copyright Nosy Crow, artwork by Ed Bryan.  The writing in this narrative is ironic and funny, which is an important source of playfulness. For instance, on screen 3, the dialog between the parents and little pigs is funny, demonstrating that the two older pigs are a bit lazy and that parents now want to have some free time, while the third pig is ready for adventure.  The wolf is also sarcastic in his words, and tries to fool the pigs with conversation before blowing down the houses. 4.7.3 Illustration The illustrations in this app are very attractive, colourful and bright. They take advantage of the high resolution of the iPad screen, showing textures and details of the elements represented. The images were created digitally but are not flat or vectored, instead incorporating textures and lines that simulate pencil. The settings are detailed and the illustrations provide a lot of information the verbal text does not mention in its briefness. The little pigs are represented as anthropomorphized animals, dressed in clothes and reproducing human behaviour, which contributes to transmitting the message that this story is a metaphor for human behaviour, although the story is not explicitly moralistic. The illustrations show a domestic setting typical of a middle class family, in which the parents play traditional family roles, with the mother cleaning and the father reading the newspaper. The clothes the little pigs wear contribute in revealing their   104 personalities: the first pig in a striped shirt and wearing glasses looks older and conservative; the middle pig, a girl, carries a purse and has flowers on her head, which may indicate vanity, depiction also suggested by the verbal text; and the third, the youngest pig, with a huge backpack, looks determined and adventurous. Their way through the forest plays an important part in the narrative, and this setting is resumed in several parts of the narrative. The forest shows some chopped trees, an indication of semi-urbanized areas, which match the humanized characterization of the pigs. The houses they build also express their personalities, not only through the materials they choose but also through their design, with the first pig building a simple traditional-looking house, the girl building a simple but richly decorated house, and the third pig building a plain but bigger and sturdier house.   Figure 4.9. The Three Little Pigs, screen 4. The illustrations are colourful and the representation of each of the pigs helps in transmitting their personalities, which is also in accordance with the words they say. The setting with several tree stumps help in the anthropomorphization of the pigs. Copyright Nosy Crow, artwork by Ed Bryan.   105 The wolf’s representation makes much of the fact that he is hungry–which is also extensively repeated by his words: he drives a food van, and when he is at the doors of the pigs’ houses he carries a big fork and wears a chef hat. His aggressiveness is represented through various signs: sharp teeth, pointy ears, a mean gaze, long flailing arms and his hand in the shape of a gun–all while persecuting the pigs on screens 12 and 15. The illustrations present a parallax effect and their elements change in point of view as readers move the device, which allows the view of slightly different perspectives of the same scene. This creates a sense of depth and permits that some elements in the illustration become “hidden” depending on the angle from which they are seen. In this app, this affect is used to hide two extra characters, a bunny and a spider. The bunny comments actively on the story, while the spider is usually passive. They both communicate with the reader if they are activated by touch. The bunny even plays hide-and-seek with readers on screens 10 and 11, where readers may look for him under the piles of straw. The parallax view also contributes to the sensation of immersion into the narrative, since, as readers move the device, their view of the scene moves as well, as if they were part of the scenario. 4.7.4 Animations Movement in this narrative materializes via various procedures. It is usually present in the characters, who are never completely still; their movements are a bit exaggerated, but the fact that they are always slightly moving contributes to creating a somewhat naturalistic representation, enhanced by the parallax effect–naturalistic in comparison with a traditional picturebook, although a completely animated scene would represent a much higher modality in this aspect. This app also shows a certain mobility of the “camera,” zooming in and out of certain elements during some of the scenes.    106 Movement is the response to most of the interactions in this app. For instance readers can make any character tilt in the air, which has a purely ludic effect. Movement is also significant in the transformation of the building materials into the final houses, and, conversely, in the act of destroying the two first houses. In the construction scenes, although there are three steps to building each of the houses, the animation is not detailed. The pigs throw a bunch of materials and it instantly transforms into part of the house. This simplification of the animation makes it seem magical and not the result of effort, which contradicts the central message of the story; the representation of effort, however, is present in the verbal text.  Body movement expression is also important in this app: facial expressions and body postures indicate the personality of the characters but also their momentary states. For instance, the two little pigs shake with fear when the wolf is around, while the third pig keeps his adventurous and brave body posture and movement. The wolf’s gaze and eyebrow movements are also important in his characterization as evil and unreliable–all these movements are exaggerated and sarcastic, and as an effect, become humorous. The movement of the characters can be partially controlled by readers. For instance, when in the forest, readers can make them run faster or make the wolf go slower. 4.7.5 Narration All narrators in this story are children, which makes it very relatable for young readers, and also contributes to the playful effects of the narrative, as even the wolf’s voice is made by a child, who manages to transmit tension and sinister intent as his tone of voice is low and the voice rumbles. The narration’s significant shifts and its stresses on certain words have dramatic effects, which make the reading very engaging. The social distance, though, varies as narrator and characters speak; the former uses a tone of public distance, as it is telling the story to a   107 massive audience, while the latter show personal distance as they speak among each other and with the reader, indicating proximity.  4.7.6 Sound The sound effects and soundscape in this app are realistic and promote an immersive environment. Some elements produce sound when tapped; in other cases, the sounds of objects in the setting are used as part of the soundscape, which is elaborate and sophisticated. Besides these naturalistic sounds, one tune is played throughout the story, but the instruments, style and rhythm change at every scene according to the character or characters represented and the situations they face, creating leitmotivs. These leitmotivs contribute significantly to the creation of meaning in the story and generate an atmosphere that involves the readers. For instance, when the wolf appears, the tune gets slower and deeper, creating mood of mystery and danger; also, on screen 5, the rhythm is quite slow and transmits laziness, contributing to the characterization of the little pig building the house of straw, while in the scene where the girl pig is building the house of sticks, the tune is played as a waltz, which matches the content of the girl’s words, in which she mentions liking dancing and parties. The volume of the sound effects and scape is lowered when the narration is being played, building a naturalistic perspective, which stratifies the sounds and prioritizes the verbal text.  4.7.7 Interaction In this app, several elements in the scenes are hotspots that allow interaction, but their level of importance varies with regard to telling the narrative. Yet, most of them are significant to the story, which requires reader participation to unfold. The key interactions are indicated to readers by a blue flashing dot. There are various ways of interacting with the touchscreen: • Tapping characters: activates dialogues or actions.   108 • Flicking characters and objects: throws them up in the air.  • Dragging: makes characters move. • Spreading fingers: zooms in. • Pinching: zooms out. Some of the most significant interactions in this book take place when readers “help” the pigs to build their houses by tapping the character holding the construction materials. Except when playing the option “Read to Me,” the story will not continue without these interactions. The most original interaction in this app happens in the scenes when the wolf blows the houses down. After the wolf’s and each pig’s dialogue, the icon of a microphone accompanied by the word “blow” appears on the screen. By blowing the microphone of the device, readers activate the animation of the wolf blowing down the houses.  4.7.8 Summary of The Three Little Pigs This app presents a very interactive version of the narrative of The Three Little Pigs, where the various modes are rich and complement each other, adding various layers of meaning to the story. The verbal text has two instances: one that tells the story from an external point of view and the dialogue between the characters, which also addresses the reader in some occasions. The visual text is colourful, dynamic and attractive, showing constant movement, which seems more similar to videogames than to animations. The parallax view allows a slight change in readers’ point of view, permitting that some elements to be hidden and enhancing playfulness. The sound effects vary at every scene creating the atmosphere of the narrative and showing leitmotivs associated to the characters. The interactions allow readers to assume the roles of various characters, varying from little to extreme bodily engagement and theatricalization, which is extremely playful.   109  4.8 The Trip - Little Critter Reading Adventure by Mercer Mayer 4.8.1 Synopsis The Trip tells the story of Little Critter’s road trip with the family to Lake Wakatooke. While in the “Just Read” mode, a version of the story similar–but not identical–to the original print picturebook can be read without any interactions, animations or sounds, in a very simple and limited narrative; in the “Reading Adventure” mode, a much longer and interactive narrative unfolds. Readers help Little Critter pack (and collect objects he will use later on), choose the route among different options on the map, and solve the problems that show up while on the road, which include moving a horse out of the way, cooling the overheated car engine, asking directions after being lost, and fixing a flat tire.  4.8.2 Writing  This app is based on a print picturebook for early readers, and the writing is extremely simple, one sentence per scene, focusing on the activities Little Critter does with his family during the trip. In general, the words are limited to briefly saying what is already obvious in the images. The sentence structure is very simple, in a direct order, and does not contain any poetic use of language, rhymes, or any other literary artefact that could contribute to its playfulness. The narrator is Little Critter himself, telling the adventure in first person. The first scene of the narrative is told in near future, which invites readers to join Little Critter on the adventure, but from the second screen on, the verb tense is changed to simple past. The use of past tense suggests that the story is already over, while the interactions suggest a narrative happening at the present, with readers participating in its development.    110 4.8.3 Animation and narration Besides narrating the written text, the narration in this app includes a much longer verbal text and dialogs between characters, both of which markedly extend every scene from the original narrative. After narrating the written text, usually Little Critter starts a dialog with another character or with the reader. This dialog has varied length depending on the scene, but in many cases it is long enough that the book is transformed into a short animated scene. After the first automatic dialog, the hotspot cues appear on the screen, allowing readers to start other shorter conversations among the characters. These conversations are much more playful than the central narrative itself, and they often make intertextual references to games and play. For example, on screen 8, language play is integrated into the narrative as the ice cream vendors and Little Critter tell riddles to each other. In several scenes, Little Critter and Little Sister play games: the Alphabet Game and Silence Race–a quiet game–(both on screen 5), Yellow Car and singing “Row, Row, Row your Boat” (both on screen 2). While playing these games, they tease each other, which works as another source of playfulness with humorous effects. The narration style is naturalistic: adult actors play adults and child actors play children. They speak to each other in a personal tone, typical of the familiar situation they share. The same tone is maintained in the narration of the written text, or the main narrative, and thus Little Critter treats readers as equals, as companions on his journey. Accents are played as they encounter other people during the trip: on screen 6, they meet a man characterized as being from the country by his vestments and by his accent. This stereotypical representation is used with the objective of generating humour, contributing thus to the narrative’s playfulness. Narration and animation in this app are closely connected since there is little writing and most of the verbal text operates as dialogues that are coordinated with movement in animated   111 passages inside every scene. Because of this characteristic, the app resembles an animated movie, with the difference that readers can control the pace and the order in which these animations take place by controlling the “turn of the pages” or by activating the conversations through the interactions. As readers enter a scene, movement is often limited to small areas, usually the eyes of one or various characters, and is extended as readers interact with the characters and “give them life.” Here, gaze and facial expression are also important sources of meaning. For instance, due to the several problems that occur during the trip, Father gets stressed and angry. This information is never mentioned by the verbal text, but is visible through his gestures in many scenes. The change from one scene to another usually involves the start of an animation that creates a bridge between one situation and the next. For instance, when the family is in the city (screen 2), as readers press the button to move to the next screen, before the scene is changed the car engine “moves” and makes noises, indicating there is a problem with the car, thus creating the bridge to the following scene in which the they stop to fix the car and eat.  One source of playfulness in this app (which is also related to the animations) happens when Kitten touches Little Sister’s bow. This happens in every scene, and, not knowing the kitten has entered the car and is accompanying them on the journey, she blames Little Critter or Father; the animations, though, show that Kitten is indeed the one teasing her. Due to the readers’ point of view through the illustrations being different from that of the girl, the readers see the movement as the kitten touches her. The omniscience of the readers as participants in the narrative–in contrast to the limited view of the characters–promotes dramatic irony, and readers may be especially interested in activating Little Sister’s actions in order to reassert this omniscience, with playful effects.   112 4.8.4 Illustration The illustrations in this app reproduce Mercer Mayer’s traditional illustration style, developed in the 1970s. Thus, the illustrations in this app have a traditional cartoony look of that time, with thick black outlines and watercolour painting. Little Critter is an anthropomorphised rodent animal, not identified as any specific species, with human proportions, in relation to the natural setting and the non-anthropomorphised animals. The family is portrayed as a typical middle class North American family, who goes on a road trip to Lake Wakatookee, on their way crossing through urban, rural and costal landscapes–all settings that are made visible and detailed only through the illustrations. There are other animals in the settings, some of them anthropomorphized, which usually interact with Little Critter and his family, and others not anthropomorphized, which are not able to communicate with the characters and instead participate as pets or wild creatures. The blue car in which the family is travelling has a deep blue colour, which contrasts with the more pastel tones of the rest of the images, giving it salience. In every scene there is a cricket, which, when tapped, moves away from the scene, to reappear in the next image, in a playful hide-and-seek activity. 4.8.5 Interaction The authors have organized interactions within every scene in three categories, which are marked by coloured dots that signal the hotspots. Readers may tap the orange points and collect flash cards. They may tap the blue dots and activate actions and dialogue among characters. In many cases tapping one character generates responses not only from the particular character, but also from the others, who respond to the first’s comment. Through these interactions, readers also have access to parallel stories or to comments from secondary characters who may appear only in that specific scene, but who contribute to extending the meaning of that specific scene.   113 They may also tap the objects marked with green dots and collect objects that are used later on in the narrative; as these objects are needed, Little Critter will communicate with readers and the menu button will flash, indicating readers should access the “backpack,” in which these objects are stored, to retrieve them. These interactions are the most meaningful in terms of reader participation contributing to the narrative because by using these collected objects readers, along with the protagonist, solve problems that appear during the journey. If readers do not collect the objects as they pass through the scenes, they must go back later when they face the situation in which they are needed. Games are also incorporated in the app, as some of the scenes include a simple association game. They are mostly not connected to the narrative and unnecessary to its development, and their objectives are to develop specific skills. When successfully completed, readers receive jam jars; after collecting six jars, they receive an award.  A last way readers can interact with the narrative is by choosing which route the family may take. At several moments, readers are taken to the map and may choose between two paths. These choices do not significantly change the narrative because no matter the path they choose, the family will also end up passing through the other route. Nonetheless, it allows readers to change the order of the scenes, creating slightly different stories. This form of interaction also contains game-like characteristics, and the map through with readers navigate, which works as a contents page, is very similar to the ones present in video-games, in which readers progress to different stages until reach a final objective. 4.8.6 Sound Each scene comes with a naturalistic soundscape, helping to establish the different settings. For instance, when the family goes to the city, noises and sounds of cars honking create the soundscape; in contrast, when they go through the country, they hear sounds of crickets and   114 other animals. There are also sound effects associated with the animations activated by the interactions. Some are realistic, but some are exaggerated, such as when tapping the cricket, which makes a loud jumping noise. Although non-realistic, they work as a response to reader interaction, which feels rewarding, while the real noise would be imperceptible and not give the same effect to the animation. 4.8.7 Summary of The Trip - Little Critter Reading Adventure This app merges the characteristics of picturebooks with animations and games to tell a narrative built on problem-solving situations in which readers must help the protagonist and his family get to Lake Wakatooke. The writing is extremely brief but is considerably expanded by animated scenes that contain dialogues between the characters. The visual text is also expanded by the interactions that allow characters to perform actions and parallel narratives to unfold. Sound effects are naturalistic and support the animations without being used as a significant meaning-making and playful resource. The interactions allow readers to collect and use objects that help in solving the problems they face while on the road.   4.9 Little Red Riding Hood by Nosy Crow 4.9.1 Synopsis In this retelling of Little Red Riding Hood, set in contemporary times, both Little Red and readers are given the power and agency to chose–literally–their own path. First, readers are invited to help Little Red fill her basket of food for Grandma. While in the woods, she faces three forks in the path. By deciding one path or the other, Little Red experiences different situations. In each one, she meets a different animal and collects a different object. When arriving at Grandma’s house, Little Red is not easily fooled. She notices something is wrong, and   115 when the wolf reveals himself, readers are again called to help her run away from him. Yet, Little Red is not afraid and uses the objects she has collected to fight back and scare the wolf away. Depending on the “weapon” she uses at last–honey and consequently bees, a whistle that brings the police to Grandma’s house, or a giant spider–the wolf faces a different ending. Finally, readers help Little Red find the key to unlock Grandma and to unpack the basket and serve up the feast. Considering all the different paths, there are eight different possible stories within this storybook app.  The same team created this narrative, which is part of the same series as the Three Little Pigs app analyzed above. Thus, its structure is similar in many ways. To avoid repetition, this analysis will focus on the aspects that are unique to this app. 4.9.2 Writing Like The Three Little Pigs, the writing in this app is organized around a central narrative complemented by intraiconic sentences said by the characters, which constitute dialogues among them and with the reader. This narrative has a hypertextual structure that allows readers to make choices during the story that change the narrative and its ending. At three points, this narrative offers readers the possibility to choose from two options, representing different paths Little Red may take, and leading her to encountering different characters, to do a different activity, and finally to collect a different object. Although these possibilities change the content of the text, they do not change its structure in any way. 4.9.3 Illustrations The illustrations in this app follow the same style as The Three Little Pigs. The colour palette is very vibrant, and consists mostly of green, which contrasts to the red of Little Red’s riding hood. The owl, who is present in every scene, is also red, which creates a connection   116 between the characters. They are the most salient elements in the images. Some of the talking animals wear pieces of clothing with a very similar pattern, also creating a certain unity among them.  Little Red and her mother live in a typical western middle class family house. Their costumes are contemporary; the famous red cape worn by the girl is a hoodie, like those worn today by mostly every child in urban Western society. Even so, it is hard to discern through the illustrations a specific time period: there is an old-style radio in the kitchen and no contemporary kitchen appliances, yet the narrative could be set in present time. This narrative also includes an animal that accompanies the main character throughout the narrative–in this case, an owl–inviting readers to search the multilayered scenes to find it. The wolf’s characterization in this narrative is different from the one in The Three Little Pigs. The wolf is similarly anthropomorphised, and although he has sharp teeth, his eyes are rounded rather than oblique, which transmits a more friendly and less menacing gaze. Even when attacking the girl, his gaze is not as scary as the other wolf’s. He also wears a beret, which gives him an appearance of sophistication, and his voice is lower and breathier, a voice more seductive than menacing. Charles Perrault (1697), in the version that is considered as the first published version of Little Red Riding Hood, describes the wolf in the final moral paragraph as “charming, quiet, polite, unassuming, complacent, and sweet, who pursue young women at home and in the streets.” (last para.). The representation of the wolf in this app version then draws on historical representations of the wolf and on the sexual undertone attributed to the tale in some versions, especially the most ancient ones.     117 4.9.4 Narration The narration of this app is read completely by child actors and actresses. The narration of the main story is read in a loud voice, simulating public distance, as if the narrative were being read to an audience. The narration of the dialogues spoken by the characters conveys personal distance, a proximity and intimacy between the participants in the communication. The wolf’s voice is low, deep, and whispery. He speaks the sentences in a calmly paced and refined manner, which is seductive or ironic, depending on the meaning he is trying to impart.  4.9.5 Animation and interaction Movements and interaction in this app are thoroughly connected because readers get to control the characters position and mobility, and to decide which direction the girl must take when encountering forks on the path. Moving from one side to the other is the major movement in this app, although characters also have naturalistic gaze and body movements. The situation of Little Red walking through the woods occupies an extensive section of the story. Every time this movement is resumed after she finishes the activities, the wolf is seen hidden and reveals bits of his evil intentions. In these scenes, Little Red is always walking from left to right, and cannot be stopped, but readers can make her run faster or just walk peacefully. This structure resembles that of side-scrolling platform video games, but she is not required to complete any specific action or even defend herself from the wolf. In this app, reader participation is pivotal on every screen, whether interactions activate the conversation among the characters, control their movements, or help them in their activities. The narrative starts with the activity of helping Little Red pack the basket of food for Grandma. Readers must choose the objects from the screen and drag them to the basket. If they drag anything that is not food, Little Red tells them to leave it there. Until readers pack all food, the   118 scene will not appear as “complete,” which activates the blue arrow to move forward. In case readers take too long to finish, characters give them cues as to foods they need to search for. Some of the objects are in areas of the screen that are not completely visible at first glance, so users must explore the setting through interaction to find what they need. When reaching the forest, there are nine possible activities, but readers only get to choose three at a time. Each activity is unique and requires different interactions with the screen to be solved. That might involve simply dragging objects into the basket, moving the character to catch objects that fall from the air, moving the device to pour honey into a jar, or even making music by copying a musical tune. As Little Red reaches Grandma’s house, she again requires the reader’s assistance, helping her run away from the wolf and then using the objects she has collected, and finally by finding the key to unlock Grandma and unpacking the basket of food. In all these instances in which readers participate, they take the role of Little Red, making the participation of the reader more or less uniform in taking the girl’s point of view. One movement that is not related to interactions and that is extremely meaningful in this app is the change in the point of view, which may zoom in and out to create emphasis in a certain situation. This movement is dramatic when Little Red asks about the eyes, nose and mouth of the wolf (screen 13). At each stage, the camera comes closer to both characters, emphasizing the suspense of the moment.   119    Figures 4.10 (A), (B)DQG (C). Little Red Riding Hood, screen 13 in three different moments. The zoom of the camera is used in the moment of maximum tension in the narrative to intensify this feeling. Copyright Nosy Crow, artwork by Ed Bryan. 4.9.6 Sound  The soundscape in this app is very complex and changes at every part of the story to transmit the emotions that help build the scenes. There is one tune for happy moments, played while Little Red is at home (screens 1, 2 and 3), while she is playing games in the forest (6, 8 and 10 in all their versions), and when the wolf has been scared away and she is safe with Grandma having a feast (screens 19 and 20). The second tune conveys suspense. It is played while she is walking through the woods but not under real threat (screens 4, 7, 9 and 11). As this tune is being played, it suddenly stops as the girl reaches Grandma’s door (screen 12), creating an emphatic pause that increases the feeling of suspense. A third tune takes over when the wolf features in the scene (screen 5, 13, 14, 15, 16, 17). It is a tango tune, which simultaneously transmits seduction and danger. There is a progression from a more seductive to a more dangerous effect throughout the story when this tune is played. On its first instance, when girl and wolf meet in the forest, seduction is emphasized (screen 5). Then, when she enters the house and faces the wolf, the tune becomes more ominous (screen 13) and is sustained on a very deep, repeated sound as the wolf reveals himself and attacks her (screen 14). The tune is transformed again into a more active tune, which parallels Little Red running away and fighting back against the wolf (screen 15, 16   120 and 17). Its final version transmits defeat, as the wolf is running away after being thwarted by the girl (screen 18 in all its versions). Thus, the sensorial qualities of sound in this app are used to create meaning in accordance with the other modes present in this app.  The sound effects are limited to supporting the interactions with the screen. So when doing an activity, for instance, grabbing the food for the basket (screen 2), a sound is played. These sounds are not realistic, and work in transmitting to readers the information that the action has been completed. When all the actions in a certain scene are complete, a different sound is played, confirming the “end of the game.” 4.9.7 Summary of Little Red Riding Hood This very interactive version of Little Red Riding Hood allows readers to create different versions of the story, in which the wolf faces different outcomes, depending on the choices readers make throughout the narrative. All the modes in this app complement each other significantly and contribute to the playful aspects of the narrative. The writing consists of two sections: a section that tells the story from a third person narrator point of view, and also a section of dialogues among characters, including the reader, which participates actively in the narrative. Sound contributes significantly to creating the atmosphere of the each scene, which varies from moments of happiness and joy to tension and back to joy. The interactions allow the participation of the reader as a co-author, but also allow readers to control and assume the roles of the characters. Animation is very related to the interactions in the sense that most movements that characters perform are related to the control the reader is exerting upon them.     121 4.10 Summary  In this chapter, I describe how each of the modes that compose the texts of the selected picturebook apps participate in their construction of meaning, especially the meanings associated with their playful features. The close readings reveal some of the narrative potentials of each mode, and these potentials are explored differently in each app. The analysis and descriptions presented in this chapter will serve as a point of departure for the discussion of how each narrative exhibits different forms of playfulness, presented in the next chapter.    122 Chapter 5: Discussion: Playfulness in Picturebook Apps 5.1 Introduction In this chapter, I discuss how each form of playfulness works in each of the books analyzed and cross the results to define patterns of how playfulness can be understood to manifest in picturebook apps in general and how they differ as born-digital or transmediated picturebook apps. Finally, I apply the findings and discussion to answering the research questions of this thesis.  5.2 Playfulness through the interanimation among modes 5.2.1 Introduction As Nikolajeva (2008) posits, playfulness is an intrinsic characteristic of picturebook apps since, as multimodal texts, the decoding of the different modes is, in itself, playful especially if these modes construct a relationship of counterpoint. This section discusses how the different modes interanimate to create playfulness in each of the eight apps in this study, and wraps up by comparing this form of playfulness across the apps and establishing the differences between the born-digital and the transmediated texts.   123 5.2.2 What Does My Teddy Bear Do All Day?  Figure 5.1. What Does My Teddy Bear Do All Day?, screen 7. Irony is created as the visual mode reveals to readers information that is unknown by the main character. Here, the doll is walking behind the girl, but she is focused on spying on Teddy and does not notice anything. Copyright Auryn Inc., artwork Birte Müller. This narrative is built upon the irony created between the verbal text, which combines writing and speech, and shows the girl’s point of view, and the visual/kinetic text, including illustration, movement, interaction and audio, and shows an external point of view. This irony depends on readers to make sense of it and to orchestrate the different modes into one narrative.  The narrative is based on the game between the girl and teddy bear, directly, and these two characters and the reader, indirectly. The readers, in their outsider position, can see both the girl spying on Teddy and the other toys moving without the girl’s knowledge. Readers play a double role in this game: as accomplices of the toys through their omniscient view, while the   124 girl, at the same time, through her informal and personal tone and language, also claims readers to be on “her team.” Considering the multimodality of the narrative, the verbal text is playful in itself as it is rhymed, but it is also playful in the contrast with the rest of the story, contrast that generates the irony that is the central element of the narrative. The images are playful in themselves regarding the parallel visual narrative represented by the mice, while also being central in counterpointing the verbal text, especially through the visual depiction of the doll’s actions. Movement carries a lot of meaning in this narrative, since its presence or absence represents being alive or lifeless, a meaning that expands significantly the visual text in the building of the ironic multimodal text. Interaction, since it involves the participation of the reader and engagement with the materiality of the device through touch, always implicates a certain level of playfulness. Considering its interanimation with the other modes, in this narrative interaction creates the shift from the third person point of view to the first person when the girl spies on teddy through the window and through the keyhole, which also participates in building the ironic effect of the multimodal text. Sound seems to contribute less to the playful aspects, since most sounds are not playful in themselves and just have the role of making the animated scenes more realistic. An exception happens when the mariachi song is played.  5.2.3 Lil’ Red  In this narrative, the relationship between visual, audio, movement and interactivity is very intimate; all are necessary to convey the narrative, and are interdependent. Different from What Does My Teddy Bear Do All Day?, there isn’t any form of counterpoint between modes. Here, all the modes are in agreement, complementing one another and creating one complex, but harmonic narrative. Although harmonic, the narrative depends on the readers; first, because it   125 depends on their interactions to progress, but also because it depends on readers to combine modes and make sense of what these combinations mean. Since the narrative is wordless, the lack of the objectivity and linearity of language makes the process of combining the modes into one narrative even more important and individualized, yet harder describe, since it is not guided or explicated through language. For instance, Little Red’s characterization as vulnerable is built on the combination of modes: her small size, expressive gaze, playful way of moving/walking, her ingenuous impression of the wolf, and childish xylophone “voice.” All these elements build a character that is coherent with the progression of the narrative, as she is in fact eaten by the wolf, and differ from some versions in which Grandma is the one eaten and Little Red is helped by the woodsman, who rescues Grandma. Also, this narrative is extremely playful because many situations that result from the combinations of modes are delightful, and many–but not necessarily all–generate humour. The multimodal scenes have both ludic and narrative functions, but in each scene the level of predominance of one of these functions varies. For instance, screen 7 shows little narrative function, and the scene is unnecessary for the understanding of the narrative. Nonetheless, it shows Little Red having a delightful moment in contact with nature, feeling the wind on her face, observing a frog hunting flies and a dragonfly flitting about. This is an example of a scene that does not rely on interactivity to convey its playfulness, but on the interanimation between modes. Screen 6 shows balance between the functions, since through this scene readers learn that Little Red is being followed by the wolf (narrative function, mostly conveyed by the animation), and readers collect the apples (narrative and ludic), and they also can play by throwing apples at the wolf's head (ludic, conveyed through interaction).   126 5.2.4 The Monster at the End of This Book  This app makes use of every mode as a meaningful resource to convey the metafictional narrative. All the modes are playful, as they present internal irony or build this irony as they contrast with other modes. The speech mode, for example, is ironic in itself. Grover demands that readers not to turn the page while taunting them and giving them cues on how to do the opposite, which is comical. The verbal cues are also accompanied by sound and movement (e.g., as the corner of the pages move) and this combination of modes makes the turn/do not turn the page conflict even more playful. Sound effects also generate tension, contributing to the sense of seriousness, urgency and gravity given by Grover’s words and tone of voice, which is also central to building the irony of this text. The visual mode’s main function is to re-establish the boundaries of fiction and reality, thus contributing in creating playfulness through metafiction. Traditionally, everything that is situated inside the margins of the book or the screen is fictional. Here, the margins are redrawn. The borders of the screen are ignored and the ones in the illustration become the new frontier between fictional and real worlds, with Grover positioned in the latter.  The movement from print to digital media transformed how some of the modes work in this narrative. The main function of the print words on the page has changed significantly with the inclusion of speech in the app format. If in the print book they served to convey the narrative and generate the dialog, in a highly animated app with narration, they are unnecessary to convey the story. Nonetheless, as a metafictional narrative, having the words printed on the screen is fundamental in stating that this is a book, rather than an animated movie, and reinforces the character of the narrative. Movement and interaction are modes present in the print version of this book, realized through the movement of turning the pages, but they suffer significant   127 interferences when transmediated to the screen. In the print book, the physical turn of the pages itself is made playful by the narrative. The virtualization of this movement reduces the playfulness generated by the physical turn of the pages (which was natural and mandatory in print media) to a turn of the page that is artificially reproduced on the screen, on a device that in fact does not require this movement to transmit the narrative. On the other hand, the inclusion of other kinds of interactivity enhances playfulness in other aspects, giving the reader a different form of control and different ways to challenge Grover, and dismantle the barriers he creates between readers and “the book.” 5.2.5 Don’t Let the Pigeon Run This App! This app is a hybrid between book and animated movie since readers cannot turn the page or control the flow of the narrative in any sense and the written text is absent or secondary. Also, this app can be classified as a metafictional comedy for being a very humorous text which comments upon its own nature as a fictional text. This narrative’s playfulness and its proximity with an audio-visual text are completely dependent on multimodality. The combination of speech and body language (which combines visual and kinetic modes), typical of movies and animations, is present here at its fullest expression. The irony created by the pigeon’s communication with the reader depends on the modal ensemble created especially by his gaze, tone of voice, and the lack of content in his discourse. In the game in which the pigeon wants to do something, repeatedly trying to cajole readers to allow him to, the playfulness lies in the fact that readers always refuse. The narrative automatically includes reader responses, which are likewise multimodal: they include the text “NO” written as part of the visual text, the voice of children shouting “NO” and the movement of the pigeon being violently shaken. This is an example that represents what in fact takes place during the whole narrative, in which the   128 construction of the playful signs depends on their constructions as modal ensembles. The gesture of “shaking” the pigeon is also extremely ludic, both in terms of physically interacting with the device, and as a metaphor for the “magical” creation of stories. By entering the game, the reader assumes his role as the one who rejects the pigeon’s desires. 5.2.6 The Heart and The Bottle The Heart and The Bottle is a multimodal narrative par excellence. None of its modes alone could transmit the narrative. Although the structure of the writing resembles that of traditional tales, it has in fact various gaps then filled by the other modes, especially by the illustrations. As an extremely subtle narrative, in which the meanings are never stated but implied, the orchestration of the modes is fundamental in extracting its deep meanings. The verbal text traditionally has the function of clarifying and specifying the vagueness of other modes, but in this case the verbal text is itself brief and vague, generating an extremely touching and sensorial narrative, whose meanings rely heavily on the reader’s personal history and individual interpretation. Sound and movement–as diffuse, immaterial and evanescent as they are–contribute significantly in creating a very sensorial narrative which conveys the meanings of sorrow, loss and grief in ways that the objectivity of the verbal text and the materialization of the visual text could not equally convey. Interactivity converts a narrative, to different extent in each case, into a play-like experience, and this happens in The Heart and the Bottle. It seems odd to talk about playfulness in a book about death and grief, but this book serves as an emblematic example of the fact that playfulness and comicality (although related in many situations) are different concepts. Humour can be an effect of playfulness, but not necessarily. Schons (2011) criticizes the interactivity in The Heart and the Bottle book app, which she calls a “playground-like experience” (p.125), for   129 diverging from the content of the story. Nevertheless, as discussed, the playful aspects of the book in fact contribute to conveying an even more subtle and profound meaning, promoting reflection on the themes of loss and grief.  5.2.7 The Three Little Pigs This book shows a significant balance among the modes, which are all extremely connected to the central narrative, increasing its depth and generating multiple layers of interpretation. The modes are sophisticated and complementary; there is no counterpoint or irony as they interanimate, but they contribute to building different complex multimodal elements that are amusing and ironic among themselves. For instance, the characterization of each of the pigs is constructed in the combination of the words with which they choose to express themselves, in the way they dress up, in their gestures and according to the soundscape used when they are the focus in a certain scene. This level of complexity is not always achieved in traditional picturebooks, in which image and text interanimate yet leave many gaps. In digital texts, much of these pages are narrowed with the superposition of the additional modes. The complex personalities of each of the pigs contrast one another, creating another level of irony which is not specific to any mode and whose effects are amusing and playful.  5.2.8 The Trip - Little Critter Reading Adventure This app presents a multimodal text that is complex and exhibits simultaneously characteristics of a picturebook, an animated movie, and a game. The structure of the screens is that of traditional picturebook: a short writing of usually one sentence and an image that represents what is said by the words, mirroring each other without irony or contradiction. In fact, in most scenes, the writing is not necessary for makings sense of the story because it simply states what is already obvious in the illustrations. Considering the relationship between words   130 and images, the narrative shows little playfulness, yet the narrative is expanded significantly by the addition of animations that go beyond the central narrative. Thus, inside the book structure develops an animated movie structure that it depends on reader participation and control. However, the transitions between one screen and the next reveal mandatory animations creating the bridge between one situation and another. In the print book, through the changes in setting and in the actions and through the linearity of the narrative, the transitions are implied; here, though, they are made explicit. These multimodal animated texts manifest playfulness when they reproduce different forms of children’s play; in addition, some scenes are ironic when they contrast the omniscient point of view of readers and the limited point of view of characters. Finally, this narrative is organized in a problem-solving game-like structure: Little Critter collects and uses objects to solve the adversities that take place during their trip. Every scene of the narrative works like a level in a video game, through which players must progress to get to their final objective. This narrative works, then, in a very complex and hybrid multimodal structure but its playfulness does not seem to be concentrated on its book-like qualities, which are ordinary, but on its movie-like and, especially, game-like characteristics. 5.2.9 Little Red Riding Hood In terms of multimodality, this app has a lot in common with The Three Little Pigs. Verbal, visual and sonic texts make complex layers of meaning that complement each other in building the narrative and its playfulness, each according to its characteristics as modes. The verbal text is playful in its direct dialogue with readers, but also in its complex characterizations. For instance, the words said by the wolf, in a composition of content and intonation/dramatization, are ambiguous and ironic. The visual text includes the game of finding the owl, which is also playful in her commenting on the narrative. The sounds are playful in   131 conveying the shifts in emotions and tension throughout the narrative. Notwithstanding, this app’s differentiation relies on the importance interaction plays in building the narrative; almost every scene in this app depends on reader participation, not only by activating the character’s actions, but by enacting Little Red through implicit and explicit performances. In fact, this retelling of the tale is significantly different from traditional versions because it has explored and expanded actions that were implicit in the original narrative (for instance packing the basket) into explicit activities readers must perform. The number and the variety of interactions, many of them consisting of simple but complete games, makes this narrative extremely playful, exploring the boundaries between books and games. As in The Trip, the structure of this narrative strongly resembles that of video games, in which players have stages they must overcome to get to a final objective. Here, they do not play all the stages: the hypertextual structure allows readers to choose certain paths and thus ignore others, which impacts the result, but yet allows readers to get to a conclusion. The table of contents of this app reflects this structure as it shows a map with first and final stages (from Mother’s house to Grandma’s house), with diverse possible paths to get through it. Nonetheless, different from a video game, the narrative aspect of the app remains strong: although not necessary or repetitive in some passages, the traditional literary verbal text remains in every scene, both in its spoken and typographic expressions. 5.2.10 General discussion As described in the close readings, the interanimation between modes in the analyzed apps differs significantly from one narrative to another. In some apps, the individual modes present various devices that make them playful by themselves. Writing can have rhymes, be comic or have intertextual references; the illustrations can show parallel narratives, hide-and-seek games, and intervisual references; narration can make use of sound quality and social distance to express   132 irony; animations can result in exaggerated body expressions that are comic. In most cases, however, playfulness is built on the combination of modes, and the very cognitive process of combining these modes in the creation of one multimodal text constitutes, as discussed by Nikolajeva (2008), playfulness. For instance, in Don’t Let the Pigeon Run this App!, humour is created through a combination of modes, which is typical of audiovisual and face to face communication: there is exaggeration both in the content of writing, in the dramatization of this writing in narration, in the facial and bodily expressions that accompany this verbal message and  sound effects that enhance this humorous juxtaposition. The expression of humour is a modal ensemble, and could not be transmitted the same way by one mode alone. In most apps, irony is a central concept in generating playfulness, and its construction depends on the interanimation between the modes. Irony can be created by the counterpoint between modes, as part of the structure of the narrative. This is the case of What Does My Teddy Bear Do All Day?, whose narrative is based on a verbal and visual counterpoint; it also happens in The Monster at the End of this Book , in which there is contradiction between what Grover says and the interactions required to continue the narrative, including hints that involve animation and sound–besides the fact that Grover’s discourse is contradictory and ironic. On the other hand, a narrative can present irony even if its modes are in consonance, complementing each other without counterpoint; a narrative may have an ironic content without an ironic structure when it presents multimodal signs that transmit certain meanings contradicting signs with different meanings present in the narrative. For instance, in Lil’ Red the modes work harmonically; the modes complement each other superposing information that generates complex multimodal signs. These signs generate irony in contrast with other signs, for instance, the smallness and naivety of the girl in contrast with the aggressive and massive wolf.    133 The Heart and the Bottle does not use its multimodal features to generate irony, but to convey its sad and deep meanings in subjective and sensorial ways. Playfulness is generated mostly in this app through interaction and reader participation; however, it is a playfulness that does not generate humour, but one that allows the generation of abstract and melancholy meanings. Similarly, The Trip’s playfulness is centered on its interactive features; it is the poorest in terms of narrative in this study, not using multimodality to convey complex meanings. The story itself is extremely simple and the relationship between image and words is of symmetry. The additional modes in digital media are used mostly to generate the game-like experience of the narrative. The weight each mode brings to bear in creating playfulness also varies from one app to the other. Certain resources were thoroughly used in most narratives (except in The Monster at the End of This Book, Don’t Let the Pigeon Run this App! and The Heart and the Bottle) such as the presence of a secondary character seen on every illustration, one that must be found by readers. The visual mode, then, is considered an important source of playfulness. The verbal text, which is traditionally a central mode in the picturebook narrative in partnership with the visual, does not function in most narratives as a source of playfulness. Only What Does My Teddy Bear Do All Day? shows a writing rich in literary devices, such as verses and rhyme, that amplify playfulness. In The Monster at the End of This Book, Don’t Let the Pigeon Run This App!, The Three Little Pigs, The Trip and Little Red Riding Hood, however, although the writing is simple and direct, the narration plays an important role in bringing the text to life in a playful and usually delightful way. The fact that six of these narratives include an extensive part of their verbal texts as dialogue–or the totality in The Monster at the End of This Book and Don’t Let the Pigeon Run this App!–reveals that playfulness is often explored through narration. In The   134 Monster at the End of This Book and The Trip, the narration is in fact significantly more extensive than the writing, as part of the verbal texts have their typographical expression on the screen suppressed, which also happens in “Chick” and “Big Pigeon” modes for Don’t Let the Pigeon Run this App!. In The Three Little Pigs and Little Red Riding Hood, oral communication is also extensively explored in transmitting the narrative, but their visual manifestations are kept on the screen with the use of speech bubbles. Sound in most cases works more in supporting other modes than in creating playfulness by itself. Sound generates playfulness especially when associated with interaction, as a response to reader participation in the narrative.  As expected, interactivity plays a major role in generating playfulness in the narratives analysed; through the combination of the different modes by computer programing, interactivity promotes communication between narrative and reader; it allows readers to reproduce game-like experiences through both physical and cognitive engagement with the narrative. Interactivity makes uses of the modes to generate playfulness, but a playfulness that depends totally on the reader to unfold, creating participatory narratives which present themselves differently at every reading depending on reader’s choices. This is not new or exclusive of digital media since reader-response theory (e.g., Rosenblatt, 1978) and multimodality (e.g., New London Group, 2000; Kress, 2010) have already claimed that every reading experience is unique, but the potential for this uniqueness is expanded through the possibilities of computer programing.  5.2.11 Differences between born-digital and transmediated apps  As the sample in this study is small, the analysis of the apps did not reveal clear patterns or differences that could be generalized as characteristics of transmediated or born-digital apps. The way the apps in each group articulate the modes varies from one app to another, but some aspects that are emphasised in one or the other group can point to some tendencies in the category. The   135 first aspect of note is that both apps that use counterpoint between modes to produce irony are transmediated apps. Nikolajeva and Scott (2001) consider books that show counterpoints are more sophisticated and playful (Nikolajeva, 2008) than books in which the interanimation among modes show symmetry or complementarity. There are a few hypotheses as to why none of the born-digital texts show a more complex and ironic relationship between modes. First, books that show counterpoint usually stand out among the great majority of books in which image and text mirror each other, and publishers are looking for books that are already quite successful to be transmediated, so they may tend to choose books with ironic relationships to be transmediated into apps. On the other hand, the inclusion of interaction seems to add an almost automatic playful element, so creators of born-digital texts might not feel the need to build a narrative with a complex counterpoint structure in order to generate playfulness, since these books can be playful in other ways. Lastly, producers may prefer traditional narrative structures to create born-digital picturebook apps, since the addition of modes, be these modes playful or not, already add complexity to the narrative. In addition, all born-digital apps make their verbal texts heavily dependent on dialogue rather than traditional narration of the story, typical in children’s tales. Even in Lil’ Red, which dispenses the verbal text altogether, the dialogue structure is present visually and through the communication the characters establish with readers through gaze: when they require the reader to interact, they shift their gazes from the action towards readers. This may also be related to the inclusion of interaction in the digital version, which automatically adds a level of direct communication between narrative and readers, then mirrored in the verbal text through the dialogues. Among the transmediated apps, dialogue was central in the verbal text in The Monster at the End of This Book, but this narrative already has in the print version high levels of   136 interaction between fiction and the world of the readers due to its metafictional nature. The Heart and the Bottle’s text is the most traditional of all, not presenting any dialogue, and it has not suffered any changes in its transmediation into digital media. In the case of the transmediated apps, when the written text was created, it shared with the illustrations a much more equal weight in conveying meaning, since these were the only two modes available. In born-digital apps, the verbal text is one of the multiple modes available to convey meaning, thus in the apps analyzed, the creators seem to rely less on the poetic potentials of verbal language to convey meanings and generate playfulness. The visual texts in the transmediated apps also reproduce the tendency of not communicating with readers directly, as manifested through their gaze, which is usually focused on the activities they are performing, except in The Monster at the End of This Book, whose activity in fact involves talking directly to the reader. On the other hand, in the born-digital apps there are more instances in which characters communicate with readers through gaze. In Lil’ Red the characters look at the reader when they are expected to interact with the screen. In every scene of Don’t Let the Pigeon Run This App!, the pigeon is at the center of the page talking to readers, although he is depicted from a side angle. In The Three Little Pigs this happens in the first scene, where characters present themselves, while in Little Red Riding Hood most scenes have the girl looking directly at the reader. The visual text thus works in concert with the verbal text and interactivity in generating direct communication between fiction and readers. Considering sound, the comparison between transmediated and born-digital apps suggests that born-digital apps tend to have more sophisticated sound design, which participates more significantly in the construction of meaning and, consequently, of playfulness. The born-digital apps Lil’ Red, The Three Little Pigs and Little Red Riding Hood present extremely sophisticated   137 soundscapes, which make use of the semiotic resources of sound quality to convey meanings and create a sonic narrative. They also included naturalistic sound effects that promote the readers’ immersion into the settings. Don’t Let the Pigeon Run This App! has a simpler soundscape because the focus of its sonic communication is in the narration. Nonetheless, the sounds incorporated contribute significantly to express the excessive dramaticity of the pigeon’s communication. Most of the transmediated apps, though, also present quality and meaningful sound. The Trip, The Monster at the End of This Book and The Heart and the Bottle utilize high quality, naturalistic sounds that generate an immersive environment and add meaning. What Does My Teddy Bear Do All Day? is the app that presents the poorest sound design, with technical issues that somewhat impair the reading experience and that add limited meaning to the narrative and to the creation of playfulness. In this app, sound was not given the same importance as other modes, especially in comparison with the verbal and visual modes. The only app that waives the book structure in which reader’s get to control the flow of the narrative through “turning the page” is Don’t Let the Pigeon Run This App!, which points toward the fact that born-digital apps may have the tendency to blend the book and the movie formats, while transmediated apps, which present narratives that were created based on this structure, would tend to remain attached to it. Nonetheless, The Trip shows that it all depends on the level of transformation these narratives undergo as they are transmediated. Although readers still keep the control of the moment when they want to move to the next scene through tapping the green arrow, as they do so an animation that connects the scenes is presented, filling the characteristic gap between scenes of print picturebooks, and approximating the format of an animated movie. Also, the movement of changing scenes is conditioned to the end of the animated scenes, so if readers decide to move forward or backward they must wait for the animation to end, which may   138 take as long as a few minutes. The backwards movement is also not as automatic as in a book; there is no backwards arrow and to go to a previous scene readers must access the menu, then the map, and then select a previous scene, in a structure that resembles that of video games.  5.3 Playfulness through metafiction 5.3.1 Introduction Metafiction as the breaking of the conventions of literature to call attention to its fictional nature is, as discussed by Nikolajeva (2008), another form of playfulness in picturebooks. This section analyzes how metafiction manifests in each app and in the sample as a whole, comparing born-digital with transmediated apps.  5.3.2 What Does My Teddy Bear Do All Day? What Does My Teddy Bear Do All Day? is a narrative in which metafiction is present in a subtle and sophisticated manner. This picturebook app questions the notions of fiction and reality when it demands readers to participate in the narrative, leaving the “real” world to play the fictional game, in many instances performing explicitly. Also, this narrative is metafictive in the sense that it is a story about play that is a game in itself, thus discussing the playful aspects of picturebooks as multimodal literary artefacts and the boundaries between literature and play.  Kanatsouli (2012) states that literature in itself is a game, and this narrative promotes this discussion in almost every aspect of its construction. First, this picturebook is about toys and toys are present throughout the narrative as characters and elements in the setting. The girl wants to know what her teddy does when he comes to life, but in other words she wants to know what he plays when she is not around: jump and run on the top of the bed (screen 3), dance with the doll, or paint on the wall (both screen 4). The girl is playing in every scene, even when the toy   139 characters are not involved, such as on screen 5, when she is assembling a puzzle. The writing is rhymed, which, as Kanatsouli (2012) also discusses, is a form of playing with words and sounds. The visual narrative has hidden elements to be found by the reader. Some interactions are games, such as the puzzle (screen 5) or simply spying (screens 7 and 8), and screen 12 literally represents a hide-and-seek game. Finally, the narrative is a game in itself, whose players are the girl, the toys and the reader, who acts simultaneously as a claimed participant in the dialogue the girl, and allied with the toys without “revealing” the truth to her.  5.3.3 Lil’ Red This app is metafictional in the sense that characters’ and readers’ roles are mixed and intertwined. Characters communicate with readers through their gaze, and the narrative induces reader participation through the different artefacts described in the close reading, recognizing the presence of a reader, thus questioning the traditional boundaries between fiction and real world. Nonetheless, this relationship and control given to the readers is somewhat illusory; the narrative asks readers to be part of the story, it cannot unfold without reader participation and they have significant control of its rhythm, yet at the same time readers cannot control the content or change the story.  The naturalistic and immersive environment created by the soundscape and high quality, smooth animations, contributes to softening the boundaries between fiction and reality, and reader participation–as in screen 4 (waking up the wolf)–transforms the reader into an agent inside the narrative, made responsible for its development. Finally, as a wordless retelling of Little Red Riding Hood, this app also counts on readers’ previous knowledge of the tale to fill in the gaps of the narrative. It points the readers to the possibilities of fiction while eliminating verbal text but including other modes.   140 5.3.4 The Monster at the End of This Book The Monster at the End of This Book is a narrative built completely upon the notions of metafiction and self-referentiality. It compiles many metafictive devices that call the attention of the reader to the fictive narrative itself especially by exploring the materiality of the print book. Although transmediated into an app, the story is constructed upon the conventions and gestures of the print book, investigating the performative act of manipulating the book as an artefact. Although a digital reading experience, this app seems to preserve the enchantment of the print book as a dynamic form of text for children. Although now in the format of an app, The Monster at the End of This Book makes a parody of the conventions and gestures of the print book while at the same time teaching young readers these same gestures and conventions, and how they change in the transmediation from print to app. Nonetheless, print and literacy are represented in this app in a traditional way, as “skills” readers must learn and that are taught by Grover also in an traditional manner: the words are highlighted, and Grover reads one word at a time, tracking the writing with his finger. The metafictional nature of this app also manifests as the boundaries between fiction and reality are being challenged. The multimodal narrative proposes the re-establishment of the boundaries between fiction and reality: the readers are addressed directly; Grover is simultaneously in the fictional and in the “real world,” and readers must step inside the narrative to interact. This discussion is taking place as readers are posited as agents in the act of reading a book, as masters of the development of the narrative, as they control its realization or not by the simple gesture of turning the page.  Another metafictional questioning takes place in this narrative in the fact that Grover is not a reliable narrator. Narrators, especially in children’s fiction, are usually presented as authorities   141 and the fictional world, even if fantastic or surreal, is presented as truth. Here, nonetheless, readers are invited to a new kind of narrator, which, in his contradictory nature reassures readers’ power and responsibility in controlling the narrative. Finally, intertextuality as metafictional device is also present in the app on screen 7, where Grover references the story of the Three Little Pigs. 5.3.5 Don’t Let the Pigeon Run This App! Don’t Let the Pigeon Run This App! utilizes a series of metafictive devices to build a narrative that plays upon the relationship between character and reader and discusses its own narrative structure through the different levels of reader participation. The narrative is a metafictional game in which the power readers have upon the pigeon is at play. No matter what the pigeon wants or his arguments, readers can and will deny it; the playfulness relies significantly on this back and forth of demand and denial, and the pleasure it provides to readers as their power is constantly restated. The boundaries between fiction and reality are unclear as the book is theatricalized, Pigeon addresses the reader directly, he comes out of the narrative into real world. Simultaneously, readers are automatically included inside the narrative via their expected responses–“NO” and laughter. Randomness is also used in this story as a metafictional device. Here, in the “Egg” mode when stories are randomly created, but also in the “Chick” mode, because the options readers may choose from as they build the narrative may vary every time they start interacting with the engine. Randomness is playful because it adds the element of surprise. Moreover, since the contents that are added have an absurd and nonsensical tone, it is extremely funny to find out what will be the next unconvincing argument used by the pigeon. This mechanism, though, reveals that in fact the app does not tell many different stories, but only one, repeated with some   142 internal variables. The app is thus telling the story at the same time that it is teaching readers about its own narrative structure, a knowledge that will be put into practice in the “Big Pigeon” mode. Such possibilities of interaction with the narrative posit readers as co-authors–a position which is even credited as readers’ names are incorporated in the home screen–a possibility which also has high metafictional potential. As authors, readers are invited, again, to reflect on the structure of the narrative. Here, they are not positioned in or outside the narrative, but prior to the narrative. This possibility reaches its maximum potential in the “Big Pigeon” option, in which children, having mastered the narrative structure through their experience with the other modes, are free to incorporate their own ideas and express their creativity as well as their dramatic potential through audio recording. This narrative is also a replication of the familiar situation of child and parent struggle for power, in which children usually play the pigeon’s role, with parents denying them things. Through this narrative, readers are then shifting from a situation of being controlled to being in control. All the artefacts used by children to manipulate parents in giving them what they want are deconstructed here as they are enacted by the pigeon, revealing to the children that in fact parents know their strategies, and making them realized that, as the pigeon, they may not be able to always get what they want. Don’t Let the Pigeon Run This App! is thus a role-playing game which discusses the parent-child relationship through the metaphor of the reader-character relationship, and the possibility of playing the role with power may be extremely rewarding and fun for children.   143 5.3.6 The Heart and the Bottle The Heart and The Bottle is not essentially a metafictional narrative, yet as a digital text in the context of postmodernity, it makes use of some metafictive devices. Interactivity and participation promotes the questioning of the boundaries between fiction and reality since readers’ actions are integrated into the narrative. In this case, reader participation is diverse, varying from situations where they only activate an animation to situations where their interference is fundamental in the meaning-making processes. Nonetheless, the reader is never directly recognized as a participant in the narrative and referenced or addressed by it, so the distinction between fiction and reality is still pretty well defined, and the interactions represent merely a different way for readers to experience the narrative, similar to, for example, a change in point of view. A final metafictive device present in the narrative is the promotion by the text of fiction as a ludic and joyful experience, which opens the hearts and minds of people. At the end of the narrative, as the girl has her heart back, she goes back to a joyful state by sitting on Grandpa’s chair and opening a book, in a direct reference to the value of books. 5.3.7 The Three Little Pigs This narrative is also not essentially metafictive, but it presents some devices that discuss its fictional nature. This happens mostly through the fact that readers are recognized by the characters from the very beginning and overtly expected to participate in the narrative, even if their power is limited by a predetermined story they cannot change. On the home screen, the three pigs teach readers how to interact with the narrative. As the narrative starts, the first scene (screen 2) works as an introduction between characters and readers (e.g., the pigs address readers with “How are you?” and “Nice to meet you,” among other options, and discuss the fictional quality of the story as they say, “We hope you like our story”). Also, when in danger, the pigs   144 ask for help (screens 12 and 15), to which readers can respond through interaction by making them run away faster or slowing down the wolf’s van. On screen 3, the wolf is hiding outside the window, and when tapped he asks readers not to tell the pigs he is there, establishing the dual role of the readers: that of accomplices for both the pigs and the wolf. This relationship is confirmed latter through the interactions, in which readers can play the parts of both “heroes” and “villain.” As readers enter the narrative, they play the roles of different characters through the different kinds of interactions, sometimes taking action as pigs and sometimes as the wolf. The metafictional nature of this app, thus, manifests when the boundaries between fiction and the real world are established as the pigs address the reader directly, talking about “their story”; yet, these boundaries are permeable, since readers can get in and out of the narrative in various forms, controlling the characters or even enacting their roles through performance. 5.3.8 The Trip - Little Critter Reading Adventure The Trip is not a quintessential metafictive narrative, but it presents some metafictive devices that promote, even if indirectly, the discussion of its nature as a fiction. Here, readers are recognized as participants in the narrative and necessary for the problem-solving structure of the text. On the first screen, Little Critter presents himself to the readers as he explains how to play the game on the first screen. He also addresses the readers as he suggests they collect the objects and later asks them to check the backpack when these objects are needed to solve the problems that show up along the way. The exploration of this device in this app is made superficially, being restricted to the moments when reader participation is necessary, and addressing the reader in this case seems to have been incorporated as a design solution for how to naturally integrate into the narrative the instructions of how to use the app, rather than as a literary device. Yet, this   145 strategy, accompanied by the interactions themselves, which promote the participation of the reader, change the nature of the narrative, giving it metafictional characteristics. This app still counts with another metafictive device: intertextuality. Traditional children’s songs and games are referenced by Little Critter and other characters, reinforcing the narrative’s playfulness. 5.3.9 Little Red Riding Hood Similarly to the previous examples, this app presents reader recognition and participation as metafictional devices. The boundaries between fiction and reality are broken as readers are explicitly addressed by the characters, who dialogue with them and ask for their help to finish certain tasks; readers’ actions are critical to the development of the story. Also, as the hypertextual structure allows readers to build different narratives with distinct endings, participating as co-authors. Co-authoring here is more limited than in Don’t Let the Pigeon Run This App! as readers are not allowed to propose new endings, but are restricted to a certain range of possibilities that are pre-determined. Yet, their participation in this narrative is necessary at various stages and with different functions and results, so the feeling of immersion and of integration between fictional and real worlds is constant in this app.   146   Figure 5.2. Little Red Riding Hood, navigation screen. The hypertextual structure of the narrative is represented visually through the navigation menu, which shows all the paths readers and Little Red can take and the different objects they can collect at each activity. Copyright Nosy Crow, artwork by Ed Bryan.  5.3.10 General discussion Linda Hutcheon (1980) posits that metafiction can be classified into diegetic, when metafiction manifests through the narrative structure, or linguistic, when it manifests through language–which here will contemplate not only verbal language, but also visual, sonic, kinetic, and interactivity. Both can still be divided into overt metafiction, when the narrative shows self-consciousness, communicating directly with readers, or covert metafiction, when the metafictive nature of the text is realized through the story, but not clearly stated. In the sample of this study, all narratives expressed metafiction in some way. The Monster at the End of this Book and Don’t Let the Pigeon Run this App! were selected for analysis because they are one of the few picturebook apps available whose narrative address metafiction overtly and as the theme of the narrative, thus dietetically. All the other selected stories, present some linguistic metafictive devices. In most cases, the device is the breaking of boundaries between fiction and reality through the direct address of the reader by the narrator or a character. In What Does My Teddy Bear Do All Day? , it is ambiguous whether readers are being addressed directly or not, since there are no language markers that confirm towards whom the girl is posing questions, but   147 considering the fictional game which is established early on, readers are deemed participants of the narrative in one way or another. Similarly, in The Trip, although the reader is directly addressed, this happens in limited situations during the narrative, and always with the intent to make readers interact with the screen and do the actions necessary for the development of the game that takes place inside the story; it does not have a direct literary function but yet this instructional function impacts the narrative and transfers to it metafictional attributes. In Lil’ Red, although there is no verbal text, readers are clearly addressed through the character’s movements and gaze, and their participation through interaction is fundamental for the development of the narrative. In The Monster at the End of this Book, Don’t Let the Pigeon Run this App!, Little Red Riding Hood, and The Three Little Pigs–the last to a lesser extent than the others–readers are central participants in the narrative; these narratives are built taking into account reader participation and cannot unfold without it. The Heart and the Bottle is the only case of completely covert metafiction. Although readers can participate through the interactions, their participation is not acknowledged by the narrator at any time.   Addressing the reader directly is not so common in the universe of print picturebooks, metafictive narratives being a very small number in the thousands of books published every year. So why, in picturebook apps, are metafictive devices so pervasive? I believe this is due to the fact that all the narratives analyzed are interactive. The inclusion of interactivity in digital media automatically implicates the participation of the reader into the story. Whether this participation is meaningful or interferes in the story is besides the point, but all interactive picturebook apps expect that readers will actively engage to some extent, so having the narrator or a character addressing the reader, just as a means of giving instructions (as in The Trip) or establishing a game and a power relationship (as in The Monster at the End of this Book and Don’t Let the   148 Pigeon Run this App!), seems to be a natural feature of the medium. Also, for this same reason, print books that are interactive in some way (like What Does My Teddy Bear Do All Day? and The Monster at the End of this Book) are natural candidates for transmediation into apps. Thus, the inclusion of interactivity in picturebook apps automatically works as a metafictive device, promoting the breaking of the boundaries between fiction and reality. Readers enter the fictional world as participants becoming, thus, part of the fiction as well as its co-authors. Since literature is always a form of play, as discussed in the literature review, the difference is that the power given to readers is amplified as they can interfere, at different levels in each narrative, in the story, and that their bodily engagement is more significant in this form of reading than in reading most print picturebooks–remembering, of course, that reading a print book also involves bodily engagement, which becomes invisible in the conventions of reading, but is discussed, for example, in The Monster at the End of this Book. Discussing the nature of fiction by positing readers as co-authors may also have different levels and implications. This is explicitly stated in Don’t Let the Pigeon Run this App!, as readers are included in the credits, alongside Mo Willems, as authors. This app promotes a higher level of co-authoring than the other examples, since here readers have the option of creating elements from scratch that are incorporated into the narrative, blurring the boundaries between the production and consumption of texts for children. In Little Red Riding Hood, the co-authoring possibilities are more limited since readers have to choose among the possibilities given by the authors, yet their choices generate different endings to the narrative. In The Trip, readers are allowed to make choices during the narrative, but their co-authoring is limited since, in the end, they are forced to pass through all scenes without any significant change in the story as a whole.   149 5.3.11 Differences between born-digital and transmediated apps  The two narratives that present covert metafiction in the sample of this study–The Heart and the Bottle and What Does My Teddy Bear Do All Day?–are in the group of the transmediated apps. In The Trip, the overt metafiction comes into play as the reader is addressed directly, but such address happens in limited situations, which seem to work more as instructions than with the intention of questioning the boundaries between fiction and reality. From print books with little to no metafictional devices, these narratives were transmediated and significantly transformed by the addition of modes and, specially, by the addition of interactivity. Readers are always participants in literary texts since they are the ones who make sense of it, but the inclusion of interaction materializes their participation as the narrative becomes dependent on their physical engagement with the text to develop completely. Thus, the very property of being interactive advocates, in the modes that constitute the text, the recognition of readers as participants. This happens in the verbal text–writing and narration–usually by creating a dialogue between narrator or characters and readers. In the visual mode, this communication can be established through the facial expressions and gazes of the characters, as we see in Lil’ Red.  Such an address can also happen through interaction itself, when readers do not play the role of a character but act on their own while performing, which will be discussed in more detail in the next section. As a conclusion, born-digital apps, which are created from scratch as interactive texts, tend to use the devices described to overtly promote the communication between reader and narrative, while transmediated apps come from more traditional narratives where these devices were not foreseen. Yet, in the process of transmediation, new devices can be integrated, as in the example of The Trip, in which the digital version includes the direct address of the readers at least in the form of instructions on how to interact with the screen, and as in What   150 Does My Teddy Bear Do All Day?, when readers tickle the girls feet and the character responds to an action made by the reader and not by another fictional character.  5.4 Playfulness through performance 5.4.1 Introduction The engagement of the body is an important source of playfulness while reading and interacting with books, and it can happen through explicit performances, when the reader enacts the role of the character by reproducing his gestures, or implicit ones, when readers use their bodies to activate an action in the narrative, but without reproducing the movements of the character. In this section, I analyze how performance takes place in each of the apps in the sample of this thesis. Finally, I systematize the various levels of implicit and explicit performances and also the value each performance has in terms of fictional (when reader performance takes the place of a fictional element or character in the story) or metafictional, (when readers participate as characters themselves or as co-authors of the narrative). I also compare the differences and similarities between born-digital and transmediated apps in regards to performance. 5.4.2 What Does My Teddy Bear Do All Day? Implicit performance in this picturebook app is recurrent and happens when the reader touches the screen and activates an action by a character. They usually activate a movement and its sound in actions that may contribute to the counterpoint between modes discussed in the section on playfulness as interanimation. In other scenes, these actions constitute the central message, which is described in the verbal text and visualized though the images and animations. For instance, on screen 2, the text talks about Teddy jumping and running, but the initial image   151 shows Teddy sleeping. Through the activation by the reader, verbal and visual/kinetic texts connect and make sense.  On screen 6, as readers tap the girl’s foot, her reaction is as if they were tickling it. In this instance, there is proximity between the gesture and its effect. Also, readers are not activating or controlling the character, but directly establishing a communication with her. Explicit performance is present in this narrative through the interactions that put the reader into the role of the girl, playing with a puzzle, drawing, looking though the keyhole or through binoculars. All these interactions are deeply related to the visual mode, involve little movement, and do not involve any sound effects. They connect with the verbal text–both speech and writing–in the sense that they evoke visually the point of view of the girl while the rest of the visual narrative uses an external point of view. Through these explicit performances, readers have the chance to view the world through the narrator’s eyes, and in these cases, there is no contradiction between verbal and visual text. Reader participation, then, is intertwined in the construction of the forms of playfulness previously discussed, promoting the construction of an ironic multimodal narrative and exploring the boundaries between fictional and real. 5.4.3 Lil’ Red Most interactions in this app constitute implicit performances. Readers tap a character or an object and activate an action from the character or, in the latter, an action towards the referred object. These interactions are mandatory for the continuation and unfolding of the narrative. All the images in this app show an external point of view, and readers, in these kinds of performances, remain apart, acting on the surface, initiating the actions from the characters, but mostly without becoming a part of the narrative themselves.   152 As a linear highly animated app whose narrative is defined and unchangeable, this book approximates an animated movie without loosing its essence as a book. As Klassen says, one of the central differences between a movie and a book is that in a book “the viewer is moving at their own speed through the story” (Arrant, 2011, n.p.). In this app, this is especially true because not only do readers have the power to choose their pace on changing the screens, but they also choose their pace when activating the actions by the characters through these implicit performances, which is not possible in a movie. The level of playfulness of these interactions varies significantly. Many constitute little games inside the narrative, such as the mushrooms that become a musical instrument, or the apples that fall and are collected by the girl (both in screen 6). The importance of these interactions for making sense of the narrative make the story as a whole extremely playful because it becomes a game of action and reaction between reader and book. The importance given to interaction in this app creates a significantly playful reading experience. The mandatory need for reader participation at every moment and the lack of verbal text approximates this picturebook app to a video game, but the rewards given to the readers as they participate are the narrative itself. Instead of points or levels, they are rewarded with bits of the story that, in sequence, build the narrative.  There aren’t clear examples of explicit performance in this app, but in some instances, the role of the readers change from starting the actions of the characters to interfering in the story. This is the case of screen 4, when the reader is responsible for waking up the wolf, and on screen 6, when they throw apples at him. In these situations, readers are interfering in the narrative as an omniscient participant; readers are not playing a secondary role but their own. In the first case, the action has consequences to the narrative: the reader is the one who sends the wolf out to encounter Little Red–although the story continues the same way if readers move to the next   153 screen without waking him up. As Stichnothe (2014) mentions, the interactivity in this narrative makes “author and user become ‘partners in crime’ in making the gruesome events of Lil’ Red’s temporal demise happen” (Required user participation and narrative structure, para. 7). In the second case, reader’s actions do not interfere in the narrative. The reader throws apples at the wolf’s head, while Little Red remains apparently unaware of both the wolf and the reader’s actions on him. The wolf reacts with surprise, confused and suspicious, unaware of the presence of the reader, which produces a comical affect and puts readers in control of the situation. In that context, readers’ actions against the wolf, besides being humorous, may be interpreted as an attempt to protect or warn Little Red of the presence of the wolf. In some secondary interactions, when tapping, the response is also direct as if readers were inside the narrative. For example, when they tap certain wooden objects, the objects emit the sound of tapping on wood. Here, also, readers remain themselves, i.e., they are not taking on the role of any character–when tapped, Little Red does not go herself to the fence and tap on the wood. The readers are immersed in the setting, entering the secondary world of fiction, but, at the same time, the setting has become an extension of the real world, and tapping wood inside the screen has the same effect as tapping wood outside it. Thus, although these interactions are not necessary to the understanding of the narrative, they play with the readers’ role in the narrative, their movement in and out of fiction, as a metafictional device. 5.4.4 The Monster at the End of This Book In this narrative, readers are characters and play a central role in the story, their own role as readers, the ones with the power to turn the page and the responsibility to let the narrative unfold. The narrative is established through the dialog between Grover and readers, but, in most instances, readers do not control Grover through the interactions. This narrative is conceptually   154 built upon one very specific performance, which is at the same time a convention of the act of reading: the gesture of turning the pages. Both print and digital versions of this narrative are built upon this performance, but, as an app, the performance has been virtualized: on digital devices, readers tap the corner of the screen to “turn” it. Yet, the meaning generated by the virtual performance remains the same as the one from the actual performance in the print book. Although the gesture necessary for turning the pages in a digital book differs, its function remains the same: moving the narrative forward, in this case, against the narrator’s will. Most other interactions also constitute the reader’s actions as a character in the narrative; for instance, on screen 5, when readers tap Grover, who has his arms open, they are “tickling” Grover, and when tapping the knots, they are untying them. Readers are not in fact reproducing the gestures of tickling or untying a knot, but there is significant proximity between their gestures and the consequences of these gestures; it seems that these performances lie somewhere between an explicit and an implicit performance. The app version of this story was extensively enhanced in its playfulness because several performances that were implied in the print version could be accomplished here, enabling the reader to challenge Grover over and over again, reasserting their power as readers and characters of this metafictional story. Implicit performances take place when readers touch Grover causing him to say something out of the central narrative, such as repeating where to touch to turn the page. However, even if readers do not interact with him, after a little while, the character says those sentences anyway, which makes these interactions secondary. Also, readers can tap on the print words, causing Grover to repeat that part of the text (e.g., screen 11) or generating visual typographical expressions of Grover’s mood (screen 6). These interactions, then, are not necessary to the   155 development of the story, but they contribute to the construction of meaning in the multimodal narrative.   5.4.5 Don’t Let the Pigeon Run This App! Performance in this app is hard to be categorized as either implicit or explicit, because this narrative shows a great level of participation with very limited direct interaction with the touchscreen during the storytelling experience. After the story starts, the only interaction required by the reader is to shake the pigeon–which means shaking the device–to start the narrative, in a gesture that requires explicit bodily engagement. This gesture can be interpreted as a metaphor for the process of “scrambling” the elements of the story, especially when playing the option “Egg,” in which a random version of the story appears and the reader has no faculty for decision-making. This performative act can also be understood as an assertion of the readers’ power, which shakes the pigeon as soon as he appears on the screen, affirming their position of control. Despite being limited to one instance per story, this gesture is very meaningful in the overall structure of the narrative. Although readers are not required to shake the device as they deny the pigeon his will, accompanying the narration of “NO,” the pigeon is automatically shaken, repeating the initial shaking interaction. The possibility of controlling the pigeon’s actions in this app takes place prior to the unfolding of the narrative in the “Chick” and “Big Pigeon” reading modes. Nonetheless, through these interactions, readers are not playing the role of the pigeon, but in fact are playing the role of authors, or co-authors. Controlling the pigeon’s actions would be considered an implicit performance since readers are not enacting this role. Yet playing as co-authors, readers are engaged both intellectually and physically. The intellectual input readers lend the story, especially in the “Big Pigeon” mode, has them enacting the role of authors, for they are free to   156 create and incorporate their ideas into the story. Their participation as authors works not only in the conceptual level, when they can incorporate whatever they want into the story, but also with significant physical engagement, since readers record their own voices, thus enact and interpret their ideas, participating also as narrators. Finally, the engine that allows them to participate in authoring the story makes explicit what in fact happens in any narrative: readers are always co-writing the narrative as they read it. It transforms the narrative into a reading-writing metafictional game. This app is unique amongst the sample of this study because it presents performance without interaction. It includes, as part of the text, readers’ anticipated performances as characters who interact with Pigeon: as typical in comedy shows, the audience’s expected reactions are incorporated into the narrative–laughing, and in this case, saying “NO”–a response which is automatic and cannot be controlled by the reader in any way. This device creates a dubious relationship between narrative and readers. If on the one hand readers are recognized as extremely participative and given the power to co-author the narrative and to deny the pigeon his wishes, on the other hand the laugh track (which is implicit in the other Pigeon books published in print version) suggests a rather passive role, more associated with the role of the audience in movies, in which they are not allowed to formulate their own responses but rather to follow what is suggested in the narrative. 5.4.6 The Heart and the Bottle In this app both implicit and explicit performances are present. In most instances, the implicit performances constitute interactions in which the reader activates an action for the characters, which is accompanied by appropriate naturalistic sound effects. They are more frequent than explicit performances and are generally playful in giving life to the characters. In   157 the first part of the narrative, these interactions transmit the joy the little girl used to have with her grandfather. Later on, they work in transmitting the girl’s anguish, as she cannot take the heart out of the bottle. The interactions, thus, allow readers to experience more emphatically the shift in the girl’s emotional state throughout the narrative in ways that are hardly possible in print media. In the implicit performances present here, the gestures used by the reader to control the characters do not have any relationship with the movement the character is doing. By tapping the screen, readers activate the girl’s actions of trying to open the bottle, which includes using tools such as a hammer, a saw, a drill, etc. In this scene, readers are given the option to decide which tool the girl is going to use, which is ludic, and yet has extremely uncomfortable effects as she tries to break the bottle, mostly conveyed by the sound of glass almost breaking.  Explicit performance happens in several instances in The Heart and the Bottle. First, readers can imagine and draw forms on the starry sky with their fingers, as if they were living that situation. This transition into the fictional world is also made possible by the realistic soundscape of the scene, creating an immersive environment. The point of view is dubious, since the reader can see the girl looking at the stars and, simultaneously, draw stars on the sky as if they were seeing it through the girl’s eyes.  Figure 5.3.  The Heart and the Bottle, screen 7. This scene shows simultaneously an external point of view, as we can see the girl and Grandpa laying on the grass, and the point of view of the girl, as the reader can draw shapes on the sky, assuming her role through the interaction. Copyright Oliver Jeffers.   158 Another central example of explicit performance occurs when the girl is making a drawing, and a new screen, in which readers can make their own drawings, opens; here they assume the point of view of the girl as she draws. This performance has added meaning when the drawing created by readers is incorporated into the story and shown on a frame on the wall as the girl faces the empty chair. A metaphorical performance happens as readers erase the starry night and the beach landscapes from the screen (screens 15 and 16). This is an example of explicit performance in which readers enact an emotional state, thus the girl’s oblivion materializes via the gesture as she takes out her heart. An explicit performance in which readers’ and characters’ gestures mirror each other happens when the girl decides to take her heart out of the bottle; readers are asked to shake the device, which activates the girl’s action of shaking the bottle (screen 20). This act of shaking indicates a certain level of violence, and the image and sound of her heart rattling inside the bottle transmits pain and agony. Explicit performance in this app, thus, can be ludic without in any sense being fun or humorous, and can induce readers to enact not only the character’s actions but also their emotions. Such performances contribute significantly in evoking the sad and touching journey of this girl as she overcomes the pain of her grandfather’s death.  5.4.7 The Three Little Pigs  Most interactions in this app constitute implicit performances. Readers control character’s movements or activate their dialogues by tapping or dragging them; their control over the characters is slightly different from the previous apps analyzed since they can move, with certain limitations, the characters through the setting, in a control similar to that available in side-scrolling video games. In every scene, by interacting with a character, there will be some sort of response, which is extremely rewarding, but readers cannot in fact change the narrative. For   159 instance, even if readers delay the pigs when the wolf is chasing them, he will never really get them. Yet, the fact that, at least in the “Read and Play” mode, the narrative does not evolve without reader participation makes readers essential to the development of the story. The most significant implicit performances take place when readers “help” the pigs build up the houses. In these instances reader participation is very indirect, since the action is reduced to tapping the characters; yet readers may feel responsible for the construction of the houses as their interaction with the screen is required for the pigs to execute the construction. As a result, these constitute extremely playful and meaningful performances.  Explicit performance in this app happens when readers are invited to blow the device to start the wolf’s blowing down of the houses. This is a unique use of the technical possibilities of the device to promote an interaction that is theatrical, in which readers enact the role of the wolf dramatizing the story. This is the only explicit performance in the app and it is repeated three times, as the wolf blows down–or tries to blow down–the three houses. Nonetheless, the possibility of playing the narrative is extremely ludic; this performance approximates the narrative with a make-believe game, in which children engage their bodies to live other situations and theatricalize the narrative. In this instance, readers assume the point of view of the wolf, while the writing and illustrations present an external point of view. Nonetheless, this explicit performance is not mandatory for the development of the narrative. Instead of blowing, readers can simply tap the character, transforming it into an implicit performance, in which they are only activating the action without properly enacting it.  5.4.8 The Trip - Little Critter Reading Adventure This narrative presents diverse interactions as described in the close reading. Activating the character actions and collecting/using objects to solve the problems which appear throughout   160 allow readers to participate and control the narrative, mostly through implicit performances. Tapping each character, readers get that character’s particular response to the problem faced in that specific scene, permitting readers to shift through the various points of view. These performances are a little ludic per se, since reader engagement is limited to tapping the screen, while their content may be ludic, since it may activate, for example, a joke or a game between the characters. The interactions that allow readers to collect and use objects according to the development of the narrative are considerably more ludic because they transform the narrative into a game. Readers assume the role of partnership with Little Critter to solve the problems and continue the journey. The role of the reader in these performances, though, is not clear. If, in one way, Little Critter addresses the reader as an external participant, readers are, at the same time, always playing the role of Little Critter in their actions. Readers are in control, for they are the ones who collect the objects and then access the backpack to use them. Yet, they do not perform explicitly because when they tap an object to be collected, this object goes into the hands of Little Critter, who then puts it in the backpack; when readers want to use the object, they tap the menu, enter the backpack screen, and finally choose the object, which likewise appears in the character’s hands. This procedure of accessing the object through the menu changes the point of view for a short instant, and gives the false sensation that in this situation readers are acting as characters in the narrative, but in fact this is just a complicated path to accomplish an implicit performance. Finally, through choosing which routes to take in order to reach the lake, readers are also performing. The possibility of choosing different paths gives readers a slightly higher sense of control over the narrative in comparison to traditional stories.    161 5.4.9 Little Red Riding Hood Most of the interaction in this app constitutes implicit performances, in which readers control the characters without actually enacting their actions. In some instances, readers control the movements of characters by dragging them from one side of the screen to the other, similar to traditional video game controls, thus not enacting the movement, but instead controlling a character who is performing an action. The possibility of controlling the characters and the similitude with video games is one of the sources of playfulness in this narrative. Be that as it may, in many cases, even when readers do not enact, the movement necessary in the interaction has a clear connection to the movement necessary to enact it. For instance, when filling the jar with water, readers must reproduce with their finger the same movements they would do if filling the jar in the pond, but then using the whole hand/arm. Though implicit, this form of interaction approximates an explicit performance. Also, in many of these interactions, readers are recognized as participants in the story, as Little Red asks for their help, and in many cases, their action does not have them clearly controlling the character or making the character reproduce the actual movement necessary for that action. For example, when colleting objects (e.g., filling the basket with food or picking up acorns) readers drag the object in question to the basket, and Little Red does not move. The point of view of the interactions is that of the reader and the visual/kinetic responses of these interactions suggests readers are being themselves, and thus working as characters in the narrative. One significant form of performance in this app occurs as readers have the power to choose between different paths at three points in the narrative; this is another device making this narrative resemble the structure of a video game, thus contributing markedly to its playfulness. These performances are implicit, because readers must move Little Red (dragging her or   162 touching the path) the way they want her to go. Although readers do not realize it at first, these choices interfere in the conclusion of the narrative, or how Little Red Riding Hood defeats the wolf, which in turn leads to another instance of performance in the app: readers playing the roles of co-authors. It is hard to classify this performance as implicit or explicit, though I believe it is more on the implicit side of the spectrum because, different from Don’t Let The Pigeon Run This App!, readers do not enact the role of the author as the options are already pre-determined and they are not clearly recognized as co-authors, although they are recognized as participants. Finally, this app also includes several instances of explicit performances: readers must hold and tilt the device as if pouring the honey into a jar; when attacking the wolf, they blow the dandelions into his eyes or blow the whistle, which calls the police. By enacting these actions, readers engage in a physical role-playing game, as distinct from a symbolic role-play which happens as they interact implicitly in the instances described above.   5.4.10 General Discussion The framework for the analysis of performance for this study was established considering that performance can be either implicit, as when readers do not enact an action but control the action performed by the character through their interactions, or explicit, as when they engage their bodies in reproducing the same gestures as the characters, theatricalizing the narrative. However, after close reading the picturebook apps, the performances turned out to be much more complex than this simple dichotomy. The level in which the body was engaged in these performances varied significantly from one app to the other or even within one narrative. Thus, the instances of implicit and explicit revealed to be, in fact, extremes of a spectrum, with many examples that fit in between, flirting with both extremes at different levels and simultaneously.    163 On the extreme of implicit performances lay the interactions in which readers tap a character and activate their actions. This is a very simple and common form of interaction that happens in all apps analyzed, with the exception of Don’t Let the Pigeon Run this App!, in which most interactions preceded the moment in which the story is told. This form of performance usually happens in scenes whose point of view of the illustration is external, so as readers interact with the screen, they see the performance of the character unfold in front of him.  Moving towards the middle of the spectrum, in some instances the movement required to perform a certain interaction simulates, although with significant simplification, the movement required for that action to be performed. For instance, the Little Red Riding Hood app contains various instances of this kind of performance like when readers fill the jar with water or when they collect acorns in the forest.  Approaching the explicit end of the spectrum, we have performances in which readers assume the role of the character and reproduce gestures that are very similar to those acted by the character. This occurs, for example, when readers look through the keyhole and through binoculars as if they were the girl spying on Teddy in What Does My Teddy Bear Do All Day?, or when readers draw shapes on the starry sky in The Heart and the Bottle. In several of these instances, the narrative is transformed temporarily into a device or platform that must be manipulated by the readers, as is the case when readers draw a picture in the place of the protagonist in What Does My Teddy Bear Do All Day? and The Heart and the Bottle. At the very explicit end of the spectrum, there are performances in which readers theatricalize and engage their bodies in reproducing the very same actions as the characters. Readers must shake the device in Don’t Let the Pigeon Run this App! and The Heart and the   164 Bottle, causing the pigeon to be shaken and the heart to be shaken, respectively; they blow the device in The Three Little Pigs and Little Red Riding Hood. All forms of performance are extremely ludic and evoke game playing in some sense. At the implicit end, these performances approximate playing a video game, while at the explicit end they simulate role-playing and make-believe games. Another aspect that wasn’t foreseen in the initial framework with regards to performance is that readers not only assume the role of other characters by controlling or copying their actions through their performances, but they may, in fact, play their own role. As interactive narratives published in the context of postmodernity, the boundaries between fiction and reality are often being explored in all the narratives analyzed. The reader participates through interaction in all these narratives, thus performance and metafiction feed each other with regards to playfulness, and performance can manifest metafiction as the reader is recognized as a participant in the narrative. As discussed by Hutcheon (1980), metafiction can be overt or covert, and this also applies to how metafiction is expressed through performance. Nonetheless, regarding performance, I believe these are not closed categories, but again a spectrum, which I will call the fictional-metafictional spectrum. I prefer to use a different term because a narrative can be overt in the way Hutcheon claims yet manifest a covert form of performance, as we will see below.  On the fictional end of the spectrum, readers assume through performance the role of a character and are not recognized by the narrative as participants–except by the fact that the narrative is interactive, which, as picturebook apps, they all are. This is the case of The Heart and the Bottle where the story follows a traditional narrative structure with a third person narrator telling a story to an audience which is never recognized as being there. Through multiple   165 interactions, readers implicitly and explicitly perform the role of the protagonist. What Does My Teddy Bear Do All Day? is also positioned close to this end of the spectrum. In almost every performance, readers play the role of another character. The exception would be when readers tickle the girl’s foot: the action is not being played by any other character, here; instead readers are participating momentarily as characters themselves. Similarly, but with a slightly higher level of acknowledgement of the reader by the narrative, there is The Three Little Pigs. Here, the verbal text addresses readers as participants in the narrative, but while performing readers mostly assume the role of the characters, not taking on a position as characters themselves or interfering in the narrative (an exception being when readers play peekaboo with the bunny and they are not controlling the animal but interacting with him).  Moving a little bit further to the center, Lil’ Red presents a narrative in which readers are recognized by the characters through their gazes, which ask for their participation in the narrative. In terms of performance, readers are recognized as participants in two instances: as they wake up the wolf, in an action that grants new meanings to the narrative, and as they throw apples at the wolf, which is fun and at the same time positions them as defending Little Red. Around the middle of the spectrum we have The Trip. This narrative is difficult to position because it involves many different forms of performance that parallel one another. Readers control the characters through implicit performances, which are positioned on the fictional end of this spectrum. Nonetheless, readers are directly addressed by Little Critter, and they perform to help the character collect objects. As discussed in the analysis of performance in this app, these performances seem merely a complicated way to control Little Critter; yet, the recognition of the reader cannot be denied, and thus this performance would move closer to the other end of the spectrum, the metafictional end.    166 Continuing towards the metafictional end we find Little Red Riding Hood. In this app, there is a lot of variation between performances in which readers control the characters–thus at the fictional end of the spectrum–and others in which they are recognized as participants and are asked to perform in helping Little Red Riding Hood–thus playing as characters in performances that approximate the metafictional end. Finally, readers participate with the narrative by choosing which path Little Red must follow and thus interfere in the conclusion of the story. In these performances, readers play at both ends of the fictional and metafictional spectrum simultaneously. At the fictional end because as they chose a path, they compel the characters to take this path. Even so, the most significant meaning in this action relies on readers performing out of the fictional sphere and as co-authors of the narrative, which positions this performance at the metafictional end. The various forms of performance in Little Red Riding Hood then present different fictional and metafictional levels. Finally, on the other extreme end, readers move out of the fictional domain into the metafictional. Readers stop playing the role of the characters to assume their position as readers; in the metafictional sphere, they also get to interfere in the narrative, playing as co-authors. At this end of the spectrum lie Don’t Let the Pigeon Run this App! and The Monster at the End of this Book. As diegetic and overt metafictional books, the narratives are conceptually founded on the performance played by the readers as readers. In Don’t Let the Pigeon Run this App!, readers are recognized as participants of the narrative in multiple ways. First, as the narrative establishes a dialogue between pigeon and reader, it cannot exist without the reader counterpart. Considering performance, when readers shake the device and the pigeon, they are acting as characters and responding to the pigeon’s demand. But, most significantly, readers participate outside the   167 fictional realm as they participate in the creation of the narrative: they incorporate their own inputs and voice and are identified as co-authors.  Finally, The Monster at the End of this Book occupies the extreme of the metafictional end because the essence of reader participation in this story is to perform as readers, without assuming any other role rather than reading and moving the story forward through the gesture of “turning the page.” This story is conceptually based on this performance, the act of reading. Besides turning the pages, in most of the other interactions, readers perform actions that have direct consequences; they are characters in the narrative who engage in a game and in a communication with Grover (e.g., when they tap the wall built to “separate” them from the page, they destroy it). This narrative is exemplary in what constitutes metafictional performances.  5.4.11 Differences between born-digital and transmediated apps  The differences in performance between born-digital and transmediated apps are more visible when analyzing the implicit-explicit spectrum. While implicit performances are simple and present in most apps, when we get to the explicit end of the implicit-explicit spectrum, the performances of transmediated apps involve less theatricalization than those of the born-digital apps. The extreme level of explicitness, which involves blowing, tilting, and shaking the device are only present in born-digital apps, with the exception of one instance in the The Heart and the Bottle. The explicit performance present in transmediated apps involves performance of a more cognitive level, such as drawing and playing games, although they obviously also involve bodily engagement. On the fictional-metafictional spectrum, it is harder to locate the presence of implicit or explicit performances according to whether or not an app was transmediated or born-digital. Almost all apps presented performances that fit on the fictional end of the spectrum, which in   168 fact substantially mirrors the implicit end of the performance scale. A few apps, both transmediated and born-digital, show performances that rest around the middle of the scale, in which readers are somewhat recognized as participants or play under limited circumstance roles that do not belong to the fictional characters. Similarly, the recognition of readers as characters does not seem to be related to the fact that an app was transmediated, but to its diegetic metafictional characteristics. On the other hand, the participation of readers as co-authors of the narrative, which happens in Little Red Riding Hood, Don’t Let the Pigeon Run This App! and The Trip, is primarily present in the born-digital apps. The possibility of positioning readers as co-authors with such a level of participation is directly related to the affordances of digital media. When born-digital narratives are generated, this resource is already a significant element of consideration, and a possibility for innovation. On the other hand, in print media, this constitutes a limitation that marks significantly the narratives that are created for this medium. In transmediated apps, the overcoming of this limitation and the integration of these possibilities result in significant changes in the story, so it seems natural that these apps don’t make use of this resource. In Don’t Let the Pigeon Run This App!, the participation of the reader as a co-author is essential in defining the metafictional nature of the narrative, and it is built into an electronic device that is exclusively of the digital mode, with the possibility of generating infinite narratives and also of including the reader’s own voice. Similarly, Little Red Riding Hood’s co-authoring is based on the hypertextual structure characteristic of digital texts. Finally, if in the transmediated The Trip readers have some level of control over the narrative, it is significantly limited, and it was made possible with the substantial changes the narrative underwent in its transmediation.     169 5.5 How this discussion addresses my research questions In this section, I summarize how the discussions presented in this chapters answer my two research questions. 1. How do the different modes–writing, images, speech, audio, movement and touch–contribute to the playful aspects of picturebook apps?  The analysis of these eight picturebook apps reveals that there is no unique, direct answer for this question; each picturebook constitutes a unique amalgamation of these modes with the objective of telling a story. As Lewis (2001) posits, there are infinite possibilities for combining images and texts in picturebooks, and even a single verbal text constitutes an unique narrative as it is interpreted by illustrators and, here, also by animators, musicians, sound designers, and programmers. The addition of modes transforms the picturebook narrative into an even more complex text, and the possibilities of its playfulness rely both on the nature of each of these modes alone but mostly on their capacity to interanimate and form modal ensembles. Irony is a central aspect of the creation of playfulness in terms of conceptual play and it can be created through the counterpoint between modes as well as through the counterpoint between signs, which usually combine various modes in a complementary relationship.  From a multimodal perspective, reading a text always involves its rewriting, and thus the active participation of the reader. Interactivity in digital media represents new possibilities of combining the different modes through computer programing, materializing the participation and control of the text by the reader through bodily engagement involved in the interactions. Therefore, interactivity can be considered another form interanimating the various modes with playful effects. Finally, this sample revealed that the reader’s input to the narratives through the   170 various possibilities of interactions also come in multimodal form: readers can incorporate verbal, visual, sonic and kinetic signs into the narratives. How does playfulness manifest in picturebook apps transmediated from print books and in born-digital picturebook apps? The differences between born-digital and transmediated apps were not as obvious as it was expected at the beginning of this study. In fact, all apps present at different levels the three types of playfulness proposed in the framework. Nevertheless, some tendencies can be drawn from this comparison. Considering the interanimation between modes, there seems to be a tendency for the transmediated apps to rely more on the counterpoint between modes as a source of playfulness, while the born-digital apps rely on relationships of complementarity among modes, which have less playful effects; these apps rely more on interaction as a source of playfulness. The interactions between characters and readers through the verbal and visual texts, which contribute significantly to the construction of metafictional play, are more present in the born-digital apps. This seems a result of the fact that these apps were conceptualized as interactive narratives, hence claimed that other modes also exhibited this relationship between narrative and reader, which is expressed in the verbal text through the presence of dialogues and in the visual text through the characters gaze towards readers. The participation of sound in the creation of meaning–and thus of playfulness–showed a tendency to be more significant in the born-digital apps, with extremely complex and sophisticated soundscapes and sound effects, while one of the transmediated apps–What Does My Teddy Bear Do All Day?–presents a poor use of sound which does not add much to its playful aspects. Yet, this cannot be generalized since sound is meaningfully used in creating playfulness in the other three transmediated apps. In The Heart   171 and the Bottle, for instance, the use of sound effects works in conveying deep and abstract concepts and emotions. Although all apps manifested metafiction as a source of playfulness, the transmediated apps presented a greater tendency to exhibit covert forms of metafiction, while all the born-digital apps showed overt metafiction, which suggests an agreement from the different texts to the fact that all these apps are interactive and so count on reader participation. In terms of performance, the central difference is that born-digital apps tend to include more theatrical forms of explicit performance, while the explicit performances of the transmediated apps do not include the same level of bodily engagement but explore forms of intellectual participation (such as drawing). In addition, metafictional performances through which readers participate as co-authors and have the possibility to interfere in the narrative’s development are only present in the born-digital apps.  5.6 Summary In this chapter, I examine how the apps analyzed in this thesis manifest playfulness through the interaction between modes, through metafiction and through performance. All the apps selected manifest the three forms of playfulness at different levels. In terms of the analysis of the interanimation between modes, some apps express playfulness through the contrast and counterpoint between modes, while others use the modes in unison to create irony on the narrative level. Regarding metafiction, few apps in this sample manifest diegetic metafiction, with their narratives clearly addressing the nature of fiction, reading, and the book format. All apps, nonetheless, manifest linguistic metafiction as their modes, in some cases verbal language, but also visual language and especially interaction, recognizing to a certain extent the reader as a   172 participant in the narrative, blurring the boundaries between fiction and reality. Finally, performance turns out to be a manifestation of playfulness with various degrees, and that can be organized along the axis between implicit and explicit, according to the level of proximity between the gesture of the performance and the real action it initiates, but also classified into fictional and metafictional performance, depending on the role the reader enacts while performing; as such, he may be assuming that of a character, or may be being recognized as a participant in the narrative, as a character himself, or as a co-author.    173 Chapter 6: Conclusion 6.1 Introduction In this final chapter, I outline the contributions of this work to picturebook scholarship, especially with regards to the study of picturebook apps, and how the close readings of the texts and the discussion of how they manifest playfulness may inform the design of high quality picturebook apps and may have implications for the adults who mediate children’s access to such texts. I also discuss the limitations of this study and, finally, make suggestions for future research in expanding the study of playfulness and picturebook apps, but also in understanding this emerging form of digital literature for children.   6.2 Research contributions  In this thesis, I have analyzed how playfulness manifests in eight picturebook apps through a framework that considered playfulness as exhibited through the interanimation among modes, through metafiction and through performance–at first divided into implicit and explicit performances. This framework has been derived from the studies of Mackey (2002) and Nikolajeva (2008), which analyse playfulness in children’s ecology of texts in general and in postmodern picturebooks in particular. The close reading of these picturebook apps was informed by multimodal social semiotics, especially the works of Kress and Van Leeuwen, and its application to the analysis of picturebooks as proposed by Lewis (2001) in his ecology of the picturebook. Through social semiotics this work has discussed how the different modes of meaning-making participate in the construction of playfulness in picturebook apps.  The picturebooks analyzed were What Does My Teddy Bear Do All Day? by Bruno Hächler and Birte Müller, Lil’ Red by Bart Blocmen and Brian Main, The Monster at the End of   174 This Book by Jon Stone and Michael Smollin, Don’t Let the Pigeon Run This App! by Mo Willems and Readers, The Heart and the Bottle by Oliver Jeffers, The Three Little Pigs by Nosy Crow, The Trip - Little Critter Reading Adventure by Mercer Mayer, and Little Red Riding Hood by Nosy Crow. They were analyzed in the context of postmodernity in which playfulness, metafiction and the breaking of boundaries through reader participation and interaction are considered characteristics of postmodern picturebooks as described by Sipe and Pantaleo (2008). Although presenting some characteristics of postmodernism, Lewis (2001) posits that most children’s picturebooks do not present the disruption typical of postmodern art. This proved true in the sample analyzed. Although all books manifested traits associated with postmodern literary production, they remained attached to the values and conventions of traditional literature; even the narratives that allowed readers to perform as co-authors had a defined and traditional narrative structure.  Due to the novelty of picturebook apps in the realm of children’s literature, scholars are trying to define and propose terminologies and categories to understand what picturebook apps are, how they work and how they differ (or not) from print picturebooks. Al-Yaqout (2013) has proposed an aesthetical analysis of picturebook apps in trying to define this new format of digital literature for children, without, however, analyzing the multimodal literary text itself. Stichnothe (2014) proposes a narratological approach in which picturebook apps can be classified as ‘multiple fabula apps’–where readers can participate in deciding some aspects of the plot–and as ‘alternative story apps’–where readers can participate only in deciding how the story is told. This thesis contributes to picturebook scholarship by proposing multimodal theory as a framework to the textual analysis of picturebook apps, taking into account its multidisciplinary characteristics. The use of the vocabulary proposed by Kress (2010), Van Leeuwen (1999, 2005, 2006), and   175 Kress and Van Leeuwen (1996, 2002) to the description of how the different signs in different modes convey meaning proved to be useful in describing how the texts in picturebook apps work. This work expands the use of multimodality proposed by Lewis in the ecology of the picturebook for accommodating the contribution of sound, movement and interactivity to the meaning-making processes of digital picturebooks.  The analysis of these picturebook apps reveals that the categories proposed by Nikolajeva (2008) as to how playfulness manifests in print picturebooks are also valid for the analysis of picturebook apps. All the apps manifest to some extent all three types of playfulness: through the interanimation between modes, through metafiction and through performance. This suggests a difference in relation to print picturebooks since the physical engagement with the print picturebook through performance is a source of playfulness only in certain specific kinds of picturebooks, such as pop-up books and some metafictional narratives.  This research also contributes to the consideration of playfulness as an important concept in the discussion of picturebook apps, without limiting it to its interactive features. Playfulness is a central aspect of any picturebook, manifesting both in their literary features and in their material features. Yet, playfulness takes a new dimension in picturebook apps with the inclusion of sound, movement and interactivity and blurs the boundaries between these literary forms to other forms of play, such as video games. The analysis of playfulness is thus fundamental to the understanding of how picturebook apps work.  6.3 Contributions to the design of picturebook apps Creating picturebook apps requires a deep understanding of the meaning-making possibilities and limitations of a certain media and formats. Due to the novelty of the subject,   176 most creators’ work is guided by intuition, experimentation, and trial and error, and the results vary significantly from one app to the other. In this scenario, the mediators of children’s literature–parents, teachers and librarians–face difficulties in selecting and evaluating these texts. The sample for this thesis constitutes some of the best picturebook apps published so far, and the analysis and description of the multimodal texts of these apps make explicit how they convey meaning and create a high quality digital literary texts. I would like, therefore, to propose a few recommendations to designers based on the close readings of the apps in this work. If in traditional picturebooks both text and images are considered inseparable elements that constitute the narrative, in picturebook apps, similarly, all the modes are important and inseparable and should be treated with equal care and built with consideration as to their potentials and limitations. The multiplicity of modes, though, offers an even deeper level of complexity since each mode can establish a relationship with one of the other modes but a different one with a third mode, and so on. For instance, image and text can counterpoint each other, but sound will usually have a complementary relationship with image; animations, since they are built upon the visual mode, cannot contradict it. Through interactivity, readers are able to assume various roles, including their own role as readers or their active role as co-creators of the narrative, which generates infinite possibilities for innovation and the creation of new forms of digital literature. Each mode provides readers with a certain point of view of the narrative, thus it is important while creating to take into account which point of view one wants readers to assume through that mode in order to create a coherent and cohesive narrative. Since the creation of picturebook apps is extremely multidisciplinary, it is important to have a team that can integrate their skills and knowledge.   177 Picturebook app creators must also take into account that every element matters and conveys meaning, so must be taken seriously. If a certain movement, effect or interaction is added to the narrative, it must converge with the story and add meanings that contribute to some aspect of the narrative. For instance, even character’s eye movements, as they address the reader or address a character or do not address anyone, will result in different kinds of relationships between the reader and the narrative and between the characters within the narrative. The lack of consciousness of some creators to the meanings a certain sign may convey results in narratives whose texts are weak, contradictory and do not explore all the potentials each modes possesses, while strong literary texts make use of every artefact to transmit the narrative coherently and cohesively. Creators of picturebook apps should also take the level of control over the narrative given to readers seriously because it will have a huge impact on defining what kind of app the narrative is and how children will in fact read it. Page turning and the gap it creates between one scene and the next is a central form of control characteristic of books. The narrower the gap is between scenes–as, for instance, in The Trip, in which animated scenes connect one scene to the next–the closer the app will approximate an animated movie rather than a book. If the control given to readers is such that they can handle the movements of the characters, make choices that interfere in the plot, and/or choose among different paths which generate unique narratives, then the narrative will probably bear significant resemblance to a video game. Although this work does not want to suggest that one format is better than the other, they have different impacts on the kind of reading performed by children; therefore creators must make a conscious choice on this matter in order to select the format that most explores the narrative they are trying to convey.   178 In terms of playfulness, creators need to realize that playfulness can be conveyed in various ways beyond interactivity. Interactivity is certainly an important approach into creating playfulness in picturebook apps, but other subtler forms of playfulness may be used to generate a more complex and sophisticated narrative. The possibility of counterpoint between modes as a form of playfulness is amplified in picturebook apps, and new uses of these relationships are yet to be explored, the effects of which will certainly dazzle children and adults alike. This form of playfulness requires from readers a different form of participation that relies on their intellectual ability and leaves gaps to be filled in by their imagination, participation that is different from the one achieved through interactivity.  Metafiction is a form of playfulness that combines both the cognitive and the performatic aspects of playfulness. Only a small portion of print picturebooks possesses metafictive devices, but, in app format, they seem to be more prevalent. The comprehension of these metafictive devices and how they influence reader participation can result in apps whose multimodality works more coherently in enhancing this playful aspect.  As shown in this section, the analysis of picturebook apps as presented in this study reveals important details that must be taken into account if one wishes to design quality picturebook apps.  6.4 Implications for the mediators of children’s literature The emergence of picturebook apps as a new form of children’s literature poses new challenges to parents, teachers and librarians as mediators of children’s literature. The lack of reliable review sources and the difficulties of navigating through the app stores make it extremely hard for these mediators to choose the best picturebook apps for their children. At the   179 same time, when evaluating for themselves the quality of an app, they face new hybrid forms of reading that questions their concepts of what traditionally constitutes a good book. Although this study did not aim to compare print books and digital books, it is clear that playfulness is enhanced in picturebook apps since the number of instances in which performance and metafiction occur in these texts is visibly higher than in print picturebooks. Also, the added complexity of the multiple modes that compose the digital texts and their ecology make possible new and more frequent levels of playfulness regarding the process of decoding the multimodal text. In regards to playfulness, it is important that mediators have in mind that this is an aspect of both print and digital picturebooks. Some traditional views of education and literature may consider that play and reading are separate activities, and if children are playing they cannot be “seriously” reading, but playfulness is an important aspect of literature itself and, therefore, cannot be considered detrimental to reading. Play is also an important element in promoting children’s engagement with texts and transmitting the joy of reading, and texts that are specially ludic, such as picturebook apps, may be particularly useful for attracting reluctant readers. Playfulness, nonetheless, should not be taken simply for fun and amusement, although they are usually connected. As the close reading of The Heart and the Bottle revealed, playfulness can be used to convey extremely deep meanings and even sadness. In pedagogical terms, The New London Group (2000) posits, “literacy pedagogy must now account for the burgeoning variety of text forms associated with information and multimedia technologies” (p. 9). In this context, picturebook apps have an enormous pedagogical potential concerning this broader sense of literacy, or multiliteracies, which goes beyond the skill of decoding the written text, and encompasses the capacity to read, to interpret and to create verbal,   180 visual, auditory, cinematic and multimodal texts. The picturebook apps analysed presented complex and rich multimodal texts with potential yet to be explored in their details and their wholeness in the pedagogy of multiliteracies. It represents a new form of expression of the ‘Available Designs’ (New London Group, 2000, p. 20) for children and permits new forms of ‘Designing’ and ‘The Redesigned’ (p. 22-23) which exhibit the characteristics of the postmodern era in which multimedia communication is ubiquitous in the lives of children and adults. The pedagogy of multiliteracies as proposed by the London Group also addresses the importance of plurality, diversity and inclusion, which also concerns the variety and diversity of types of texts and languages to which children are exposed. Picturebook apps are an emerging format of text in the ecology of children’s texts, and they should not supersede the reading of traditional picturebooks or other print texts but share the reading time at home and at school with all the various forms of texts available for children. The multimodal nature of picturebook apps also works in terms of inclusion when it motivates children to read whose preferred mode is not verbal due to the visual, sonic, kinetic and interactive characteristics of the apps, while simultaneously promoting these other forms of literacy to children who are foremost verbal.  In conclusion, picturebook apps must be understood as an opportunity for new forms of reading and learning that are in accordance with the communicational landscape children are immersed in today and will continuously be in the future. These digital texts for children have the potential for multiliteracy development, which is fundamental to their growth and critical understanding of the world.    181 6.5 Limitations The first limitation of this study was its reduced sample of only eight picturebook apps. As a consequence, the results presented here cannot be extended to the majority of picturebook apps. Nonetheless, I believe the insights that emerged from the close analysis of these books can inform the analysis of any picturebook apps. In addition, the analysis of these apps revealed how differently and uniquely each narrative is constructed with the variety of devices that promote playfulness within each mode and through the interanimation between modes, an understanding that is important to the design of picturebook apps. Another limitation was that the criterion for the selection of the apps in this study resulted in a sample whose apps present similar characteristics. By selecting apps that visibly manifested at least one of the forms of playfulness from the framework, we ended up selecting apps that are extremely playful, which may not reflect the reality of the thousands of apps available. This selection may have interfered in the comparison between the born-digital and the transmediated apps resulting in limited differences between the two groups. In the case of the transmediated apps, the fact that I selected apps that explicitly manifested playfulness automatically excluded a great number of transmediations that remain attached to their print counterparts and therefore do not articulate the additional modes, neither to transmit the narrative nor to generate playfulness, a situation that was commonly seen in the 100-app sample of The Future of Children’s Texts project, from which this work derives. Another aspect that can be considered a limitation of this work lies in the fact that most of the picturebook app production to date consists of transmediated apps. It was a challenge to select high quality, born-digital picturebook apps that fit into these categories. It resulted in two of the selected apps belonging to the same creators and publishers, thus possessing many features   182 in common. In addition, although none of the transmediated apps lack an antecedent print version with the same text and illustrations, they maintain narrow connections with print picturebook tradition: three of them are versions of fairy tales and Don’t Let the Pigeon Run This App! belongs to a print picturebook series from which it imported many features of its narrative structure and content.  6.6 Recommendations for future research One of the central conclusions of this study is that due to the novelty of this literary form there is much to be studied for a more comprehensive understanding of how picturebook apps work and what the implications of the addition of modes are to how they construct meaning and playfulness. First, as a master’s thesis, this study did not allow for the analysis of a bigger sample, so the first recommendation would be to reproduce this study using a bigger sample where the differences between born-digital and transmediated apps could be traced more clearly. As there are numerous studies that compare book and film versions of a narrative, another aspect that was only briefly mentioned in the analysis of these picturebooks–but one that could constitute an entire new study–is how the transmediation from print into app changes the meanings of a picturebook, which could result in the establishment of categories that map the different levels of transmediation taking place in the transition of books from print into digital media. In The Trip, for example, the digital version offered a significant extension of the contents in comparison to the print version, a transformation that did not happen in the other books analyzed. In addition, this study analysed the manifestation of playfulness from a theoretical point of view, considering the implicit reader and analyzing how the text of these picturebooks suggests   183 playfulness. The next step in this research would be to contrast the theoretical assumptions formulated here with the observation of responses from children in order to understand how they interpret these signs of playfulness and what in fact turns out to be playful for them. This study could reveal how children with different cultural backgrounds and from different age groups differently interpret the suggested forms of playfulness. This study has also analyzed superficially how each of the modes creates meaning. As described in the literature review, each of the modes can be analyzed in extensive detail according to the metalanguage proposed by Kress and Van Leeuwen. For example, each of the metafunctions can be analyzed across modes in a picturebook, or concepts like modality–as discussed by Nikolajeva and Scott (2001) with regards to print picturebooks–can be analyzed regarding picturebook apps.  Finally, a theme that became visible through various parts of this work is the emergence of new hybrid forms of narrative, which merge characteristics of books, games, and animated movies. Some of the elements that traditionally characterize a book are already disappearing in some apps, such as the need to turn the pages for the progression of the narrative or even the inclusion of the text in print form on the screen, since the verbal texts are also available as speech, and these narratives are mostly aimed at children who are yet incapable of decoding the print text. As a result, many picturebook apps show a clear resemblance to animated movies. In addition, the interactive features of these apps transform them into experiences that resemble a video game. Nishizawa and Schwartz (2012) coined the term “b-a-g.” to define the hybrid form “book-application-game,” which would indicate that, instead of trying to establish the differences between the digital books and other forms of narrative, it is necessary to assume the emergence of new formats and concepts.    184 6.7 Summary In this chapter, I position this study in terms of its contributions to picturebook theory and the study of digital children’s literature in general and of picturebook apps in particular, suggesting the use of social semiotics multimodal theory and the ecology of children’s texts as frameworks for the analysis of these texts. Based on the close reading of the books, I have also made suggestions for the design of high quality picturebook apps, which include the importance of understanding the affordances of each mode constituting the multimodal text of these literary works in order to build coherent and cohesive texts. This chapter also includes a discussion of the implications of this study for the mediators of children’s literature, in which I have discussed the importance of playfulness as an essential element of children’s picturebooks and the potential of picturebook apps in the pedagogy of multiliteracies. The small sample of the study, the difficulty of finding high quality transmediated apps and the fact that the apps analyzed may not be representative of the panorama of apps available for children were the main limitations encountered by this study. Finally, this chapter presents recommendations for future research which include the comparison between print books and their transmediated counterparts, the possibility of the empirical analysis of playfulness in picturebook apps in the observation of children’s responses to these texts, the deeper analysis of how each mode creates meaning in picturebook apps, and the study of the emergent hybrid narrative forms observed in the sample of this study.    185 References Primary texts Blocmen, B., & Main, B. (2012). Lil’ Red. (Version 1.03). Developed by Brian Main. [Mobile application software]. Retrieved from http://itunes.apple.com. Hächter, B., & Müller, B. (2004). What does my teddy bear do all day? Bargteheide, Germany: Minedition. Hächter, B., & Müller, B. (2010). What does my teddy bear do all day? (Version 2.4.0). Developed by Auryn Inc. [Mobile application software]. Retrieved from http://itunes.apple.com. Jeffers, O. (2010a). The heart and the bottle. New York, NY: Philomel Books.  Jeffers, O. (2010b). The heart and the bottle. (Version 1.0). Developed by HarperCollins. [Mobile application software]. Retrieved from http://itunes.apple.com. Mayer, M. (1988). Little Critter’s the trip. New York, NY: Inchworm Press.  Mayer, M. (2013). The trip - Little Critter reading adventure. (Version 1.0.1). Developed by Silver Dolphin. [Mobile application software]. Retrieved from http://itunes.apple.com. Nosy Crow. (2011). The three little pigs. (Version 1.4.3). Developed by Nosy Crow. [Mobile application software]. Retrieved from http://itunes.apple.com. Nosy Crow. (2013). Little red riding hood. (Version 1.0.4). Developed by Nosy Crow. [Mobile application software]. Retrieved from http://itunes.apple.com. Stone, J., & Smollin, M. (1971). The monster at the end of this book. New York, NY: Sesame Street. Stone, J., & Smollin, M. (2011). The monster at the end of this book. (Version 3.3). Sesame Street.   186 Willems, M. (2012). Don’t let the pigeon run this app! (Version 1.1). Developed by Disney. [Mobile application software]. Retrieved from http://itunes.apple.com.  Secondary texts Agosto, D. E. (1999). One and inseparable: Interdependent storytelling in picture storybooks. Children's Literature in Education, 30, 267-280.  Al-Yaqout, G. (2011). From slate to slate: What does the future hold for the picturebook series? New Review of Children's Literature and Librarianship, 17, 57-77.   Anstey, M., & Bull, G. (2004) The Picture Book: Modern and Postmodern. In Peter Hunt (Ed.), International companion encyclopedia of children’s literature (pp. 328-339). New York, NY: Routledge. Armstrong, C. (2008). Books in a virtual world: The evolution of the e-book and its lexicon. Journal of Librarianship and Information Science, 40, 193-206. Arrant, C. (November 11, 2011). How Jon Klassen Leapt from Animation to Children’s Books. Cartoon Brew. Retrieved from http://goo.gl/OrPBJr. Bader, B. (1976). American picturebooks from Noah's ark to the beast within. New York, NY: Macmillan. Bakhtin, M. M. (1981). The dialogic imagination: Four essays. Austin: University of Texas Press. Barthes, R. (1985). The responsibility of forms: Critical essays on music, art and representation. New York, NY: Hill and Wang-Farrar. Bircher, K. (2012). What makes a good picture book app? The Horn Book Magazine, 88(2), 72-78.   187 Bird, E. (2011). Planet app: Kids' book apps are everywhere. But are they any good? School Library Journal, 57, 26-31. Bizzocchi, J., & Tanembaum, J.  (2011). Well read: Applying close reading techniques to gameplay experiences. Retrieved November 20, 2012 from http://www.etc.cmu.edu/etcpress/content/well-read-jim-bizzocchi-joshua-tanenbaum Cahill, M., & McGill‐Franzen, A. (2013). Selecting “app”ealing and “app”ropriate book apps for beginning readers. The Reading Teacher, 67, 30-39.  Chiong, C., Ree, J., Takeuchi, L. & Erickson, I. (2012) Print books vs. e-books: Comparing parent-child co-reading on print, basic, and enhanced e-book platforms. New York, NY: The Joan Ganz Cooney Center at Sesame Workshop. Retrieved November 20, 2012 from http://www.joanganzcooneycenter.org/Reports-35.html Coles, M., & Hall, C. (2001). Breaking the line: New literacies, postmodernism and the teaching of printed texts. Reading: Literacy and Language, 35(3), 111-114. de Jong, M., & Bus, A. (2003). How well suited are electronic books to supporting literacy? Journal of Early Childhood Literacy, 3(2), 147-164. Downing, A., & Locke, P. (2006). English Grammar. A University Course. New York, NY: Routledge. Dresang, E. (1999). Radical change: Books for youth in a digital age. New York, NY: H.W. Wilson Co. Dresang, E. (2008). Radical change theory, postmodernism and contemporary picturebooks. In Sipe, L. R., & Pantaleo, S.  (Eds.), Postmodern picturebooks: Play, parody, and self-referentiality (pp. 41-54). Hoboken: Routledge.   188 Ghonem-Woets, K., Joosen, V., Dinh, R.P., Van Lierop, H., Wenz, K., & Wolters, R. (2013). Conference Booklet. 21st Biennial Conference of the International Research Society for Children’s Literature: Children’s Literature and Media Cultures. Maastricht, Nederlands: Maastricht University. Golden, J. M. (1990). The narrative symbol in childhood literature: Explorations in the construction of text. New York, NY: Mouton de Gruyter. Goldstone, B. (1998). Ordering the Chaos: Teaching Metafictive Characteristics of Children’s Books. Journal of Children’s Literature, 24(2), 48-55. Goldstone, B. (2001/2002).  Whaz Up With Our Books? Changing Picture Book Codes and Teaching Implications. The Reading Teacher, 55(4), 362-370. Grieve, A. (1993). Postmodernism in Picturebooks. Papers: Explorations into Children’s Literature, 4(3), 15-25. Guglielmo, C. (October 31, 2013). Apple, hoping for an iPad Christmas, may get its wish as reviewers tout iPad Air. Forbes. Retrieved from http://www.forbes.com/sites/connieguglielmo/2013/10/31/apple-hoping-for-an-ipad-christmas-may-get-its-wish-after-reviewers-tout-ipad-air/  Hallberg, K. (1982). Litteraturvetenskapen och bilderbokforskningen. Tidskrift for litteraturvetenskap, 3(4), 163-168. Halliday, M. A. K. (1978). Language as social semiotics. London, UK: Edward Arnold. Hempstead, A. (2008). Effects of interactive books on phonological awareness skills in pre-kindergarten children from low-income environments. Retrieved November 2, 2013 from Florida State University ETD Collection. etd-07142008-104440   189 Huizinga, J. (1949). Homo ludens: A study of the play-element in culture. London, UK: Routledge & K. Paul. Hutcheon, L. (1980). Narcissistic narrative: The metafictional paradox. New York, NY: Methuen. Hutcheon, L. (1988). A poetics of postmodernism: History, theory, fiction. New York, NY: Routledge. Ishizuka, K. (2011). The app squad: making sense of book apps for kids-with a little help from our friends. School Library Journal, 57(5), 38-43. Jewitt, C. (2009). The routledge handbook of multimodal analysis. New York, NY: Routledge. Kanatsouli, M. (2012). Games inside books for young children. Bookbird: A Journal of International Children's Literature, 50(4), 33-40. Kiefer, B. Z. (1995). The potential of picturebooks: From visual literacy to aesthetic understanding. Englewood Cliffs, N.J: Merrill. Kim, D., & Hall, J. K. (2002). The role of an interactive book reading program in the development of second language pragmatic competence. The Modern Language Journal, 86, 332-348. Korat, O. (2009). The effects of CD-ROM storybook reading on Israeli children’s early literacy as a function of age group and repeated reading. Education and Information Technologies, 14, 39-53. Korat, O. (2010). Reading electronic books as a support for vocabulary, story comprehension and word reading in kindergarten and first grade. Computers & Education, 55, 24-31.   190 Korat, O., & Shamir, A. (2007). Electronic books versus adult readers: Effects on children's emergent literacy as a function of social class. Journal of Computer Assisted Learning, 23, 248-259. Korat, O., & Shamir, A. (2008). The educational electronic book as a tool for supporting children's emergent literacy in low versus middle SES groups. Computers & Education, 50, 110-124. Kress, G. (1997). Before writing: Rethinking the paths to literacy. London, UK: Routledge. Kress, G. (2009). What is mode? In C. Jewitt (Ed.), The routledge handbook of multimodal analysis (pp. 54-67). New York, NY: Routledge. Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London, UK: Routledge. Kress, G., & Van Leeuwen, T. (1996, 2006). Reading images: The grammar of visual design. London, UK: Routledge. Kress, G., & Van Leeuwen, T. (2002). Colour as a semiotic mode: Notes for a grammar of colour. Visual Communication, 1, 343-368. doi:10.1177/147035720200100306 Leverkus, C. (2011). Book app mania. Library Media Connection, 30(2), 50-51. Lewis, D. (2001). Reading contemporary picturebooks: Picturing text. New York, NY: Routledge. Mackey, M. (2002). Literacies across media: Playing the text. New York, NY: Routledge. McCallum, R.(1996). Metafictions and experimental work. In Peter Hunt (Ed.), International Companion encyclopedia of children’s literature (pp. 397-409). New York, NY: Routledge.   191 Mol, S. E., Bus, A. G., & de Jong, M. (2009). Interactive book reading in early education: A tool to stimulate print knowledge as well as oral language. Review of Educational Research, 79(2), 979-1007. New London Group (2000). A pedagogy of multiliteracies: Designing social futures. In B. Cope, & M. Kalantzis (Eds.), Multiliteracies: Learning and the design of social futures (pp. 9-37). London, UK: Routledge. Nichols, B. (1981). Ideology and the image: Social representation in the cinema and other media. Bloomington: Indiana University Press. Nikolajeva, M. (1988). Exit children’s literature? The Lion and the Unicorn, 22(2), 221-236. Nikolajeva, M. (2008). Play and playfulness in postmodern picturebooks. In Sipe, L. R., & Pantaleo, S.  (Eds.), Postmodern picturebooks: Play, parody, and self-referentiality (pp. 75-88). New York, NY: Routledge. Nikolajeva, M., & Scott, C. (2000). The dynamics of picturebook communication. Children's Literature in Education, 31(4), 225-239.  Nikolajeva, M., & Scott, C. (2001). How picturebooks work. New York, NY: Garland Pub. Nishizawa, A. J., & Schwartz, G. (2012, October). O livro aberto da educomunicação: além da wikipedia, rumo ao socionário. Paper presented at the Conference IV Encontro Brasileiro da Educomunicação, São Paulo, SP. Nodelman, P. (1988). Words about pictures: The narrative art of children's picture books. Athens: University of Georgia Press. Nodelman, P. (2003). How, but not what or why. Children's Literature, 31, 192-200. doi:10.1353/chl.2003.0012   192 Nugent, C. (2013, June). Extending picturebook multimodal theory to encompass the affordances of the iPad. Paper presented at the ACQL Annual Conference: Texts and Technologies in Canadian and Quebec Literatures, Congress of the Humanities and Social Sciences, Victoria, BC.  Ornstein, R. E. (1975). The psychology of consciousness. New York, NY: Penguin. Oxford advanced learner’s dictionary. (n.d.). Retrieved December 5, 2013 from http://oald8.oxfordlearnersdictionaries.com/ Parish‐Morris, J., Mahajan, N., Hirsh‐Pasek, K., Golinkoff, R. M., & Collins, M. F. (2013). Once upon a time: Parent–child dialogue and storybook reading in the electronic era. Mind, Brain, and Education, 7, 200-211. Penelas, C. T. (2013). Álbumes ilustrados en pantallas. Cuadernos de Literatura Infantil y Juvenil, 256, 6-12. Perrault, C. (1697, 2014). Little red riding hood. In Little red riding hood: The ultimate collection: Every single version (n.p.). Enhanced e-books. [E-book]. Retrieved May 20, 2014 from https://play.google.com/store/books/details?id=7b8eAwAAQBAJ. Poole, S. (2000). Trigger happy: The inner life of videogames. London, UK: Fourth Estate. Ritterfeld, U., Shen, C., Wang, H., Nocera, L., & Wong, W. L. (2009). Multimodality and interactivity: Connecting properties of serious games with educational outcomes. Cyberpsychology & Behavior: The Impact of the Internet, Multimedia and Virtual Reality on Behavior and Society, 12, 691-697. Robb, M. B. (2010). New ways of reading: The impact of an interactive book on young children's story comprehension and parent-child dialogic reading behaviors. UC   193 Riverside: Psychology. Retrieved February 16, 2013 from: http://escholarship.org/uc/item/5xm8n8xk Rose, G. (2001). Visual methodologies: An introduction to the interpretation of visual materials. Thousand Oaks, Calif: Sage Publications. Rosenblatt, L. M. (1978). The reader, the text, the poem: The transactional theory of the literary work. Carbondale: Southern Illinois University Press. Salisbury, M., & Styles, M. (2012). Children's picturebooks: The art of visual storytelling. London, UK: Laurence King Pub. Schons, L. M. (2011). Is the picture book dead? The rise of the iPad as a turning point in children’s literature. Journal of Digital Research & Publishing, 2, 120-128. Schwarcz, J. H. (1982). Ways of the illustrator: Visual communication in children's literature. Chicago: American Library Association. Sipe, L. R. (1998). How picture books work: A semiotically framed theory of text-picture relationships. Children's Literature in Education, 29(2), 97-108.  Sipe, L. R., & Pantaleo, S. J. (2008). Postmodern picturebooks: Play, parody, and self-referentiality. New York, NY: Routledge. Smeets, D. J. H., & Bus, A. G. (2012). Interactive electronic storybooks for kindergartners to promote vocabulary growth. Journal of Experimental Child Psychology, 112, 36-55. Stichnothe, H. (2014). Engineering stories? A narratological approach to children’s book apps. Nordic Journal of Childlit Aesthetics, 5. Retrieved January 21, 2014 from: http://dx.doi.org/10.3402/blft.v5.23602 Suhor, C. (1984). Towards a semiotics-based curriculum. Journal of Curriculum Studies, 16, 247-257.   194 Sutton-Smith, B. (1997). The ambiguity of play. Cambridge, MA: Harvard University Press. Underwood, J. D. M., & Underwood, G. (1998). Children’s interactions and learning outcomes with interactive talking books. Computers & Education, 30, 95-102. Van der Linden, S. (2011). Para ler o livro ilustrado. São Paulo: Cosac Naify. Van Leeuwen, T. (1999). Speech, music, sound. New York, NY: St. Martin’s Press.  Van Leeuwen, T. (2005). Introducing social semiotics. London, UK: Routledge. Van Leeuwen, T. (2006). Towards a semiotics of typography. Information Design Journal, 14(2), 139-155. Van Leeuwen, T. (2007). Sound and vision. Visual Communication, 6(2), 136-145. Vancouver Public Library. (n.d. a). About the library. Retrieved November 30, 2013 from https://www.vpl.ca/about/cat/C996/ Vancouver Public Library. (n.d. b). Tumblebook admin. Retrieved November 30, 2013 from http://www.tumblebooks.com/library/asp/TeacherInfo.aspx?vendorid=146562 Vygotsky, L.S. (1978). Mind in society. Cambridge, MA: Harvard University Press. Wasik, B. A., & Bond, M. A. (2001). Beyond the pages of a book: Interactive book reading and language development in preschool classrooms. Journal of Educational Psychology, 93(2), 243-250. Watson, K. (2004). The postmodern picture book in the secondary classroom. English Australia, 140, 55-57. Yokota, J., & Teale, W.H. (2014, in press). Picture books and the digital world: Educators making informed choices. The Reading Teacher, 34(6). Retrieved November 30, 2013 from http://www.readinghalloffame.org/sites/default/files/ebooks__apps-yokota-teale-final.pdf   195 Zago, G., Lambello, D., Merlo, G., Goga, N., & Campagnaro, M. (2013). Book of abstracts: The child and the book. Ninth annual conference: Children’s literature, technology and imagination. Research, Problems and Perspectives. Padova, Italy: Universita degli Studi di Padova. Zucker, T., Moody, A., & McKenna, M. (2009). The effects of electronic books on pre-kindergarten-to-grade 5 students’ literacy and language outcomes: A research synthesis. Journal of Educational Computing Research, 40(1), 47-87.   Children’s literature cited Ahlberg, J., & Ahlberg, A. (1986). The jolly postman, or other people's letters. Boston, MA: Little Brown. Jeffers, O. (2009). The incredible book eating boy: Pop-up book. London, UK: HarperCollins. Klassen, J. (2011). I want my hat back. Somerville, MA: Candlewick Press. Scieszka, J., & Smith, L. (1992). The stinky cheese man and other fairly stupid tales. New York, NY: Viking. Stone, J., & Smollin, M. (2011). Another monster at the end of this book. (Version 3). Developed by Sesame Street. [Mobile application software]. Retrieved from http://itunes.apple.com. Willems, M. (2003). Don't let the pigeon drive the bus! New York, NY: Hyperion Books for Children.  Online resources consulted for picturebook app selections Appysmarts. (n.d.). Retrieved November 30, 2013 from http://www.appysmarts.com/ Best Apps for Kids. (n.d.). Retrieved November 30, 2013 from http://bestappsforkids.com/   196 Children’s Technology Review. (n.d.). Retrieved November 30, 2013 from http://childrenstech.com/ CM Magazine. (n.d.). Retrieved November 30, 2013 from http://www.umanitoba.ca/cm/ Common Sense Media. (n.d.). Retrieved November 30, 2013 from http://www.commonsensemedia.org/ Digital Media Diet. (n.d.). Retrieved November 30, 2013 from http://digitalmediadiet.com/ Digital Story Time. (n.d.). Retrieved November 30, 2013 from http://digital-storytime.com/ Kindertown. (n.d.). Retrieved November 30, 2013 from http://www.kindertown.com/ Kirkus Review. (n.d.). Retrieved November 30, 2013 from https://www.kirkusreviews.com/ Mac News World. (n.d.). Retrieved November 30, 2013 from http://www.macnewsworld.com/ Parents’ Choice Awards. (n.d.). Retrieved November 30, 2013 from http://www.parents-choice.org/ School Library Journal. (n.d.). Retrieved November 30, 2013 from http://www.slj.com/ Smart Apps for Kids. (n.d.). Retrieved November 30, 2013 from http://www.smartappsforkids.com/ Teachers with Apps. (n.d.). Retrieved November 30, 2013 from http://teacherswithapps.com/ The Horn Book. (n.d.). Retrieved November 30, 2013 from http://www.hbook.com/ Wired. (n.d.). Retrieved November 30, 2013 from http://www.wired.com/     197 Appendix: Screen references Since some of the apps analyzed in this study do not show a linear narrative with clear page distinctions, I will identify below the page sequence I have used and mentioned in the close-readings: The Trip Screen 1: Packing the car Screen 2: Stuck on a traffic jam Screen 3: Facing a horse on the middle of the road through the country Screen 4: Problem with the engine and stop for a snack Screen 5: Playing games on the road Screen 6: Ask for information Screen 7: Stop at the beach Screen 8: Stop for ice cream Screen 9: Stop because of a flat tires Screen 10: Sleeping at the cabin  Little Red Riding Hood Copyright for all the following images: Nosy Crow, artwork by Ed Bryan   198 Figure A.1. Screen 0.   Figure A.2. Screen 1.   Figure A.3. Screen 2.  Figure A.4. Screen 3.    199   Figure A.5. Screen 4.   Figure A.6. Screen 5.    200 Figure A.7. Screen 6a.  Figure A.8. Screen 6b.   Figure A.9. Screen 7.   Figure A.10. Screen 8a.  Figure A.11. Screen 8b.  Figure A.12. Screen 8c.  Figure A.13. Screen 8d.   Figure A.14. Screen 9    201 Figure A.15. Screen 10a.  Figure A.16. Screen 10b.  Figure A.17. Screen 10c.   Figure A.18. Screen 11.   Figure A.19. Screen 12.   Figure A.20. Screen 13.    202 Figure A.21. Screen 14.   Figure A.22. Screen 15a.  Figure A.23. Screen 15b.    Figure A.24. Screen 16a. Figure A.25. Screen 16b. Figure A.26. Screen 16c. Figure A.27. Screen 16d.      Figure A.28. Screen 17a. Figure A.29. Screen 17b. Figure A.30. Screen 17c.       203 Figure A.31. Screen 18a.  Figure Appendix.32. Screen 18b.  Figure Appendix.33. Screen 18c.   Figure A.34. Screen 19.   Figure A.35. Screen 20.        

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0167471/manifest

Comment

Related Items