Education, Faculty of
Educational and Counselling Psychology, and Special Education (ECPS), Department of
DSpace
UBCV
Rahal, Louai
Doctor of Philosophy - PhD
University of British Columbia
Visual supports have been found to facilitate communication with and teaching of students with Autism Spectrum Disorder (ASD) (Hodgdon, 2016; Mirenda & Fossett, 2015). However, the creation of visual supports is often a time-consuming task, and ensuring both their safety and portability can be challenging as well. The use of mobile tablets has been found to significantly reduce the time requirements for creating visual supports and facilitating their portability (Giles & Markham, 2017; Hirano et al., 2010). However, the use of these devices has introduced new challenges related to technology integration in teaching. Limited technical support and mismatch between educators’ pedagogical beliefs and the technologies that are available to them have made the process of integrating tablets harder for educators. One suggested solution to the problems of technology integration is to involve educators in the design of the technology they use and to produce with them technologies that closely match their needs and pedagogical values. The objective of this study was to codesign web applications with educators that they could use to create and display visual supports on tablets. The researcher collaborated with three groups of educators and codesigned a web application with each group. A qualitative analysis of the design process helped to elicit the factors that facilitated and/or hindered the process of designing technology with educators. Factors that facilitated participation included perceived usefulness of the designed products and participants’ knowledge about design and technology. Some of the factors that hindered participation included turnover and career changes (especially among educational assistants) and differences between stakeholders (teachers, speech language pathologists, educational assistants, and students). Based on the findings of the study, the researcher articulated recommendations on engaging in Participatory Design (PD) in special education, such as aligning the goals of PD projects with the long-term goals of participants.
Participatory Design in the Special Education Classroom: Designing Technology with Educators for Students with Autism Spectrum Disorder by Louai Rahal B.Sc., The American University of Beirut, 2007 M.A., The University of British Columbia, 2011 M.Sc., The University of Hertfordshire, 2017 LL.M., The University of Law, 2020 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Human Development, Learning, and Culture) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) December, 2021 © Louai Rahal, 2021 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, the dissertation entitled: Participatory Design in the Special Education Classroom: Designing Technology with Educators for Students with Autism Spectrum Disorder submitted by Louai Rahal in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Human Development, Learning, and Culture Examining Committee: Dr. Jennifer Shapka, ECPS department, Faculty of Education, UBC Supervisor Dr. Pat Mirenda, ECPS department, Faculty of Education, UBC Supervisory Committee Member Dr. Ido Roll, Centre for Teaching, Learning and Technology, UBC Supervisory Committee Member Dr. Rob VanWynsberghe, Faculty of Education, UBC University Examiner Dr. Joanna Erin Cannon, Faculty of Education, UBC University Examiner iii Abstract Visual supports have been found to facilitate communication with and teaching of students with Autism Spectrum Disorder (ASD) (Hodgdon, 2016; Mirenda & Fossett, 2015). However, the creation of visual supports is often a time-consuming task, and ensuring both their safety and portability can be challenging as well. The use of mobile tablets has been found to significantly reduce the time requirements for creating visual supports and facilitating their portability (Giles & Markham, 2017; Hirano et al., 2010). However, the use of these devices has introduced new challenges related to technology integration in teaching. Limited technical support and mismatch between educators’ pedagogical beliefs and the technologies that are available to them have made the process of integrating tablets harder for educators. One suggested solution to the problems of technology integration is to involve educators in the design of the technology they use and to produce with them technologies that closely match their needs and pedagogical values. The objective of this study was to codesign web applications with educators that they could use to create and display visual supports on tablets. The researcher collaborated with three groups of educators and codesigned a web application with each group. A qualitative analysis of the design process helped to elicit the factors that facilitated and/or hindered the process of designing technology with educators. Factors that facilitated participation included perceived usefulness of the designed products and participants’ knowledge about design and technology. Some of the factors that hindered participation included turnover and career changes (especially among educational assistants) and differences between stakeholders (teachers, speech language pathologists, educational assistants, and students). Based on the findings of the study, the researcher articulated recommendations on engaging in Participatory Design (PD) in special education, such as aligning the goals of PD projects with the long-term goals of participants. iv Lay Summary Educators around the world are using visual supports in their work with students with Autism Spectrum Disorder (ASD) and they regularly utilize technology to make the process of creating visual supports more efficient. The objective of this study was to design, with educators, web applications for creating and displaying visual supports and to understand the challenges and facilitators that affect educators’ engagement in design. An analysis of data from this study uncovered several factors that impacted the process of designing with educators. Factors that facilitated the design process included educators’ perceived usefulness of the designed applications and the feedback loops facilitated by the paper and electronic prototypes. The challenges included a mismatch between the designed applications and the local infrastructure at the research sites, and a mismatch between the designed applications and participants’ established habits around technology use. v Preface This dissertation is original, unpublished, independent work by the author. The research study reported in this dissertation was designed and executed by the author with guidance from committee members. Ethics approval for this research was obtained from the Behavioural Research Ethics Board at the University of British Columbia (certificate number: H15-03081). vi Table of Contents Abstract ......................................................................................................................................... iii Lay Summary ............................................................................................................................... iv Preface ............................................................................................................................................ v Table of Contents ......................................................................................................................... vi List of Tables ............................................................................................................................... xii List of Figures ............................................................................................................................. xiii Acknowledgments ...................................................................................................................... xiv Chapter 1: The Use of Visual Supports with Students with Autism Spectrum Disorder ...... 1 1.1 The Use of Visual Supports with Students with ASD ....................................................... 3 1.1.1 Visual Symbols ........................................................................................................... 4 1.1.2 Within-Activity Visual Schedules .............................................................................. 5 1.1.3 Between-Activity Visual Schedules ........................................................................... 5 1.1.4 Contingency Maps ...................................................................................................... 7 1.2 The Use of Mobile Tablet Devices to Create and Display Visual Supports ...................... 8 Chapter 2: Literature Review .................................................................................................... 11 2.1 Benefits and Challenges of the Effective Use of Visual Supports in Teaching Students with ASD .................................................................................................................................. 11 2.1.1 Increasing the Effectiveness of the Use of Visual Supports in the Classroom ......... 11 2.1.2 Institutional Factors .................................................................................................. 13 2.2 Barriers and Facilitators of Effective Technology Integration in Teaching .................... 15 2.2.2 First and Second-Order Barriers ............................................................................... 16 2.2.3 Technology Acceptance Models ............................................................................... 18 vii 2.3 Designing with Educators ................................................................................................ 19 2.3.1 Technical Action Research ....................................................................................... 21 2.3.2 Practical Action Research ......................................................................................... 22 2.3.3 Emancipatory Action Research................................................................................. 24 2.4 Codesigning Tablet-Based Applications with Educators ................................................ 26 2.5 Gaps in the Literature, Research Question ...................................................................... 29 Chapter 3: Methodology and Methods ..................................................................................... 31 3.1 Applying PD with Educators as Design Partners and Informants ................................... 31 3.1.1 Recruitment ............................................................................................................... 34 3.1.2 Participants ................................................................................................................ 34 3.1.3 Procedures: Overview ............................................................................................... 35 3.1.4 Procedure Description: Brenda ................................................................................. 36 3.1.4.1 First Meeting: Semi-structured Interview (April 2016) ..................................... 37 3.1.4.2 Second Meeting: Creating a Paper Prototype (April 2016) ............................... 37 3.1.4.3 Third Meeting: Feedback on the Electronic Prototype (May 2016) .................. 38 3.1.4.4 Fourth Meeting: Delivering the Final Product (September 2016) ..................... 38 3.1.4.5 Teaching the Educational Assistants to Use the Application (May 2017) ........ 38 3.1.4.6 First Day of Using the Application (June 2017) ................................................ 38 3.1.4.7 Observation (June 2017) .................................................................................... 40 3.1.4.8 Final Visit (March 2018) ................................................................................... 41 3.1.5 Procedure Description: Shannon and Ferreira .............................................................. 41 3.1.5.1 First Meeting: Semi-structured Interview and Paper Prototype (November 2016)........................................................................................................................................... 42 3.1.5.2 Second Meeting: Introducing the Electronic Prototype (February 2017) .......... 42 3.1.5.3 Third Meeting: Feedback on the Electronic Prototype (June 2017) .................. 42 3.1.5.4 Fourth Meeting: Delivering the Final Product (June 2017) ............................... 43 3.1.5.5 Observation (June 2017) .................................................................................... 43 3.1.5.5 Meeting after the Observation (February 2018) ................................................ 44 viii 3.1.6 Procedure Description: Caleb and Educational Assistants ....................................... 44 3.1.6.1 First Meeting: Semi-structured Interview and Paper Prototype (October 2016) 44 3.1.6.2 Second Meeting: Electronic Prototype (January 2017) ..................................... 44 3.1.6.3 Third Meeting: Feedback on the Electronic Prototype (November 2017) ........ 45 3.1.6.4 Fourth Meeting: Delivering the Revised Product and Prototype (December 2017) ................................................................................................................................. 45 3.1.6.5 Fifth Meeting: Delivering the Final Products (April 2018) ............................... 45 3.2 Data Analysis ................................................................................................................... 48 3.2.1 Description, Analysis, and Interpretation ................................................................. 49 3.2.1.1 Description ......................................................................................................... 49 3.2.1.2 Analysis.............................................................................................................. 50 3.2.1.3 Interpretation ...................................................................................................... 51 3.3 Generalizability and Reliability ....................................................................................... 53 3.3.1 Supporting Generalizability Through Thick Descriptions ........................................ 53 3.3.2 Assessing Reliability ................................................................................................. 53 Chapter 4: Study Findings ......................................................................................................... 55 4.1 Chronological Descriptions ............................................................................................. 55 4.1.1 Description of the Collaboration with Brenda .......................................................... 56 4.1.1.1 Articulating Goals .............................................................................................. 56 4.1.1.2 Design Process ................................................................................................... 57 4.1.1.3 Introducing the Product...................................................................................... 60 4.1.1.4 Summary ............................................................................................................ 62 4.1.2 Description of the Collaboration with Shannon and Ferreira ................................... 62 4.1.2.1 Articulating Goals .............................................................................................. 62 4.1.2.2 Design Process ................................................................................................... 64 4.1.2.3 Introducing the Application ............................................................................... 65 4.1.2.4 Summary ............................................................................................................ 66 4.1.3 Description of the Collaboration with Caleb and his Educational Assistants .......... 67 ix 4.1.3.1 Articulating Goals .............................................................................................. 67 4.1.3.2 Design Process ................................................................................................... 68 4.1.3.3 Introducing the Applications.............................................................................. 70 4.1.3.4 Summary ............................................................................................................ 71 4.2 Critical Events .................................................................................................................. 71 4.2.1 Critical Events from the Collaboration with Brenda ................................................ 72 4.2.1.1 Visual Supports Supported Both the Students and the Educational Assistants . 72 4.2.1.2 Emailing the Researcher When the Uploading of Pictures did not Work ......... 73 4.2.1.3 Taking Pictures of Paper Visual Symbols and Uploading them ........................ 74 4.2.2 Critical Events from the Collaboration with Shannon and Ferreira ......................... 76 4.2.2.1 Tensions between Control and Usability ........................................................... 76 4.2.2.2 Creating a Quick Booklet................................................................................... 77 4.2.2.3 Alex was More Interested in Another Application ............................................ 78 4.2.3 Critical Events from the Collaboration with Caleb and Educational Assistants ...... 79 4.2.3.1 Different Reactions from Different Educational Assistants .............................. 79 4.2.3.2 From Low Perceived Competence to Adopting the Technology and Making Suggestions ....................................................................................................................... 81 4.2.3.3 Kathryn Tried Creating a Schedule, But the Web Application Failed .............. 82 4.3 Common Themes Across Cases....................................................................................... 84 4.3.1 Mismatch between Planned Outcomes and Actual Outcomes ................................. 85 4.3.2 Mismatch Between the Technology Created and the Technology Used by Participants ............................................................................................................................ 86 4.3.3 The Importance of Time Constraints for Educators ................................................. 89 4.3.3.1 Brenda ................................................................................................................ 89 4.3.3.2 Shannon and Ferreira ......................................................................................... 89 4.3.3.3 Caleb and Educational Assistants ...................................................................... 90 4.3.4 Turnover among Educational assistants in the Special Education Classroom ......... 91 4.3.5 It is Not Always about Technology .......................................................................... 92 x 4.3.6 Hesitation with the Paper Prototype Activity .......................................................... 95 4.4 Eliciting Feedback ........................................................................................................... 97 Chapter 5: Interpretation, Implications, and Conclusions ..................................................... 99 5.1 Research Implications for Conducting PD in the Special Education Classroom ............ 99 5.1.1 Planning for and Introducing the Project ................................................................... 101 5.1.1.1 Develop a plan for continuously assessing the internal reliability of the project 102 5.1.1.2 Identify and Manage Participant Expectations of the Project ................................. 104 5.1.2 Field Work ............................................................................................................... 105 5.1.2.1 Observe the visual support systems and technologies used by participants ....... 106 5.1.2.2 Examine the extent of staff turnover in the context (transferring between classrooms, changing careers) ........................................................................................ 108 5.1.3 Designing with Participants ..................................................................................... 109 5.1.3.1 Support participants to translate their ideas and knowledge into a design prototype......................................................................................................................................... 110 5.1.3.2 Give participants choice over their role in the design process and the required time commitment ............................................................................................................ 112 5.1.3.3 Create a Design Process that Positively Contributes to Participants’ Perceptions of Technology and their Self-Efficacy for Technology ...................................................... 112 5.1.4 Introducing the New System .................................................................................... 113 5.1.4.1 Design a supportive context in which the new system is introduced ................. 114 5.1.4.2 Design teaching material and activities for stakeholders .................................... 114 5.2 My Story with PD: Autoethnographic Reflections and Lessons Learned ..................... 115 5.2.1 Initial Objectives: Codesigning a Cultural Tool to Create Pedagogical Opportunities............................................................................................................................................. 116 5.2.2 Reality Check .......................................................................................................... 117 5.2.3 Changed Focus, Some Uncritical Optimism ........................................................... 118 5.3 Conducting the Study: Unresolved Tensions, Turning to the Literature ....................... 119 xi 5.3.1 Tension 1: Solving Practical Problems or Creating Pedagogical Opportunities .... 119 5.3.2 Tension 2: Giving Choices to Participants or Making Choices for Them .............. 120 5.3.3 Tension 3: Timing Introduction of the Application ................................................ 122 5.4 Perceived Failure and Renewed Commitment ............................................................... 123 5.5 Assessing Internal Reliability: Alignment between Knowledge, Values, Stakeholders, and Outcomes.......................................................................................................................... 125 5.6 Summary and Conclusions ............................................................................................ 128 References .................................................................................................................................. 132 xii List of Tables Table 5.1 Recommendations for conducting PD …………………………………………… 100 xiii List of Figures Figure 3.1 First visual support created by Brenda and her assistants ........................................... 39 Figure 3.2 Appearance of the visual support after the picture is clicked ...................................... 40 Figure 3.3 Second visual support created by Brenda and her assistants ....................................... 41 Figure 3.43 Multiple choice question created by Shannon and Ferreira ...................................... 43 Figure 3.5 Screenshot from the matching game used by Harry with his student ......................... 48 Figure 4.1 Prototype sketched by Brenda ..................................................................................... 58 Figure 4.2 Paper Prototype: Shannon and Ferreira ....................................................................... 63 Figure 4.3 Paper Prototype Sketched with Caleb ......................................................................... 68 Figure 4.4 Uploaded Pictures ........................................................................................................ 75 Figure 4.5 Question from the booklet created by Shannon ........................................................... 77 xiv Acknowledgments I want to thank my supervisor, Dr. Jenna Shapka, for her consistent support, generous feedback, and the knowledge she shared with me. I am dedicated to following Dr. Shapka’s example and I will do my best to empower my future students in the same way that Dr. Shapka empowered me. I want to thank Dr. Pat Mirenda for motivating me and challenging me to continuously improve my work and refine my ideas, Dr. Mirenda’s commitment to quality and rigor is an inspiration. I want to thank Dr. Ido Roll for his knowledge, his generosity, and for introducing me to Participatory Design, a methodology that helped me bridge my passion for knowledge and my passion for social change, thank you for inspiring me to think and rethink. Two years ago, I had the privilege of meeting Dr. Beth Haverkamp; since the day we met, things kept getting better in my studies. I want to thank Dr. Haverkamp for the difference she made on my academic path. I want to thank my mentor Dr. Shawna Faber for teaching me about teaching, and for motivating me to make teachable moments happen in the classroom, no matter the format, no matter the constraints. To Faith and Maryam, my friends and my mentors, thank you for your support in difficult times, and for giving me happy memories. To my research participants, thank you for sharing your knowledge and for the dialogues we had, so many unforgettable lessons I learned from you. To my wife Po-Hsueh and to our families, no words can convey my gratitude to you, I will continue to express my gratitude with my deeds. It is in community that individuals grow and evolve, I belong to many communities, in each one of them I have friends, mentors, students, and colleagues, they help me grow and persist, their voices make my thoughts, they make me who I am. In the sea of their insights, I disappear. In the sea of their insights, I become. 1 Chapter 1: The Use of Visual Supports with Students with Autism Spectrum Disorder Many students with autism spectrum disorder (ASD) struggle with communication due to difficulties in processing verbal and auditory stimuli (Quill, 1997). To work around these difficulties, educators have used visual symbols as augmentative and alternative communicative tools. For example, picture symbols have been used to instruct students with ASD to transition from one activity to another (Dettmer et al., 2000), to communicate to them a sequence of expected activities (Knight et al., 2015), and to inform them about the consequences of various behaviors (Brown & Mirenda, 2006). Students with ASD also use visual symbols to make requests and choices by pointing to, touching, and/or clicking on a visual symbol (Miao-En Chien et al., 2015; Vaughn & Horner, 1995). There are several types of pedagogical tools that can be created with visual symbols; collectively, these tools are often referred to as visual supports. The use of visual supports in teaching students with ASD is an evidence-based practice; numerous researchers have carried out intervention studies testing and confirming the efficiency of this practice (Fage et al., 2016; Knight et al., 2015). Traditionally, visual supports are created on paper and then laminated and displayed on cards or in three-ring binders, photo albums, or portable mounted boards. However, the use of paper to create visual supports creates several logistical challenges for educators, including the time commitment that is required to print and laminate papers and difficulties with storage, access, and portability (Donato et al., 2014; Hayes et al., 2010). The greater the number of visual supports, the more difficult it becomes to find and retrieve a specific visual support; the heavier the visual supports binders get, the more difficult it becomes to move them around. 2 To overcome the logistical challenges imposed by paper-based supports, educators and researchers have turned to mobile tablets. With mobile tablets, the process of creating visual supports may require less time and the portability of visual supports vastly improves (Giles & Markham, 2017; Hirano et al., 2010). Using tablets’ built-in cameras, educators can instantly take pictures or video clips that they can then retrieve by navigating the tablets’ albums. There are also several commercially available apps/software to create visual supports with pre-recorded pictures and videos. Yet, while mobile tablets could potentially reduce many of the logistical challenges associated with paper-based visual supports, the adoption of this new medium may also create new challenges for educators. For example, it might be challenging to learn how to use the tablets or software for creating visual supports. In addition, the relatively high cost of tablets and the difficulty of replacing them when they get damaged might be a barrier (Hirano et al., 2010). Despite the challenges created by mobile technologies, the benefits and flexibility associated with tablet-based visual supports make them a promising avenue to explore for supporting students with ASD. Hence, in this qualitative study, the researcher set out to contribute to the academic literature by working with educators to codesign and implement tablet-based applications for creating visual supports. Educators participated in the design process with the hope that their participation would enhance and facilitate the introduction and integration of mobile devices into the process of teaching with visual supports. In this dissertation, the researcher presents findings from a study of three Participatory Design (PD) projects conducted with three groups of educators in the Lower Mainland of Vancouver, British Columbia. The educators who participated in the research were teachers, educational assistants, and speech-language pathologists. The researcher analyzed and 3 interpreted the data for this dissertation focusing specifically on challenges encountered during the design and integration of the designed products, the knowledge that educators shared in the process of designing the products, and the changes and resistance to change that occurred throughout the process. The dissertation builds on three different yet interrelated academic subfields: (a) the research on visual supports, their pedagogical benefits, and the challenges of using them; (b) the literature on technology integration in teaching; and (c) the literature on the benefits and challenges of designing technology with educators. In this chapter, the researcher introduces visual supports and discusses examples of different types of visual supports along with their pedagogical benefits and the benefits of using tablet technology in creating them. The rationale for involving educators in the design of technologies is also discussed briefly. 1.1 The Use of Visual Supports with Students with ASD As noted previously, visual supports can have a beneficial impact on the communicative, social, and psychological development of students with ASD. There are several types of visual supports that have been created, each of which serves a different pedagogical function. Four types of visual supports are introduced here: visual symbols, within-activity schedules, between-activity schedules, and contingency maps. These kinds of visual supports are most relevant and similar to the visual supports designed in the current study. Note that other visual supports that are less relevant to the study, such as visual social stories, are outside the scope of this study and therefore will not be discussed here. The description of each type of visual support presented in this section includes an explanation of its function and examples of anecdotal and empirical evidence that demonstrated the effectiveness of the support. 4 1.1.1 Visual Symbols The simplest form of visual support consists of a single visual symbol printed on a card or displayed on a screen; the symbol can be a picture, photograph, line drawing, or any other visual representation. Educators have used visual symbols for a long time to communicate instructions, expectations, and reminders to students with ASD (Kidder & McDonnell, 2017; Mirenda & Fossett, 2015). Students with ASD have also used visual symbols to make requests and make choices by pointing to a picture attached to a board (Vaughn & Horner, 1995) and by clicking on a picture displayed on a mobile tablet device (Miao-En Chien et al., 2015). Prizant (2015) presented an anecdotal account of visual symbols’ effectiveness by using a visual symbol to support a student with ASD to manage his anxiety. A student in one of the classrooms that Prizant observed in the study was continuously asking whether his mother was coming to pick him up from school. Despite the teacher’s many replies to the student that his mother was coming to pick him up, the student kept asking, and when the teacher ignored the questions, the student’s questions became “louder and more urgent” (Prizant, 2015, p. 25). Prizant suggested that the teacher print a photo of the student’s mother and place it on his desk to placate his need for his mother’s presence. The intervention was quite effective, as indicated by a significant reduction in the frequency of the student’s questions about his mother after the photo was placed on his desk. Vaughn and Horner (1995) conducted an empirical study on the effectiveness of visual symbols. They used visual symbols with a young man with ASD, Karl, who struggled with food intake, to support him to make choices between food items. They took photographs of different food items and attached them to a board on one wall in Karl’s home, and when they asked Karl to choose the food item that he wanted, they asked him to point to the picture of the item. To 5 empirically assess their intervention, Vaughn and Horner (1995) compared Karl’s acceptance rate of food under two conditions: (1) when choice between food items was presented to him verbally and (2) when choice between food items was presented to him both verbally and visually. Karl’s acceptance rate of the food offered to him improved significantly when food choices were presented to him both verbally and visually as opposed to verbal presentation alone. As the above examples demonstrate, the use of even simple visual stimuli may improve communication between students with ASD and their caregivers (e.g., parents, teachers). Visual stimuli have also been used to create pedagogical supports that consist of multiple images and serve more advanced pedagogical functions. 1.1.2 Within-Activity Visual Schedules A within-activity visual schedule consists of a series of visual symbols wherein each visual symbol represents a step in a task (Mirenda & Fossett, 2015). Educators and parents have used within-activity schedules to scaffold the process of skill acquisition and skill development by breaking down skills into smaller steps and displaying the steps in order. Some of the skills that have been taught successfully with within-activity schedules include daily living skills such as preparing meals, folding clothes, and getting dressed (Pierce & Schreibman, 1994; Van Laarhoven et al., 2010), and leisure skills such as playing hide-and-seek (Brodhead et al., 2014) and playing a video game (Blum-Dimaya et al., 2010). 1.1.3 Between-Activity Visual Schedules Between-activity schedules have a similar layout to within-activity schedules in that they consist of a sequence of visual symbols. However, while visual symbols in a within-activity schedule represent different steps in the same task, visual symbols in a between-activity schedule represent different activities in the order that students are expected to complete them. Between-6 activity schedules have been used to add predictability to the routines of students with ASD by showing them in advance of a sequence of activities. Greater predictability has been shown to result in pedagogical gains such us increasing independence and reducing challenging behaviors during transitions between activities (Cihak, 2010). Evidence to support the effectiveness of between-activity schedules is extensive and includes both empirical studies and systematic literature reviews. For instance, Massey and Wheeler (2000) conducted a study that exemplified the impact of between-activity schedules. The researchers created a between-activity schedule by using Polaroid photos attached to a board and used it with a 4-year-old child with ASD attending an inclusive preschool, to give him a preview of the sequence of upcoming activities. The student had a history of challenging behaviour that included crying and stereotypic behaviour during both planned and unplanned transitions. After mastering the use of the visual schedules, the student had a significantly lower rate of challenging behaviour during transitions and a significantly higher rate of task engagement. Hall et al. (1995) also used between-activity schedules with elementary school students with ASD and other disabilities to help them transition between activities. They taught the students to refer to the schedule, complete an activity, refer to the schedule again, then complete a subsequent activity. The intervention increased students’ task-engagement, reduced their need for prompts from educational assistants, and increased the rate of independent transitions. For example, Larry, a 7-year-old boy with autism, previously struggled to transition from a class meeting activity to an individual writing activity. After receiving instruction on using his visual activity schedule, Larry was able to transition independently and his task engagement in the writing activity significantly improved (Hall et al., 1995). Importantly, the intervention also included training for educational assistants on using visual schedules and on the 7 importance of reducing verbal prompts when working with students with ASD. As demonstrated by this study, and as argued by Mirenda and Fossett (2015), interventions that include additional elements to enhance the impact of visual supports are more likely to be successful. 1.1.4 Contingency Maps Contingency maps are another way to use visual stimuli. These maps allow students to visually preview the consequences of their behaviors (Brown & Mirenda, 2006; Kidder & McDonnell, 2017). Contingency maps consist of visual cues that can help to deal with problem behaviors by highlighting positive replacement behaviors. This is typically done by including visual representations of the following five elements: (1) The antecedent that precedes both the problem and the replacement behavior; (2) the problem behavior itself; (3) the consequence of the problem behavior (usually a negative consequence); (4) an alternative replacement behavior; and (5) the consequence of the replacement behavior (usually a positive consequence) (Brown & Mirenda, 2006). Brown and Mirenda (2006) demonstrated the effectiveness of contingency maps in reducing problem behavior. The study included one participant, a 13-year-old student who demonstrated a lack of self-initiating behaviors. Brown and Mirenda (2006) used contingency maps to support the student in initiating the following behavioral chain: standing up after finishing a task, walking towards the educational assistant, and telling the assistant that he finished the task. The contingency map included an image of the reward the student would get if he successfully initiated the interaction. The use of contingency maps led to a decrease in the student’s passive behavior (i.e, sitting down after finishing a task) and an increase of the target behavior (i.e., self-initiating an interaction). 8 1.2 The Use of Mobile Tablet Devices to Create and Display Visual Supports As demonstrated in the previous section, visual supports are used to enhance communication with students with ASD and create pedagogical opportunities for them. To take advantage of the power of visual supports, educators need to develop a system that allows them to create, retrieve, and display them. While educators have traditionally used paper-based systems to create and/or display the supports, educators and researchers have also experimented with different technologies for these two purposes (Davies et al., 2002; Mechling et al., 2009; Stromer et al., 2006). The most recent of these technologies and the one that has been researched the most over the past decade are tablets (Carlile et al., 2013; Giles & Markham, 2017; Hirano et al., 2010). Research has reported several advantages of using tablet-based visual supports over the use of paper-based supports. First, the use of tablets may reduce the time required to create the supports and the time required to complete instructional classroom activities (Hirano et al., 2010). Second, tablet-based supports are more portable (Giles & Markham, 2017). Third, by making the process of creating visual supports easier for educators, tablets may also facilitate a positive pedagogical impact on the education of students (Hayes et al., 2010). When the process of creating visual supports gets easier, educators are more likely to create more of them, which facilitates the development of students’ vocabulary. As students develop and change, their needs change, their levels of literacy change and, correspondingly, their visual supports should be updated to match their developmental levels. The use of paper-based supports, because of the heavy time commitment that it imposes on educators, might lead to the creation of fewer supports and, as a result, fewer opportunities for students with ASD to develop their vocabulary (Hayes et al., 2010). 9 While tablet technology has had a positive impact on the process of creating and using visual supports, it has also led to challenges similar to those that educators have faced and continue to face with other technologies. First, not all schools have sufficient funding to provide access to mobile devices to all students, and not all schools have efficient systems and sufficient resources to replace damaged devices (Hirano et al., 2010). Second, educators usually need to use commercially available software to create visual supports, and, in many cases, they must learn how to use the software without specific training. Third, not all educators have the technical knowledge and technological self-efficacy that enable them to operate mobile devices effectively. In fact, decades of research in several different countries has found that educators tend to have a low self-efficacy for technology use (Ertmer, 1999; Shapka & Ferrari, 2003; Wang et al., 2004). To overcome problems of technology use in the classroom, many researchers have involved educators in the design of technology with the hope that educators’ involvement would facilitate effective future use (Matuk et al., 2016). Among the studies that involved educators in the design of technology, a subset of studies involved educators in the codesign of systems for creating, safeguarding, and displaying visual supports for students with ASD (Fage et al., 2016; Fage et al., 2019; Hayes et al., 2010; Hirano et al., 2010; Miao-En Chien et al., 2015). The objective of the current study was to expand on current research on designing tablet-based visual supports with educators and, in particular, to increase understanding of the factors that facilitate and hinder the process of designing technology with educators who use visual supports. In the current study, the researcher collaborated with three groups of educators and designed with each group an application for creating and displaying visual supports. The researcher then conducted a retrospective analysis of the design process with the objective of contributing to the knowledge 10 base on the challenges that design researchers may encounter when designing with special education educators and on the benefits of involving special education educators in the design of instructional technology. 11 Chapter 2: Literature Review Codesigning tablet-based systems for creating and displaying visual supports is a complex process that is shaped and influenced by several factors. This chapter includes a review of empirical and theoretical studies from three academic areas, each focusing on a set of factors that shape the codesign process. This literature review comprises the following: (a) the literature on factors that enhance or hinder the pedagogical effectiveness of visual supports; (b) research on factors that facilitate and/or hinder effective adoption of technology in the classroom; and (c) studies on factors that facilitate and/or hinder design partnerships between educators and researchers. 2.1 Benefits and Challenges of the Effective Use of Visual Supports in Teaching Students with ASD Introducing a visual support into the teaching process does not always lead to positive outcomes; decisions about which visual support to introduce and how and when to introduce it are all important factors that affect pedagogical impact. To create an effective visual support system and to accurately evaluate the effectiveness of existing systems, it is important to consider the factors that have been found to increase the pedagogical effectiveness of visual supports. 2.1.1 Increasing the Effectiveness of the Use of Visual Supports in the Classroom Researchers have identified several classroom practices that educators could follow to improve the pedagogical effectiveness of the visual supports they use with their students. First, the choice of which visual support to use should depend on what the educator is trying to accomplish. For example, if the objective is to teach the student a new routine, a within-activity schedule is likely to be helpful; however, if the objective is to enable a student to transition 12 independently from one activity to the next, a between-activity schedule is more appropriate. It is recommended that an educator conduct an assessment of a student’s pattern of behavior and needs prior to deciding which type of visual support to introduce (Mirenda & Fossett, 2015). Unfortunately, because such assessments are not always carried out, the use of visual supports tends to be arbitrary in some classrooms and can result in reduced pedagogical effectiveness (Mirenda & Fossett, 2015). Second, teaching educators and students on how to use visual supports is necessary. Visual supports by themselves, without systematic instruction of students and educators, are not likely to have a strong pedagogical impact. For example, Morrison et al. (2002) introduced between-activity schedules to students with ASD in a preschool setting to increase their on-task behavior across play routines. When the schedules were introduced to the students in the baseline condition without any training, students’ on-task behavior did not improve. However, when students and their educational assistants received guided instruction on how to use the visual schedules, their on-task behavior and their independent transition between play activities improved significantly. Third, the type of instruction that is used influences its effectiveness. For example, Hume et al. (2009) argued that, if students with ASD regularly receive verbal prompts to use visual supports, they might develop prompt dependence and thus be unable to attend to the supports independently. As an alternative, McClannahan and Krantz (2010) used a technique called graduated guidance to teach visual schedule use. This technique requires an adult to physically guide a student to touch a picture on the visual schedule and then guide the student to complete the activity represented by the picture; over time, physical guidance is reduced gradually in both intensity and frequency until the student can carry out the activity independently. Empirical work 13 by other researchers has also confirmed the effectiveness of the method (Hall et al., 1995; MacDuff et al., 1993). Fourth, the rapport between the student and the educator delivering the instruction affects the outcome. When students are instructed by educators with whom they have a history of reinforcement, they are more likely to be responsive to the instructions (Jull & Mirenda, 2016). Thus, training educators on how to build rapport with students can enhance the possibility of the student’s positive response to the training. Finally, students’ sensory and motivational preferences (Arthur-Kelly et al., 2009) should be considered in visual support design. For example, McClannahan and Krantz (2010) recommended including a preferred activity at the end of a visual activity schedule to motivate the child with ASD to complete the schedule. 2.1.2 Institutional Factors Institutional factors such as funding, opportunities for professional development, and peer support among educators, all affect the experiences of educators using visual supports (Donato et al., 2014). To understand the impact of institutional factors, Donato et al. (2014) asked educators about their experiences with visual supports, the barriers that prevented them from using visual supports effectively, and the factors that motivated and facilitated their use of visual supports. Some of the institutional barriers identified by educators included a lack of time to create the visual supports and lack of support from school administrators. Educators argued that having no personnel at schools dedicated to the creation of visual supports and other resources increases the time demands imposed on them. An institutional factor identified as a facilitator for using visual supports was awareness of visual supports’ positive pedagogical impact (Donato et al., 2014). In the education of students with ASD, where multiple educators must collaborate (e.g., a teacher, educational 14 assistant, speech language pathologist, and parent), consensus is important, and when some of the members in an educational team are not aware of the potential pedagogical impact of visual supports, they may end up resisting them. The more educators are aware of the pedagogical benefits of visual supports, the more likely they are to accept their adoption (Donato et al., 2014). Providing educators with professional development opportunities on visual supports is likely to contribute to an increased awareness of their pedagogical benefits and, as a result, increased adoption. The factors summarized above informed the researcher’s analysis of the visual support systems that participants used in their classrooms and the systems that the participants codesigned with the researcher. The factors themselves were not manipulated during the study; for example, the researcher did not make suggestions to educators about using graduated guidance or reducing prompts. Such suggestions would have interfered with the educators’ teaching practices, and, as Penuel et al. (2007) noted, teachers find it “unsettling” when researchers are critical of their practices: Through the course of the co-design process, teachers’ concerns often focus on usability … much more than on teaching considerations. By contrast, researchers tend to take a more analytical stance and push teachers to justify goals. Teachers can find such talk unsettling, because it violates norms of interaction with colleagues that emphasize informal sharing and avoidance of peer critique. (p. 56) Another reason why the researcher did not interfere with teaching practices was that he did not have the experience nor the training necessary to ask educators to make changes. 15 2.2 Barriers and Facilitators of Effective Technology Integration in Teaching As argued by both researchers (Hayes et al., 2010) and educators (Donato et al., 2014), tablet technology has the potential to increase the effectiveness of visual supports. Yet, the integration of tablets or any other technology can be a difficult and sometimes overwhelming task. Despite these challenges, educators continue to integrate technology into their classrooms and researchers continue to examine how to support educators in improving teaching through technology (Coleman et al., 2016; Makki et al., 2018). Understanding the barriers that have historically functioned to hinder effective technology integration among educators is important when designing technology with educators and when assessing educators’ experience with technology. An understanding of these barriers can support researchers in determining why a system is failing educators and/or how to create a tech-based system that would be easy to integrate for educators. Barriers to the inclusion of technology in teaching have been a research topic since the earliest technologies were introduced into the classroom (Cassidy, 1998; Ertmer, 1999; Makki et al., 2018; Silva & Breuleux, 1994). Insufficient technical support for teachers and insufficient availability of equipment (Cassidy, 1998), along with low technology self-efficacy (Wang et al., 2004), have historically hindered the effective use of radios and televisions, computers and the internet, and mobile tablets in teaching (Cassidy, 1998; Kim et al., 2019). Even in the 21st century, barriers to effective technology integration still exist and researchers continue to both suggest and assess ways of overcoming these barriers (Coleman et al., 2016; Makki et al., 2018). Historical research conducted by Cassidy (1998) revealed that, while technology has evolved rapidly in the 20th century, the barriers to integrating technology in classroom teaching have remained constant. Technical difficulties have been the main obstacle for educators for 16 decades. For example, when they tried to utilize radios and televisions in the 1940s and 1950s, weak signals frequently caused difficulties. Similarly, when they tried to use the internet in the 1990s and early 2000s, they ran into difficulties with weak internet signals. Even today, low WIFI connectivity, especially in schools in underprivileged neighborhoods (Stacy et al., 2017), continues to hinder the effective integration of technology in teaching. Cassidy (1998) argued that improving the technology itself is not the answer to persistent technology integration problems. No matter how sophisticated a technology is, technology integration is not likely to be effective unless the barriers that have remained constant for decades are removed. 2.2.1 Theoretical Frameworks for Understanding Barriers to Technology Integration Several theoretical frameworks have been developed to categorize and conceptualize the barriers that hinder technology integration in education. The two most relevant to the current study include Ertmer’s (1999) conceptual framework of first and second-order barriers and Davis’ (1989) technology acceptance model, which initially emerged to study acceptance of information technology systems in office environments and was later adopted in other domains. 2.2.2 First and Second-Order Barriers Ertmer (1999) coined the terms first-order and second-order barriers to classify obstacles to technology integration in schools. First-order barriers are those related to the educational context, such as problems with access to devices, WIFI connectivity in cases where the technology requires internet connectivity, and low funding. Second-order barriers are those related to educators themselves, such as low self-efficacy with using technology, pedagogical values that do not favor technology, and institutional norms that do not encourage technology use. Ertmer (2012) advocated for targeting second-order barriers when attempting to improve technology integration. When second-order barriers such as low technological self-efficacy are 17 addressed, educators are more likely to find ways to work around first-order barriers. For example, low access to devices could be managed by having students share devices while low internet connectivity could be compensated for by saving internet pages on the devices. Vongkulluksn et al. (2018) found that educators’ values are also a strong predictor of technology integration. Like Ertmer (2012), they suggested focusing on educators and not just on access to technology in order to promote technology use: …policy makers and administrators need to be mindful that while increasing technology access in the classroom is important, ensuring that teachers have positive value beliefs towards technology is also an important part of overcoming teachers’ perception of external barriers to technology integration. (Vongkulluksn et al., 2018, p. 79) Tsai and Chai (2012) expanded on Ertmer’s framework by adding third-order barriers, which include barriers due to educators’ weaknesses in design thinking. Tsai and Chai (2012) defined design thinking as a creative process that “treats all barriers as problems that need to be tackled and resolved through human creative thinking” (p. 1058). According to Tsai and Chai (2012), high self-efficacy and pedagogical values that favor the use of technology are not sufficient for educators to work around first-order barriers; instead, effective technology integration requires creativity. Makki et al. (2018) studied the effect of improving educators’ design thinking in an experiment. The independent variable was educators’ participation in a week-long summer institute where they were guided to incorporate technology into their teaching, and the dependent variable was the educators’ intention to use computer features in their teaching. The authors found that “those who attended the summer institute were 22.8% more likely to express intentions to use a variety of computer features” (Makki et al., 2018, p. 18 92). Makki et al. (2018) demonstrated that participation in design activities could positively influence the relationship between educators and technology. 2.2.3 Technology Acceptance Models To describe the process of users’ resisting or accepting technology, Davis (1989) developed the technology acceptance model (TAM). This model is based on two constructs, perceived ease of use and perceived usefulness, and stipulates that users’ perceived ease of use and perceived usefulness of a technology can predict their technology acceptance. Several extended technology acceptance models have been based on the TAM to include additional factors that predicted technology acceptance (Venkatech et al., 2003). In an educational application of the above models, Teo (2015) synthesized the TAM and other technology acceptance models to identify predictors of teachers’ acceptance of technology. He identified seven factors that predicted the acceptance of a technology among teachers: (1) perceived usefulness; (2) perceived ease of use; (3) attitude towards technology use; (4) norms; (5) facilitating conditions; (6) computer self-efficacy; and (7) the perceived complexity of the technology in question. One potential benefit of Teo’s (2015) framework is that it could help researchers to identify barriers and facilitators of technology use in the contexts they study using these seven factors. Examining educators’ perceptions and attitudes towards technology use along with the school norms and conditions for the use of technology can support researchers in making predictions about technology acceptance. Notably, among the seven factors identified by Teo (2015) as predictors of educators’ acceptance of technology, only one (i.e., facilitating conditions) is related to the context in which educators work while the other six are related to educators’ perceptions of technology and the habits and norms that they establish around technology use. This connects to Ertmer’s (2012) emphasis on the importance of focusing on 19 educators’ beliefs and perceptions of technology use instead of focusing on factors related to the context such as funding, available devices, and internet connectivity. When educators are confident and interested in using technology, they are more likely to effectively incorporate technology in their context, despite technical constraints. In the current study, the researcher incorporated TAM concepts in the analysis of participants’ patterns of participation and their adoption or rejection of the technologies that were designed with them. For example, the researcher analyzed how events during the design process, such as failures of the designed technology during testing or incompatibility between the designed technology and the available devices, may have affected participants’ perceived usefulness, perceived ease of use, and adoption or rejection of the technologies that were designed with them. Thus, in this dissertation, theoretical constructs from the TAM framework and from Ertmer’s first and second-order barriers framework are used to construct analytical descriptions of the practices of the three groups of educators with whom the researcher worked. 2.3 Designing with Educators One of the primary motivations for including educators in the design process of technology is to “ensure likelier adoption of new technologies and greater success in their implementation” (Matuk et al., 2016, p. 81). In design research that has been conducted with educators, the objective was to help educators adopt new technologies more easily and effectively. To better contextualize the current study in the design research literature, this section contains a review of different types of design research conducted with educators. Design research, including design research with educators, can be conceptualized as an application of the action research approach (Hayes, 2018). According to Hayes (2018), action research is an approach to research in which researchers collaborate with community members to 20 (1) “address some problem or challenge” and (2) “through this problem solving, [develop] scholarly knowledge” (p. 303). In design research, the problem or challenge is addressed by creating technologies or other artefacts that mediate new practices. It is important to note that the conceptualization of design research as an application of action research is not a shared position in the qualitative research literature. Other researchers have differentiated between design research and action research, arguing that design research focuses more on the product while action research prioritizes social change (Guldberg et al., 2017). The position adopted in this dissertation is that design research is action research, which is in line with Hayes’ (2018) vision. In the literature on design research that is reviewed here and that informed the current study, the rationale for including participants in the design of products was to create products that facilitate positive changes in teaching and that create better work conditions for educators. For example, in their design work with teachers, Winters and Mor (2008) adopted the definition of design articulated by Simon (1969): “everyone designs who devises courses of action aimed at changing existing situations into desired ones” (Simon, 1969, p. 129, as cited in Winter & Mor, 2008, p. 581, emphasis added). Design research with educators aims to change existing situations into desired situations; nonetheless, different researchers vary in their manner of articulating what the desired situations are and the process of going from existing to desired situations. Grundy (1982) identified three categories of action research, each one with its own approach for articulating outcomes, involving participants, and evaluating the project. The three categories are technical, practical, and emancipatory. This categorical action framework allows us to organize and make sense of the diversity in design research for educational change. 21 2.3.1 Technical Action Research In this branch of action research, outcomes are articulated by the researchers in advance and are based on a review of the academic literature and/or scientific theories: “The end goal is to improve outcomes, often on the basis of pre-existing theory” (Hayes, 2018, p. 312). One case of design research that fits in this category of action research is a study conducted by Penuel et al. (2007) on codesigning assessment software with educators. The purpose of the codesigned software was to facilitate formative assessment processes in the classroom. Researchers articulated the definition of effective formative assessment prior to the start of the study. One of the tensions that emerged during the study was a misalignment between the researchers’ conceptions of formative assessment and the teachers’ existing and desired practices: “the gap between researchers’ ideas and teachers’ ideas about what to support was so wide that the researchers feared this team would not be able to design an application the researchers would consider to be of high quality” (p. 62). The researchers persisted with their pedagogical beliefs despite disagreements with the participants, and they “push[ed] teachers toward more innovative solutions” (p. 69). Kuure et al. (2016) conducted another study that fits the category of technical action research; in this study, teacher candidates took part in PD research with school children to codesign technology-enhanced language learning activities. The objective of the study was to support the teacher candidates to switch their perspective from perceiving themselves as language teachers to perceiving themselves as designers of technology-mediated language learning practices. The main objectives of the codesign process were not articulated by the participants (the teacher candidates), but by the researchers who determined in advance, based on theory, that teacher candidates would benefit from a switch in perspective and from developing a 22 perception of themselves as designers of technology-mediated learning experiences. In both Penuel at al. (2007) and Kuure et al. (2016), researchers had the leading role and authority in the design collaboration; hence, the studies were classified as technical action research. In other codesign studies conducted with educators, the leadership was shared between researchers and educators, and the articulation of the research outcomes and objectives followed a more democratic process. 2.3.2 Practical Action Research In practical action research, participants take an active role in the articulation of the objectives and outcomes of the research project because the researchers by themselves cannot objectively determine what is best for the participants (Hayes, 2018). One example of practical action research is a PD study conducted by Hayes et al. (2010) with educators. The objective of the study was to codesign tablet-based applications for creating visual supports. The function and use of the applications were determined in collaboration with the participants; researchers learned from the participants about their contexts and articulated with the participants the desired functionalities of the tablet-based applications. For example, one of their priorities was to create a system that reduced the time burden of educators and helped them save time because saving time was one of their priorities. Another example of practical action research is a study conducted by Fage et al. (2016) in which researchers and educators codesigned a system for creating and displaying within-activity schedules. In this case, too, the researchers planned the process and decided the system’s functionalities in collaboration with their participants. For example, they learned from their participants that students respond better to within-activity schedules when the pictures in the schedules are idiosyncratic; based on that, they designed a system where each student had a 23 personalized version of the schedules with idiosyncratic pictures. The studies conducted by Hayes et al. (2010) and Fage et al. (2016) will be discussed in more detail in the review of studies on designing tablet-based systems with educators. One study that exemplifies the difference between technical action research and practical action research is Matuk et al. (2016). The researchers in the study codesigned tools with science teachers to facilitate the use of a web-based learning environment in the teaching of science. One of the codesigned tools was for grading. The researchers’ initial goal was to design a tool that provided the teachers with all available data about the students; however, the teachers were interested in an easy-to-navigate tool that would be less cluttered. Teachers were more interested in efficiency than in seeing too much information, arguing that “[it] would take me a lot of time to decipher this” (Matuk et al., 2016, p. 90). The researchers wanted to create a technology that enhanced teaching, whereas the teachers needed a tool that would facilitate logistic tasks and make them less time-consuming. Unlike Penuel et al. (2007) who persisted with their priorities when the participants had different priorities, Matuk et al. (2016) “refocus[ed] requirements [of the technology] on teachers’ actual needs and goals” (p. 91). The approach taken by Matuk et al. (2016) demonstrates how practical action research approaches the task by refocusing the outcomes and objectives on the participants’ needs. In practical action research, researchers do not go into the field with rigid priorities; on the contrary, their priorities are continuously revisited as they learn more from their participants. Sometimes, researchers must choose whether to focus on empowering participants or on designing a technology that gets effectively adopted because the empowerment agenda is not always compatible with an outcome-based agenda (Parsons & Cobbs, 2014). Ideally, design research will aim to achieve both, whenever possible. 24 2.3.3 Emancipatory Action Research In emancipatory design research, the product and the adoption of the product are secondary issues and hold less importance for the researcher. Empowerment and social change are the primary focus of such projects: “in critical—or emancipatory—[action research], the goal is often major disruption of social structure” (Hayes, 2018, p. 312). Researchers in these projects are expected to continuously engage in a self-reflective process about how their research disrupts or sustains structural inequities. The researcher did not find any emancipatory design studies where educators were the primary participants. However, one study that illustrates the emancipatory approach in design research was conducted by Collin and Swist (2016) with youth. In their study, they collaborated with youth to codesign applications that helped to promote and support youth safety. One of the codesigned products was the “Appreciate-a-Mate” website in which youth could post messages of affirmation to each other. The purpose of engaging youth in the design of “Appreciate-a-Mate” was to foster empowering connections between youth -- connections that would facilitate the formation of youth inclusive publics. The product that was designed was less important than the empowering connections created in the process of designing and using the product. Collin and Swist (2016) and other design researchers tend to focus on the designed social formations that emerge through designing and using the products. In design research conducted with educators, whether it is practical, technical, or emancipatory, several forms of misalignment may happen, such as misalignment between the researchers’ and educators’ goals or misalignments between educators’ and students’ goals. For example, when Siozos et al. (2009) collaborated with teachers and students to codesign an online assessment application, they found differences between the teachers’ requests and the students’ 25 requests. Differences between stakeholders often lead to dilemmas for the design researcher, who may have to prioritize the priorities of one group of stakeholders in order to develop a final product that gets adopted. Design researchers are likely to experience several tensions in their research that will require them to firmly commit to one of the three action research approaches, a decision that could be difficult given that commitment to one of the approaches often entails some losses. Identifying in advance the action research principles that they want to follow can help researchers to manage the tensions that are likely to emerge during the design process with educators. When misalignments happen in a technical action research project, researchers are likely to prioritize the perspectives that are more compatible with empirical evidence and the theoretical accounts that inform the objectives of the research. When misalignments happen in a practical action research project, researchers are more likely to prioritize the perspectives that facilitate the adoption of the designed technology in the work and lives of participants. Emancipatory action research takes a radically different approach. In emancipatory action research, empowering participants is more important than creating a technology that introduces evidence-based practices into the practices of participants; in other words, creating a technology that introduces evidence-based practices into participants’ work but also perpetuates power inequities in the participants’ context would defy the purpose of emancipatory action research, and such a technology should be altered or dropped from use. The current study started as both an emancipatory action research and a practical action research project because it aimed to both empower participants and help them solve practical problems; nonetheless, contradictions between the emancipatory agenda and the practical agenda led to tensions in the design process, tensions which the researcher analyzed and interpreted in the coming chapters. 26 2.4 Codesigning Tablet-Based Applications with Educators The researcher located five articles that focused on involving educators in the design of tablet-based applications for creating visual supports. Although there is a paucity of work in this area, the studies did take place all over the world, including the United States (Hayes et al., 2010; Hirano et al. 2010), France (Fage et al., 2016; Fage et al., 2019), and Taiwan (Miao-En Chien et al., 2015), which increases their robustness. Hayes et al. (2010) as well as Hirano and their colleagues (2015) codesigned with educators in the United States two tools for creating visual supports: Motocos and VSked. Motocos was a system installed on a mobile device that allowed students with ASD to make requests by selecting a picture card from an interface. VSked allowed teachers to create between-activity visual schedules and have them displayed simultaneously on a big screen in class and on the mobile devices carried by the students. For example, when working on the weather activity displayed on VSked, all students saw a question on their device asking them about the weather and saw a set of pictures representing the different weather options. When the students selected the correct answer, they saw a rewarding animation; when they selected an incorrect answer, they received corrective feedback. Once students completed an activity, they could see a checkmark next to their picture on the big screen. The design approach followed by Hayes and her colleagues consisted of first doing field work that included observations and interviewing; then developing “design principles” based on insights from the fieldwork; and, finally, developing the products. For example, in the research project that led to the development of VSked, the researchers articulated the following design principles to guide the process of developing the application: “ease the transition to new tools by mimicking old ones,” “reduce teacher burden,” “automatically generate records and reports,” and 27 “design for flexibility and robustness” (Hirano et al., 2010, pp. 1635-1636). Hayes (2018) described the work of her research team as practical action research: the aim of the study was to solve practical problems that educators encountered and that were articulated as priorities. Hayes (2018) also argued that some aspects of her work were applications of the emancipatory action research approach. For example, one of the research participants pursued a graduate degree in education after participating in the participatory design study conducted by Hayes et al. (2010), and she argued that her participation in the research motivated her to pursue graduate education. Thus, the research project conducted by Hayes and her colleagues ended up being both a practical and an emancipatory action research project: it helped resolve practical problems that educators had and it supported a research participant in making an empowering identity change in her life by pursuing a graduate degree in education. Fage et al. (2016, 2019) along with educators in France codesigned tools for creating and displaying visual supports to support the inclusion of students with ASD in a mainstream classroom. They designed a tool that allowed teachers to create within-activity visual schedules to help students interact with their teachers and their peers (Fage et al., 2016). For example, one of the schedules was the “Ask to repeat” schedule, which consisted of the following steps: raising a hand, waiting for the teacher to respond, and then saying, “Can you please repeat?” A second tool that Fage and his colleagues codesigned with educators was a tablet application that supported students with ASD in emotional regulation. The emotional regulation application displayed a thermometer and prompted students to select the intensity of their emotional state. Then, based on the selection made by the student, the application displayed a coping tool, such as a soothing video, a soothing picture, or a prompt to breathe (Fage et al., 2019). Fage et al. (2016, 2019) followed the same design approach that was used by Hayes and her colleagues -- 28 they conducted fieldwork first, used it to articulate design principles, and finally developed and introduced into instruction specific products. While Fage and his colleagues did not categorize their research study using the technical/practical/emancipatory framework, their study can be classified as practical action research because it supported educators in solving practical problems and engaged them in defining the outcomes of the project. Miao-En Chien et al. (2015) and educators in Taiwan codesigned a tablet-based application (iCan) that allows educators to create picture cards that can be used by both educators and their students to make requests and communicate expectations. Miao-En Chien et al. (2015) also conducted fieldwork with educators and then developed their application; however, they did not specify how data from fieldwork was used to come up with design principles for the development of their application. Similar to the studies reviewed above, this study can be classified as practical action research given that it supported educators in resolving practical problems and engaged them in determining the goals and outcomes of the study. However, one key difference between this study and the previous studies is related to the devices that were used to create and display the codesigned applications. In the studies conducted by Hayes (2018) and her team and by Fage (2016, 2019) and his team, the design process was based on the devices that were used by the participants; the researchers did not introduce any new devices into the contexts for which they designed. On the other hand, Miao-En Chien et al. (2015) created an application that works on one particular type of device, Andoid™ devices, and they bought devices for their participants and trained their participants on how to use them. However, as the researchers admitted, this practice may be problematic in the long run because if the devices get damaged, participants will need to replace them and might not have the financial resources that are necessary to do so. Miao-En Chien et al. (2015) did not design their 29 intervention based on the infrastructure that was already available for their participants; instead, they created an application that required a new infrastructure, a set of Android™ devices. 2.5 Gaps in the Literature, Research Question In all the studies introduced in the previous section, the process of codesigning technology with participants was presented as a linear process: fieldwork, design, deployment. Tensions and challenges that design researchers may encounter during the design process, such as differences or disagreements between participants, the challenge of motivating participants to communicate their design ideas, and the resistance to using the new technology, were not described nor analyzed in any of the studies. The only reference to challenges encountered during design was found in Fage et al. (2019) who stated that they encountered barriers when they introduced their application into classrooms. However, they did not specify how they managed the barriers and/or overcame them: School staff wanted our intervention to be as short as possible […] Additionally, some teachers had some negative beliefs about tablets and gaming platforms for adolescents’ education (Ertmer, 2005), and more particularly for adolescents with ASD [ASD] (e.g., a tablet socially isolates the child). Finally, let us note that our experimental study had an overall positive outcome in the participating schools with regard to inclusion: our intervention allowed some of the adolescents to exhibit spectacular improvements in their behaviors and autonomy (p. 17). Fage et al. (2019) mentioned the barriers that they encountered, but then merely stated that their intervention was successful despite the barriers. There was no elaboration on how time 30 constraints and educators’ pedagogical beliefs affected the process of designing and introducing the codesigned application. The objective of the current study is to address this gap in the existing research by providing more thick descriptions and elaborate analyses of barriers and challenges that design researchers may encounter when designing with educators for students with autism spectrum disorder. The main research question that informed the current study was: What are the challenges and opportunities that surface when designing technology with special education educators as design partners and as informants? Designing and introducing new teaching tools requires change, and the special education classroom, like any other context, contains variables that function to facilitate change as well as those that function to resist change. The current study contains extensive descriptions and analysis of how such variables operated in the research contexts. 31 Chapter 3: Methodology and Methods This study was designed based on the principles of Participatory Design (PD), a methodology that involves participants in the design of technologies and artefacts that they use at work and in their everyday lives. Participants in the study included three groups of educators working with students with ASD; the researcher conducted a separate PD project with each group. In each collaboration, the researcher and the participants codesigned an online web application that participants could use to create and display visual supports for their students. The researcher then analyzed and interpreted data generated in each project to illuminate the challenges and opportunities of doing PD in the special education classroom. This chapter comprises an outline of the methods that were used to conduct the study and to analyze the generated data. First, the researcher introduces the PD methodology along with the research questions that motivated the study. Second, the chapter covers the structure of the study: participants, processes, and outcomes. Third, the researcher outlines data analysis methods that were used to analyze and interpret data. 3.1 Applying PD with Educators as Design Partners and Informants PD emerged in the workplace context at a time when technology was seen as having a de-skilling and degrading impact on some work processes (Halskov & Hansen, 2015). Early PD projects were motivated by the democratic ideal of giving workers control over the work process by engaging them in the re-design of technologies they use on the job (Halskov & Hansen, 2015). PD was later adopted in projects outside of the workplace in settings that included, but were not limited to, healthcare, community building, and education. As PD was used in new settings, the outcomes produced by PD projects became more diverse and were no longer limited to technologies; PD researchers codesigned with their participants a diverse set of artefacts that 32 included healthcare material (Smith et al., 2017), social media campaigns (Collin & Swist, 2016), and educational curricula (Merijke et al., 2019). In education, PD has been utilized with both educators and students in formal schooling contexts (e.g., K-12), university contexts, as well as out-of-school contexts, such as extracurricular or tutoring contexts. Examples of contemporary PD educational projects include digital applications for creating and administering assessments (Siozos et al., 2009), the development of educational games (Bossavit & Parsons, 2016), and the development of prototypes for digital applications that prevent and mitigate cyberbullying (Ashktorab & Vitak, 2016). The outcomes of PD studies are both epistemological and ontological. Epistemological outcomes include, but are not limited to, knowledge about participants and their contexts, knowledge about the effectiveness of design methods in eliciting participants’ design ideas, and knowledge about the tensions that are likely to surface in the codesign process. Ontological outcomes include improved work processes facilitated by codesigned technologies as well as changed mindsets among participants (Halskov & Hansen, 2015). Frauenberger et al. (2015) noted that there is a diversity of approaches within PD, with some studies favoring a political agenda (aligned with the emancipatory branch of action research) and other studies favoring a pragmatic agenda (aligned with the practical branch of action research). When using Grundy’s (1982) categories of technical, practical, and emancipatory research, we can classify emancipatory PD projects as emancipatory action research and pragmatic PD projects as practical action research. The involvement of research participants in the design process can take many forms and, within existing PD research, participants have been involved in design in a wide variety of ways. This diversity of types of participation is related to different levels of rigor and validity 33 (Spinuzzi, 2005), so many published PD studies use ambiguous terms such as “involvement in design” to describe participation without explicitly describing how participants were involved (Halskov & Hansen, 2015). This has led to the emergence of categorical frameworks to support researchers in describing how their participants were involved in design. Indeed, Druin (2002) has outlined four different roles that participants may take on in a PD process: user, tester, informant, and design partner. While Druin (2002) designed those categories to classify the different roles that children can have in a participatory design project, the framework can also be applied to studies with adult participants. Each of the four roles outlined by Druin (2002) engages the participant in different ways and at different phases of the design process. The roles of user and tester engage participants toward the end of the process when the final product or a prototype has already been developed. On the other hand, the roles of informant and design partner give participants a more active role in the design process early on. Informants are usually asked to communicate their preferences about what the product should do and how it should function; they are sometimes observed by the researchers to better understand their preferences, and their preferences are then translated into actual designs by the designer or the researcher. Design partners, on the other hand, are involved in both the ideation process and the process of interface design, and they participate in the making of decisions about form, look, and functionality by creating low-tech prototypes with the researchers (Druin, 2002). In the current study, participants recruited by the researcher at the beginning of the study had the role of design partners and were involved in all the stages of the design process: ideation, interface design, testing, and deployment. Participants recruited midway through the study participated as informants, not as design partners, due to their late recruitment and their limited 34 availability. The researcher analyzed data generated by the study with the objective of answering the following research question: What are the challenges and opportunities that surface when designing technology with special education educators as design partners and as informants? 3.1.1 Recruitment After receiving approval from the Behavioral Research Ethics Board at the University of British Columbia to conduct the study, the researcher requested and received permission from two school boards to recruit participants. The researcher also requested and received permission to recruit participants from the principal of a private school for children and youth with special needs (School A). The two school boards and the private school are located in the Lower Mainland of the province of British Columbia, Canada. The inclusion/exclusion criteria for recruitment consisted of the following: • To be considered for participation, participants had to be doing educational work with students with ASD. The roles that met the inclusion criteria included teachers, educational assistants, speech-language pathologists, and behavior interventionists. • Potential participants who had a caregiving role (parents and care givers) were excluded from recruitment since the focus of the study was educational. The sample of participants in this study was a convenience sample and recruitment was facilitated by two of the researcher’s professors. 3.1.2 Participants All names that were in this dissertation are pseudonyms; these include the names used for participants (teachers, educational assistants, speech language pathologists) and those for their students. 35 The first case in this study included one participant, Brenda. Brenda was a special education teacher at a public elementary school in the Lower Mainland of British Columbia, Canada. Brenda learned about the research project from the researcher’s professor and contacted the researcher to express her interest in participating. The second case included two participants: Shannon and Ferreira. Shannon and Ferreira worked as speech-language pathologists at a private school for students with special needs. They were recruited when the researcher visited their school to present the research project to potential participants. The third case included six participants: Caleb, a high school teacher at a private school for students with special needs, and five of his educational assistants: Kathryn, Harry, Sam, Caroline, and Suzanne. The rationale for involving educational assistants in the design process was based on the researcher’s observations in Brenda’s classroom. In his collaboration with Brenda, the researcher witnessed the central role assigned to educational assistants in the process of creating and using visual supports in the classroom; hence, he asked Caleb to recruit his educational assistants as participants. Caleb welcomed the idea, and, when the researcher visited Caleb’s classroom to introduce the research project to educational assistants, they all agreed to participate and signed consent forms. 3.1.3 Procedures: Overview The objective of each case in this study was to codesign web applications that participants could use to create and display visual supports. The researcher and participants first created prototypes on paper, then the researcher developed web applications with the following programming languages: HTML, javascript, and php. The researcher also adjusted and edited the web applications based on the feedback that was received from participants. One consequence of using programming languages to develop the applications was that the development of the applications took several weeks (sometimes months) which led to large time gaps between the 36 design meetings. It is important to note that several no-code design tools, such as Bubble and Glide (https://bubble.io/, https://www.glideapps.com/), have been released after the current study was completed and the use of these tools can improve the efficiency of building web applications and consequently reduce the time gaps between design meetings. The researcher created a general procedure for the three PD cases, which consisted of the following steps: (a) engaging in a semi-structured interview to introduce the study; (b) developing a paper prototype with participants; (c) developing an electronic prototype and getting feedback from participants; (d) developing and teaching participants to use the final product; and (e) observing participants using the product. Importantly, the procedure itself was an object of analysis for the study, and its effectiveness was assessed at the end of the project. Although this was the general approach taken for each case, the number and structure of the meetings differed between cases in response to contextual factors such as participants’ availability and priorities. The procedure descriptions below contain a more detailed outline of the specific procedures that were followed in each case. Note that the descriptions presented in this chapter focus on the procedure that was followed and the artefacts that were produced, while the events that took place in each collaboration are described more thoroughly in chapter that follows. The purpose of the procedural descriptions here is to clarify the methodological approach that was followed in each case, not to describe the particular events that took place in each collaboration. 3.1.4 Procedure Description: Brenda Brenda was a senior special education teacher at a public elementary school and the head of an academic enhancement program for students with special needs at her school. Her classroom was attended exclusively by students with special needs, and she was supported by 37 several educational assistants, each working one-on-one with a student. On the days when the researcher visited Brenda’s classroom, there were at least three students in class and three educational assistants supporting them. Brenda also had access to the gym room at school, and she would sometimes send one of her students, with an educational assistant supervising, to run on the treadmill. At the time of the research, Brenda had completed a graduate degree in special education, had taught a special education course at a local university for several semesters, and had been using technology in her instruction for over 4 years. Brenda wanted to design a web application for one of her students, Jerome. Eventually, she used the web application with another student as well. 3.1.4.1 First Meeting: Semi-structured Interview (April 2016) The purpose of the first meeting was for the researcher to inquire about how Brenda used visual supports and various technologies in her classroom. The first meeting took place in Brenda’s classroom on the afternoon after a teaching day. The researcher asked Brenda about the benefits and challenges of using visual supports in the classroom (in particular, digital visual supports) and about the type of web application that she sought to develop with the researcher. Brenda stated that she had already been using several digital applications to create her supports; however, the applications were not meeting her needs. She was hoping to create a digital application that would allow her to create and save all her visual supports in one place instead of having to use multiple applications to create different types of visual supports. 3.1.4.2 Second Meeting: Creating a Paper Prototype (April 2016) The purpose of the second meeting was to design a paper prototype with Brenda. The paper prototype was meant to be a visual representation of the final product’s interface. Brenda 38 had thought about the prototype and discussed it with her educational assistants prior to the meeting and she drew the paper prototype by herself with a pencil. The prototyping process took less time than the researcher expected, as Brenda had a clear idea of what she wanted. 3.1.4.3 Third Meeting: Feedback on the Electronic Prototype (May 2016) After the second meeting with Brenda, the researcher developed an electronic prototype based on the paper prototype. The researcher showed the electronic prototype to Brenda and they tested it together. Brenda requested some changes related to the form and layout. For example, she asked for a feature that would allow her to control both the background color of the visual support and the background color of individual cells within the support. As Brenda was making requests, she explained the rationale for her requests, and her explanations allowed the researcher to better understand her pedagogical priorities. 3.1.4.4 Fourth Meeting: Delivering the Final Product (September 2016) In the fourth meeting, the researcher delivered the final product, but as he and Brenda tested it, several technical problems occurred. Brenda also asked for changes to some of the features that she had requested previously. 3.1.4.5 Teaching the Educational Assistants to Use the Application (May 2017) Brenda asked the researcher to visit her classroom and introduce the revised web application to the educational assistants. The researcher introduced the most up-to-date version, which included some of the changes that Brenda asked for in the previous meeting, and emailed Brenda a final version of the application at a later point. 3.1.4.6 First Day of Using the Application (June 2017) On the first day of using the application, the researcher was not present in the classroom, and there was a technological barrier pertaining to uploading some pictures. Specifically, Brenda 39 and her educational assistants attempted to upload two pictures to the application’s library, but the uploading process failed. Brenda emailed the researcher and told him about the issue and sent him the two pictures via email. The researcher uploaded the two pictures to the application from home and informed her via email that this was done. Brenda and her educational assistants used one of the images uploaded by the researcher to create a visual support and save it to the library. The visual support that was created consisted of an image that represents a between-activity break. When the image is clicked, the background of the image either changes to green to indicate to the student that a break is available (Figure 3.1) or to red to indicate that a break is over or is not available (Figure 3.2). Figure 3.1 First visual support created by Brenda and her assistants 40 Figure 3.2 Appearance of the visual support after the picture is clicked 3.1.4.7 Observation (June 2017) The researcher visited Brenda’s classroom to observe how the visual support was being used. The researcher observed one educational assistant use the visual support with Jerome, one of the four students present that day; the researcher obtained consent from Jerome’s mother prior to the observation. During the observation, the educational assistant displayed the visual support, informed Jerome that the break was over, and asked him to press on the image. The background of the image turned red. The educational assistant then guided Jerome to the treadmill and Jerome started running on the treadmill, which was his next task. On the day of the observation, Brenda told the researcher that, prior to using the web application, she used a two-sided picture to communicate to Jerome whether he could ask for a break or not. The two-sided picture was attached to a strap that Jerome used to wear. The educational assistant would turn the image with the green background to the front to 41 communicate to Jerome that break was available and would turn the image with the red background to the front to communicate that break was no longer available. 3.1.4.8 Final Visit (March 2018) By looking at the web application’s log files, the researcher noticed that the application was being accessed regularly (i.e., at least three times per week) and that some pictures had been uploaded to the application’s online library. However, only two visual supports had been created: the one that was used to communicate whether break was available or not (Figures 3.1 and 3.2), and a second visual support that consisted of two pictures, one representing a break and one representing “choice time” (see Figure 3.3). Figure 3.3 Second visual support created by Brenda and her assistants During his final visit to the classroom, the researcher asked Brenda about her impressions of the PD project and her plans to create additional visual supports with the web application. 3.1.5 Procedure Description: Shannon and Ferreira Shannon and Ferreira were two speech-language pathologists working at a private school for students with special needs. They delivered language enhancement lessons to students. When they heard about the research project, Shannon and Ferreira were facing difficulties with one student, Alex, who did not seem motivated during his lessons. The researcher had initially 42 recruited their colleague, Dona, as a research participant; however, Dona and an administrator at the school asked the researcher if Shannon and Ferreira could participate in the project instead, to design a personalized digital application for their student. Given the researcher’s limited time and resources, Shannon and Ferreira were recruited instead of Dona. 3.1.5.1 First Meeting: Semi-structured Interview and Paper Prototype (November 2016) In the first meeting, Shannon and Ferreira were asked about their motivation for participating in the project, and they drew a paper prototype for the web application that they sought to develop with the researcher. Shannon and Ferreira were at the beginning of the paper prototyping activity, but they eventually created the paper prototypes by themselves. They wanted to create a digital booklet of multiple-choice questions that displayed an image/video on the left side of the screen and a multiple-choice question about the image/video on the right side. They also wanted the booklet to give feedback to the student, consisting of a visual animation and pre-recorded verbal feedback when he clicked on an answer. 3.1.5.2 Second Meeting: Introducing the Electronic Prototype (February 2017) In the second meeting, the researcher showed Shannon and Ferreira an electronic prototype of the product, and they practiced using the prototype together to create multiple-choice questions. Shannon and Ferreira provided feedback on the electronic prototype and inquired about the possibility of adding new features. 3.1.5.3 Third Meeting: Feedback on the Electronic Prototype (June 2017) The researcher added the features that Shannon and Ferreira asked for and met with them a third time. He delivered a complete version of the web application, and Shannon and Ferreira tried using it. As Shannon and Ferreira used the web application, some usability issues surfaced, 43 and they worked together with the researcher to identify improvements that could be made to solve the problems. 3.1.5.4 Fourth Meeting: Delivering the Final Product (June 2017) In the fourth meeting, the researcher delivered an updated version of the web application. Shannon and Ferreira tested the application and scheduled an observation. 3.1.5.5 Observation (June 2017) On the day of the observation, the researcher observed Shannon and Ferreira use the web application to create a series of multiple-choice questions about different parts of the human body (e.g., nose, mouth, hands; see Figure 3.4). The researcher also observed them deliver a mini-lesson to Alex using the multiple-choice questions that they created. When Alex entered the office, he was accompanied by an educational assistant and had a device with him that allowed him to make requests by pressing buttons. He was highly stressed, and the educational assistant, along with Shannon and Ferreira, tried to soothe him and direct his attention towards the multiple-choice questions. The questions were displayed on a device held by Shannon. Figure 3.4 Multiple choice question created by Shannon and Ferreira 44 3.1.5.5 Meeting after the Observation (February 2018) The researcher had an informal meeting with Shannon and Ferreira after the observation, and they both agreed that an improved version of the web application might be beneficial for Alex and for other students. The improved version has not yet been developed. 3.1.6 Procedure Description: Caleb and Educational Assistants At the time of the research, Caleb was a high school teacher at a private school for students with special needs, the same school where Shannon and Ferreira worked. Caleb was recruited when the researcher visited the school to present his research project. Caleb’s students were youth with special needs each of whom worked with an individual educational assistant. Caleb and his educational assistants delivered a holistic curriculum that included math, language skills, and social skills. 3.1.6.1 First Meeting: Semi-structured Interview and Paper Prototype (October 2016) In the first meeting, the researcher asked Caleb about his experiences using visual supports (digital visual supports in particular) and his motivation for participating in the project. Caleb and the researcher sketched the paper prototype together for a web application that Caleb sought to design, which would allow him to create vertical visual schedules. Caleb wanted to use the application to create a vertical visual schedule and show it on a large screen to all students. He also wanted the application to be accessible from the students’ tablet devices, so that educational assistants were able to go through the schedules with their students individually. 3.1.6.2 Second Meeting: Electronic Prototype (January 2017) In the second meeting, the researcher showed Caleb an electronic prototype of the application, and they tested it together. Caleb then asked the researcher to visit the classroom and present the prototype to the educational assistants. 45 3.1.6.3 Third Meeting: Feedback on the Electronic Prototype (November 2017) The researcher presented the prototype to the educational assistants and then interviewed each of them individually. Some educational assistants reported an interest in using the web application. Two of them stated that a different web application, a game that involved matching, would be more useful to the specific students with whom they worked. As such, to engage these two educational assistants in the PD process, the researcher decided to create a matching game web application for them. Unlike the process that was followed with Caleb, which included creating a paper prototype with him, the process followed with the two educational assistants did not include any prototyping activities. Instead, the researcher used his judgment to make decisions on their behalf based on the ideas they shared in the interviews. This approach was adopted in consideration of the time constraints faced by the educational assistants. The researcher was able to make design decisions based on their ideas in less time than the prototyping activity would have required. 3.1.6.4 Fourth Meeting: Delivering the Revised Product and Prototype (December 2017) The researcher visited Caleb’s classroom to introduce a revised prototype of the scheduling web application and a prototype for the matching game. He received feedback on the two prototypes from the educational assistants. The researcher interviewed each educational assistant separately for few minutes while instructional activities were going on in the classroom. The researcher summarized to Caleb the feedback that he received from educational assistants. 3.1.6.5 Fifth Meeting: Delivering the Final Products (April 2018) After the fourth meeting, the researcher finalized the scheduling web application based on the feedback he received from educational assistants on the electronic prototype. The application consisted of two interfaces, one interface for users to create the schedules and one 46 interface that displayed the created schedules. Figure 3.5 shows an example of a schedule created with the application; this schedule was created by the researcher to demonstrate how schedules were displayed. The researcher also delivered two matching games, as requested by the educational assistants. The matching games had an identical format, consisting of an image centered on the top of the frame and four images on the bottom. When the bottom images were numbers, the student had to count the number of animals or objects in the top image and choose the right number. When the bottom images were colors, the student had to pick the color of the top image. The images that appeared in each matching game depended on the interests of the student who was going to use it, as described by the educational assistants in the interviews. One educational assistant used the matching game (Figure 3.6) with his student while the researcher was interviewing other research participants Another educational assistant tried to use the scheduling web application to create a visual schedule, but she ran into technical problems and was unsuccessful. 47 Figure 3.5 Screenshot of a schedule created on the web application 48 Figure 3.6 Screenshot from the matching game used by Harry with his student The PD project with Caleb and his educational assistants ended after the fifth meeting because Caleb and one of the educational assistants were planning to leave the school in the following academic year. 3.2 Data Analysis For each group of educators, data were generated from transcriptions of the audio-recorded meetings, prototypes, and field notes. The researcher also examined email correspondence with each group of educators to reconstruct the timeline and events for each collaboration. Given that emails were exchanged with the expectation of privacy, email correspondence was not included as data; email correspondence was used only to verify events that the researcher recollected and to make note of key events that were not included in the formal data set. For example, in one email, Brenda stated she was unable to meet with the researcher because she was busy training new educational assistants; the researcher added this event to his field notes. 49 3.2.1 Description, Analysis, and Interpretation The researcher conducted data analysis in this study based on Wolcott’s (1994) framework in which data are “transformed” into answers to research questions by progressively going through three transformations: (a) description; (b) analysis; and (c) interpretation. Following Wolcott (1994), the researcher recognized that the three phases are impossible to separate and that the act of describing is itself analytical and interpretive. As Wolcott (1994) noted, interpretation starts the moment we gather data: “In the very act of constructing data out of experience, the qualitative researcher singles out some things as worthy of note and relegates others to the background” (p. 13). However, even though the three processes of description, analysis, and interpretation are inseparable, one can still discuss them separately to describe a different emphasis in each stage of the analysis. Hence, in the description stage, the emphasis is on answering the question of “what happened?”; in the analysis phase, the emphasis is on presenting the themes that regularly occurred and the key differences that emerged; and the emphasis in the interpretation phase is on answering the “so what?” question and articulating the theoretical significance of the findings and their research implications (Wolcott, 1994). 3.2.1.1 Description In the description phase, the researcher produced a description of each collaboration and its context. In writing the descriptions, the researcher attempted to stay close to the data. The primary objective of the description was to provide an account of what happened in each collaboration. Chronological descriptions presented, in chronological order, the events that took place and identified the primary actors in each case. The researcher divided chronological descriptions into three phases: the phase of articulating goals, the phase of designing with the participants, and the phase of introducing the codesigned product into practice. 50 Descriptions of critical events consisted of detailed accounts of events that revealed important characteristics about the context of each case and how those characteristics affected the design process. According to Wolcott (1994), a critical event is one where “the essence […] of the whole is revealed or reflected in microcosm” (p. 19). In this study, a critical event refers to an event that revealed how multiple characteristics of a particular case interacted and affected the design process. The descriptions of critical events also included events that were unexpected whether because they revealed a gap in extant literature on the topic or a gap in the planning of the design process. Hence, a critical event, for the purposes of this study, had to meet at least one of the following criteria: • The event revealed a gap in the relevant research literature (i.e., the literature on PD or the literature on visual supports); • The event revealed a gap in the procedure that was adopted for the design process; and/or • The event demonstrated how several factors that were particular to one case affected the design process (the whole revealed in microcosm). 3.2.1.2 Analysis In the analysis phase, the researcher coded, compared, and contrasted data to make inferences about the process of conducting PD in a special education context. Inferences made in the analysis phase were cautious, and the aim was to build the inferences from the bottom up by starting with the data. The researcher used the following approaches to conduct the analysis: (a) identification of patterned regularities both within and across cases; (b) comparisons between the 51 three cases in this project; and (c) comparisons between cases in this project and other known cases of using PD in special education (Wolcott, 1994, pp. 33-34). Identification of patterned regularities. In this part of the analysis, the researcher coded data from the three cases and grouped into themes codes that emerged repeatedly within a case and/or across the three cases. Comparisons. Differences between cases created some opportunities for developing a better understanding of the impact of specific factors on the design process. For example, one key difference between the collaboration with Brenda and the one Caleb was that, in the former, educational assistants only had a limited involvement in the design process while educational assistants working with Caleb were research participants who were involved in the design and decision-making processes. This difference allowed the researcher to make inferences about the challenges and opportunities that surface when designing with teachers and educational assistants together. Another type of comparison that enhanced the analysis was a comparison between the cases in the current study and other known cases of PD in special education contexts. This type of comparison helped the researcher overcome some of the limitations of working with a small data set: “[In qualitative research], small samples and always too-brief opportunities to observe preclude us from reporting authoritative correlations, but careful reporting of what we actually observe […] can serve as a powerful restraint on overgeneralizations made by others” (Wolcott, 1994, p. 33). While this study’s small data set allowed a limited range of inferences and generalizations, data from this study have also been used to assess generalizations made by other PD studies. 3.2.1.3 Interpretation 52 In the interpretation phase, the researcher turned to theory and to personal experiences to extend the inferences made in the analysis phase. The researcher adopted a theoretical lens to reflect on the data and the research process and to articulate recommendations on conducting PD projects in special education contexts. Unlike the cautious inferences made in the analysis phase based solely on the data, inferences made in the interpretation phase transcended the data (Wolcott, 1994). The interpretation phase was divided into two parts: recommendations and autoethnographic reflection. While creating recommendations, the researcher used theories on visual supports, participatory design, and technology integration to extend the inferences made in the analysis and to articulate recommendations for future research. In the autoethnographic part of the interpretation, the researcher examined his personal experiences as a PhD candidate conducting PD with special education professionals. In autoethnography, reflections on personal experiences are employed to answer questions about social and cultural processes (Chang, 2013). By examining and interpreting his experiences, the researcher aimed to provide critical insights on the process of conducting a PD project in the context of special education. In agreement with the observations of Frauenberger et al. (2015), the researcher’s agenda and commitments shaped the unfolding of the PD process, so a reflection on these commitments was essential for providing a comprehensive account of the research process. Also, based on the social constructionist epistemology which grounds the PD methodology, the researcher conceived the relationship between himself and the research project as a dialectical process: the researcher’s values shaped the research process, and the research process shaped how the researcher examined and rearticulated his values. Hence, a dialectical account of the research process requires an account of how it transformed the researcher’s intellectual commitments. 53 3.3 Generalizability and Reliability The objective of PD studies is to produce context-specific knowledge that can be used to create technologies for similar contexts. However, PD studies are highly personalized, and the procedures that are followed differ from context to context, which is a condition that makes PD studies difficult to replicate. Context specificity and difficulty of replication requires PD researchers to address the generalizability of their knowledge claims and their reliability (Frauenberger et al., 2015). In other words, the methods that have been traditionally used in research, such as replication and large sample size, are difficult to use in PD and, as a result, trustworthiness of knowledge should be assessed by using alternative methods. 3.3.1 Supporting Generalizability Through Thick Descriptions The knowledge produced by PD might be generalizable to contexts similar to the context in which the study took place. Hence, “Understanding how knowledge depends on the context is a pre-requisite of being able to transfer it to other contexts” (Frauenberger et al., 2015, p. 99). As stated previously, the analysis process started with detailed descriptions of each context, toward the goal of supporting generalizability. By providing detailed descriptions of the unique factors that shaped the design process in each case, the researcher aimed to facilitate the process of assessing the similarities and dissimilarities between the context of the study and other contexts, and to assess the applicability of findings from the present study to other contexts. 3.3.2 Assessing Reliability Replication is fundamental for assessing the reliability of scientific knowledge. However, the reliability of knowledge claims produced by PD is difficult to establish, given that highly contextualized PD projects are difficult to replicate in new contexts (Frauenberger et al., 2015). 54 To support PD researchers in assessing the reliability of their studies, Frauenberger et al. (2015) articulated a framework that consists of four lenses: epistemology, values, stakeholders, and outcomes. Each lens is meant to allow the researcher to examine a PD project with a different focus. The epistemology lens focuses on the knowledge produced by the project. The values lens emphasizes how the researcher’s and the participants’ values influence the dynamics of the PD project. The stakeholders lens concerns the impact that the PD project has on various stakeholders, including stakeholders who are not research participants but who are impacted by the project and its outcomes. Finally, the outcomes lens focuses on the outcome that is produced by the PD project, whether it is a technology, an artefact, a changed mindset, or any other outcome that was created and/or elicited. The four lenses articulated by Frauenberger et al. (2015) can be used to assess the reliability of a PD project by examining alignments and misalignments between the different lenses. Misalignments between two or more of the lenses often point to weaknesses in the reliability of the project. For example, if the researcher’s aim is to empower participants and contribute to creating equitable power relationships between them (the values lens), yet the product of the PD project is a technology that adds more non-skilled work to one group of stakeholders (the outcomes lens and the stakeholders lens), this would be a case of misalignment, and the researcher would have to examine and address the misalignment in order to improve the internal reliability of the study. The researcher assessed the current study’s reliability by examining alignments and misalignments between the lenses of epistemology, values, stakeholders, and outcomes. The assessment is reported in the final chapter following the interpretation. 55 Chapter 4: Study Findings The researcher organized findings of the study into three categories: description, analysis, and interpretation (Wolcott, 1994). In this chapter, the researcher presents description and analysis. The description section contains a chronological outline of each case, from the stage of articulating specific goals to the design of products and to product integration in participants’ teaching. The descriptions of critical events focus on two types of events: unexpected events and events that demonstrated how different contextual factors interacted to shape the design process. In the analysis, the researcher points out common regularities that surfaced across cases and that illustrated the challenges and benefits of conducting participatory design research with educators. 4.1 Chronological Descriptions The objective of the chronological descriptions shared here is to narrate what happened in each case. The chronological descriptions are structured as narratives consisting of three stages: (a) articulating goals; (b) designing the products; and (c) introducing the products into practice. Together, these three stages form a plot that the researcher applied to the series of events that happened during the research process. By using the structure of a plot, the researcher engaged in the practice of emplotment: “the linking of the events into a meaningful sequence and structure” (Holloway & Freshwater, 2007, p. 11). The benefit of plots is that they organize experiences into meaningful narratives; however, their limitation is that they are biased, that is, they foreground some events and push other events to the background. The plot used here has been chosen because of its relevance to the research question which focused on the challenges and opportunities in the design process. In the next chapter, in the section on auto-ethnographic analysis, the researcher will revisit events from the research process again with a different plot, one that is also relevant to the research questions and that will foreground other types of events: 56 the plot of the researcher planning a PD project, implementing the project, and encountering challenges, and learning from the challenges that were encountered. 4.1.1 Description of the Collaboration with Brenda Brenda was the first research participant in the research project; she heard about the research from one of the researcher’s professors and contacted the researcher to express interest in participating. She was interested in technology and had previously created a system for creating visual supports with tablet devices by installing multiple commercial applications and uploading images to their built-in libraries. She perceived the system she had as effective but also time consuming: “when we first got the iPads, I thought that it was going to significantly reduce our prep time … I don’t know that it has, because we spend a lot of time updating and changing, so I don’t know if it’s more time effective or not, but it’s definitely, I think, more engaging for students” (Brenda, meeting 1). Brenda was supported in her work by a group of educational assistants; thus, her responsibilities included both student instruction and the educational assistant management. As Brenda was designing her web application, she thought about both students and educational assistants when making design decisions. 4.1.1.1 Articulating Goals The first meeting with Brenda took place in her classroom after she finished teaching. In the meeting, Brenda talked about her previous experience with visual supports, both tablet-based and paper-based, and about the limitations of the available technology. Brenda told the researcher that she found it inconvenient to use multiple applications to create different types of support; for example, the application she used to create visual schedules did not allow her to embed videos, and, as a result, she had to find another application for video modeling. A second issue that Brenda encountered was the lack of consistency between the visual symbols that 57 different software applications provide: each software had a different built-in library of visual supports. This was not ideal for students with ASD, according to Brenda, because when their vocabulary gets built around a specific set of symbols but then they see another set of symbols used to refer to the same words and ideas, they can be confused. By the end of the first meeting, Brenda had identified her goal: she wanted to create a digital application that would allow her to create different types of visual supports. She wanted an application that would allow her to continue creating the supports she had been creating, but in one place and more easily and efficiently: “I’d like to think about a few different ways that we use apps that I find that I have a frustration with and then try to create something that does what the ones we’re using don’t do that annoys me” (Brenda, meeting 1). 4.1.1.2 Design Process The design process with Brenda started in the second meeting. The researcher had expected to design the paper prototype with Brenda during the meeting; however, by that time, Brenda had already discussed the design with her educational assistants, and when the meeting started, Brenda started sketching the prototype by herself without the need for the researcher’s support. Figure 4.1 displays the paper prototype. 58 Figure 4.1 Prototype sketched by Brenda The prototype included three parts. The first element, on the top, was a Go/Stop sign; the second part was a horizontal visual schedule, and the third part was a picture of a target behavior and a picture of the reward a student would receive after completing a sequence of activities. Brenda wanted to use the application as follows: an educational assistant shows the screen to the student and the student starts following the schedule. If the student misses a step, the educational 59 assistant presses on the image of the step that was missed, the background of the image turns red, the universal ‘no’ symbol gets displayed on top of that image, and the backgrounds of the target behavior box and the reward box also turn red with the universal ‘no’ symbol displayed on top of both. Then, when the student performs the step he previously missed, the educational assistant presses again on the image representing the missed step, and the background turns green again. One issue that was not discussed in the second meeting but surfaced later in the design process was that Brenda wanted to use the universal ‘no’ symbol to convey two different meanings: the symbol on top of the image of the reward meant that the reward was “no longer available,” but the symbol on top of the pictures in the schedule meant that the student did not follow the step. The researcher did not initially understand this ambiguity; it was Brenda herself who noticed the issue in a subsequent meeting and requested that the universal ‘no’ symbol be dropped from the application to avoid confusion: “I would never say to him [the student] that positive behaviours are not available. It's that when he does negative behaviour the reinforcer is not available” (Brenda, meeting 4). Brenda also asked the researcher to remove the box representing the target behavior; she preferred a simpler interface with the visual schedule on the top and an image of the reward on the bottom. A consistent priority for Brenda throughout the design process was to create an application that would be easy to personalize, because she wanted to use the application to create multiple types of visual supports, such as visual schedules, video models, and contingency maps. This led her to ask for several changes, to remove elements that she had initially requested (such as the universal ‘no’ symbol), and to add new features that she identified. While the drawing of the paper prototype helped her identify some of the features that she wanted, seeing the electronic prototype that the researcher developed based on the paper prototype allowed her to 60 identify features that she had not previously considered: “I could use this to even just have like a consequence map or like a visual script on how to wash your hands, right. Like this can be used in so many ways beyond what we’ve initially talked about” (Brenda, meeting 3). By the end of the design phase, the researcher had developed an application for Brenda that consisted of two elements: first, a library where Brenda could upload her images and videos; second, an application that would allow her to use the library to create either vertical or horizontal sequences of pictures or videos (i.e., visual schedules) or a single box with one image or one video (i.e., a simple visual symbol); the application also allowed her to have a reward box on the bottom. 4.1.1.3 Introducing the Product Introduction of the designed product into Brenda’s practice took place in two stages; first, the product was introduced to her educational assistants, and then they used it with one student, Jerome. As the researcher introduced the application to the educational assistants, some of them had problems with accessing and using it. They were used to Apple™ applications -- apps specifically designed for iPads™ and iPhones™. The application created with Brenda was a web application; it required users to open a web browser and enter a web address to access it. Brenda supported the researcher in resolving the difficulties that some educational assistants were facing. She asked them to save the web address in the “Favorites” folder, so that next time they would not have to enter the web address into the browser but could access the application by first opening the browser and then clicking on the “Favorites” folder. Brenda’s help surprised the researcher: her assistance revealed her high technical knowledge and her creative ability to solve technical problems. Using Ertmer’s (1999) concept to describe this incident, we can say that Brenda helped resolve a first-order barrier to technology integration (i.e., the problem of 61 incompatibility between the designed technology and the available infrastructure) and a second-order barrier (i.e., educational assistants’ difficulties with technology) by using her technology competence (a second-order facilitator). After the meeting with educational assistants, the researcher fixed some of the technical issues that surfaced when he introduced the application and then sent a link to the application to Brenda by email. However, on the day that Brenda tried to use the application, the educational assistants were not able to upload pictures to the application’s library. Brenda emailed two symbols to the researcher while she was in class and asked him to upload them to the library; the researcher did so and she was able to use the application to create a visual support. Brenda created only two visual supports with the application -- one that she used to communicate to Jerome when he could request a break and another to communicate to another student when he could ask for a break and when he could make a choice. The two visual supports she created were neither within-activity schedules nor between-activity schedules; they were interactive visual symbols with background colors that could be changed by clicking on them. If the background color of the picture representing a break was green, the student could ask for a break/make a choice; if the background was red, then the student could not. The researcher observed one of the educational assistants using the application with Jerome. The educational assistant used a support that consisted of one picture, representing a break. When Jerome could ask for a break, the educational assistant verbally prompted him to press on the picture to change its background color from red to green. When the break was over, the educational assistant verbally prompted Jerome to press on the picture to change its background color to red, signaling to Jerome that he could not ask for a break. Both Brenda and 62 the educational assistant stated that the visual support was useful in communicating with Jerome, they also stated that Jerome was responsive to the visual support. 4.1.1.4 Summary The collaboration with Brenda revealed several challenges and benefits of conducting participatory design research in this special education classroom. The main challenge was the mismatch between the type of technology that was created with Brenda (a web application) and the technology that Brenda and her educational assistants were regularly using (native iOS applications). However, collaboration with Brenda also revealed many benefits of conducting PD in the special education classroom, including how design can function to elicit educators’ pedagogical knowledge (e.g., Brenda’s knowledge about the importance of consistency for students with ASD), consistency in the visual symbols that are used, and consistency in the meaning that is assigned to a given visual symbol. 4.1.2 Description of the Collaboration with Shannon and Ferreira Shannon and Ferreira worked as speech language pathologists at a private independent school attended by students with special needs. The school enrolled students at both the elementary and high school level. Shannon and Ferreira did not have a classroom; they shared an office where they delivered their language enhancement lessons. Shannon and Ferreira asked to participate in the project because they were encountering challenges with one of their students, Alex, and they thought that a tablet application might increase his engagement in the lessons. 4.1.2.1 Articulating Goals In the first scheduled meeting with Shannon and Ferreira, they discussed their objectives with the researcher. Shannon explained that the available technology that Alex was using allowed him to make requests by pressing on images on his tablet, but they wanted to create an 63 application that would help him develop competences beyond simply making requests, such as answering questions and having a conversation. Alex’s mother also wanted him to develop more advanced competences: Shannon: “Right now, we are working on requesting, but we want him to move on to the next step like making comments, answering, you know, answering questions.” Ferreira: “And I guess that his mom has also some expectations for him and she mentioned some of them in one of the meetings that we had. And yeah, so she really wants him to be in the classroom.” (Shannon and Ferreira, meeting 1) In the same meeting, Shannon and Ferreira drew two prototypes for two digital applications: one that would allow Alex to answer questions about images or videos (Figure 4.2), and another that would allow him to match objects to colors. Given the limitations of time and resources, the researcher created only the first application. Figure 4.2 Paper Prototype: Shannon and Ferreira 64 4.1.2.2 Design Process As can be seen from the prototype that was drawn by Shannon and Ferreira, they wanted the application to have three elements: on the top, they wanted a written question, in addition to an audio button, so that Alex could hear the question when he would press on the button. On the left side of the screen, they wanted a picture or a video. On the right side of the screen, they wanted to display multiple answers to the question, so that Alex could choose the correct answer. Each answer appeared in a box that included an audio icon, so that Alex could also hear the answers if he wanted to. The researcher developed an electronic prototype based on the paper prototype drawn by Shannon and Ferreira. Their feedback on the prototype was positive, and they did not request any changes. The researcher then developed the final product and met with Shannon and Ferreira to teach them how to use it; their feedback at that point was positive as well. To create question and answer exercises, Shannon and Ferreira had to upload the image or video that was relevant to the target question. They then they had to enter the question, an audio file of the recorded question, the images for each answer, the text for each answer, and the audio file for each answer. They also had to upload the image they wanted the student to see upon answering correctly and the picture he would see if he answered incorrectly. While the application gave Shannon and Ferreira full control of what appears on the screen, it also required them to do many tasks. This created challenges during the testing phase when Shannon and Ferreira tried to create exercises and the application malfunctioned because they missed one of the input steps. The application did not include a mechanism for preventing or flagging user errors. After creating the final product, the researcher met with Shannon and Ferreira to show them the final product, explain how to use it, and schedule a day for an observation. The meeting 65 occurred toward the end of the school year, and the following week would be the last week of the school year. Given the option to either request an observation in the following week or wait until the next school year, the researcher requested scheduling the observation in the following week. Planning the observation at the end of the school year had both benefits and limitations. It was beneficial because it allowed the researcher to observe the participants at a busy time of the year and assess the usability of the technology at such a time. On the other hand, the drawback was that introducing a new technology at a busy time may have reduced the likelihood of its adoption. 4.1.2.3 Introducing the Application On the day of the observation, the researcher arrived at Shannon and Ferreira’s office before Alex. Shannon was preparing the digital booklet that she was going to use with Alex. The booklet included questions on the different parts of the human body. It had three pages: one showed an image of a mouth on the left and two options on the right, another showed an image of a nose on the left and two options on the right, and the last one showed an image of hands on the left and two options on the right. Shannon missed many steps of the input task; thus, while the visual support was functional and usable, it was incomplete. For example, one of the pages missed feedback. The researcher concluded that the usability of the application could be further improved. When Alex entered the classroom, he was accompanied by an educational assistant and he was obviously distressed. He was carrying a tablet that allowed him to make requests by pressing on buttons. Alex used that application several times to request social interactions such as hugging and tickling. Shannon tried to direct Alex’s attention to the new digital application, but Alex resisted. Shannon, Ferreira, and the educational assistant used both verbal and manual 66 prompts to support Alex in completing the exercises in the digital booklet. With the support of the prompts, Alex clicked on the correct answers; however, his gaze was not fully oriented towards the application. The researcher noticed that it was difficult for him to point his finger to the buttons and click on them; the researcher’s impression was reinforced by Shannon who suggested at the end of the observation that the researcher could make the buttons larger on the screen. After the observation, the researcher had an informal meeting with Shannon and Ferreira, and they agreed that an improved version of the application might still be beneficial for Alex and other students. An improved version of the application could be the starting point of a new collaboration with Shannon and Ferreira. 4.1.2.4 Summary The collaboration with Shannon and Ferreira revealed that there is a need for technologies and resources that support students with ASD in developing communicative competence. Perhaps such resources and technologies already exist, but Shannon and Ferreira were not aware of them. Collaboration with Shannon and Ferreira also revealed a gap in the design process; when the researcher developed the application based on the preferences of Shannon and Ferreira, this process assumed that their preferences would represent the Alex’s preferences as well. However, educators do not necessarily know the preferences of their students, and even when they do, translating students’ preferences into usable and engaging designs may require competencies that they do not have. The design process gave Shannon and Ferreira the freedom to design the interfaces they wanted, but the process also assigned additional responsibilities to them, such identifying Alex’s preferences and translating this knowledge into designs that would be engaging for him. 67 4.1.3 Description of the Collaboration with Caleb and his Educational Assistants Caleb worked in the same school as Shannon and Ferreira and he expressed a strong interest in joining the project. Caleb’s situation had some similarities to Brenda’s; that is, he was a teacher working with several educational assistants, each of whom supported a different student. One difference between their situations was that Caleb’s educational assistants were recruited as research participants in the study while Brenda’s educational assistants were not. In the collaboration with Brenda, the researcher developed a strong understanding of the important role that educational assistants have in the special education classroom; consequently, he decided to make it possible for Caleb’s educational assistants to join the project as participants, if they were interested. 4.1.3.1 Articulating Goals In the first meeting with Caleb, the researcher discussed Caleb’s goals for the collaboration. Caleb wanted to create an application that would allow him to display a common schedule to all of his students on a large screen while also displaying the same schedule to individual students on their tablet devices. Caleb also wanted the application to allow him to do some scaffolding; for example, if the visual schedule included five items, he wanted some students to be able to see all five items together and while others would see only two items at a time. Caleb also had some pedagogical objectives; for example, he wanted the schedules to include both images and text to support students in developing literacy. 68 Figure 4.3 Paper Prototype Sketched with Caleb 4.1.3.2 Design Process Figure 4.3 show the paper prototype that Caleb and the researcher created together. Caleb wanted the application to have the following features: • First, a vertical visual schedule appears, with pictures representing an activity. 69 • Then, when the student or educational assistant clicks on the image of an activity, a set of images appear that represent choices. For example, if the first activity of the day is a math activity, then, when the student presses on the image, a set of math activities appear, each one representing a different math activity from which the student can choose. • Once the student or the educational assistant clicks on the activity that they chose, a timer appears to signal to the student how long the activity will last. Creating an electronic prototype that would match all of Caleb’s requirements was difficult. The researcher developed a prototype that included some of the requirements and told Caleb that the application could be updated at a later point. Caleb was flexible and was willing to use the application even though it did not match all his requirements. At the second meeting with Caleb, the researcher showed him the prototype and they tested it together. Caleb gave positive feedback on the prototype and asked about some additional features. For example, he asked for a feature that would allow him to easily store and access the images that he would use to create his schedules; this was a feature that the researcher could not include in the initial product but planned to incorporate at a later point. After going through the electronic prototype, Caleb requested that the researcher visit his classroom to demonstrate the application to the educational assistants. The researcher then asked Caleb to introduce the research project to educational assistants and to check if they would be interested in joining as research participants. When the researcher visited Caleb’s classroom, all educational assistants agreed to participate in the study and signed consent forms. During that meeting, the researcher showed the application to the educational assistants as a group and then interviewed them individually. In the interviews, not all educational assistants reported an interest in using the application. Harry and Suzanne preferred other types of applications for the 70 students with whom they were working, such as matching games. They argued that the individual students that they were helping did not have the required competency to use the visual schedule application. Kathryn and Sam, on the other hand, expressed a high interest in using the application. The researcher decided to continue working on the scheduling application based on the feedback that he received, and to create two matching games for Harry and Suzanne as well. 4.1.3.3 Introducing the Applications After the researcher developed the two matching games and the application for creating visual schedules, he visited Caleb’s classroom to introduce the applications. Harry and Suzanne stated that the matching games met their requirements, but both asked for more challenging matching tasks. Harry used the matching game with his student during the meeting and informed the researcher that the student enjoyed using it. In addition, Harry showed high interest in contributing design ideas and made suggestions for more matching exercises; for example, he stated that a matching task that would help the student master spatial concepts such as “below the table” and “on the table” would be useful. Suzanne stated that she was interested in using the matching game but did not use it on that day. Kathryn tried to use the visual schedule application. She took pictures with her tablet, uploaded them, and clicked on the button for creating the schedule. However, the schedule was not created. After testing the application, the researcher found that it did not function when pictures were uploaded from the tablet device; instead, pictures had to be uploaded from a computer device. The researcher informed Kathryn that he could create the schedules himself if she sent him either the pictures or a description of the schedule. He did the same with Sam. Kathryn did not contact the researcher; however, Sam contacted the researcher via email with a description of the schedules that he wanted to create. Unfortunately, there was high turnover in 71 Caleb’s classroom that made it difficult to continue the collaboration: Caleb left the school and started working elsewhere, Kathryn left the school to enroll in a teacher education program, and Sam was transferred to another classroom. 4.1.3.4 Summary The collaboration with Caleb and his educational assistants demonstrated both challenges and opportunities that could surface when conducting PD in a special education classroom. The challenges include differences in perspectives between teachers and their educational assistants and high turnover among educational assistant which may impact both their motivation and their availability to participate. The benefits include opportunities for educational assistants to develop interest in and self-efficacy for technology, this was demonstrated by Harry who, in the beginning of the project, referred to himself as “not a computer whiz” yet who ended up actively giving design suggestions towards the end of the project. 4.2 Critical Events In the current study, several events occurred that were unexpected. They revealed gaps in the researcher’s expectations, gaps in the literature on PD in special education contexts, and gaps in the research on visual supports. These unexpected events were considered critical as they helped the researcher produce relevant narratives about the challenges and benefits of conducting PD in the special education classroom. As Bruner (2002) noted, “narrative […] is a dialectic between what was expected and what came to pass. For there to be a story, something unforeseen must happen” (p. 15). In addition to unexpected events, there were several events that revealed how multiple factors interact in the special education classroom to affect the process of designing and using technology. In Wolcott’s (1994) words, these events “revealed the whole in microcosm” (p. 19). 72 Both types of events – the unexpected ones and those that revealed the whole in microcosm – are reported here as critical events. 4.2.1 Critical Events from the Collaboration with Brenda Three types of critical events occurred during the collaboration with Brenda, as described in the sections that follow. 4.2.1.1 Visual Supports Supported Both the Students and the Educational Assistants When the researcher asked Brenda about her experience with visual supports, she stated that visual supports had helped in her work with students with ASD as well as with educational assistants: “I know that having a very detailed visual schedule supports our educational assistants as much as it supports our students. Because it also… provides a common schedule that everyone is going off of and referring to” (Brenda, meeting 1). This surprised the researcher, given that the literature he had reviewed focused only on the impact of visual schedules on students; it did not examine their impact on the collaboration between teachers and educational assistants. In many other interactions with Brenda, it was evident that she thought about both students and educational assistants when she was making her design decisions. For example, in the fourth meeting, when she was making design decisions about the interface for creating the schedules, she asked the researcher to add cues next to the input fields, so that educational assistants would immediately understand how to use the interface: “And then is it also possible to like put a cue there, like 10 is a small space, 50 is a medium, to cue people? So I don't have, every time someone uses it, I don't have someone coming up to me going ‘how many pixels’ - because that's exactly what's going to happen (laughter)” (Brenda, meeting 4). In a retrospective review of the literature on visual supports, the researcher found only one reference to visual supports’ impact on the collaboration between teachers and educational 73 assistants. vSked was a web application developed by Hirano et al. (2010) in collaboration with educators to allow them to create visual supports for their students with ASD. The researchers noted that: Although many special education teachers plan team meetings for the end of each day, exhaustion and an hourly pay plan for aides that does not include time outside of class for aides make these meetings rare and difficult. Thus, teachers and their staff must frequently coordinate schedules dynamically and in the presence of the students. During our deployment, vSked supported this dynamic coordination by providing visual support not only to the students but also to the classroom staff (Hirano et al., 2010, p. 1369) As Hirano et al. (2010) noted, one additional benefit of the web application was that it facilitated the coordination between teachers and educational assistants in the classroom. 4.2.1.2 Emailing the Researcher When the Uploading of Pictures did not Work After completing the final version of the web application, the researcher sent the application’s web address to Brenda via email. On the day when Brenda tried to use the web application for the first time, she and her educational assistants were unable to upload pictures to the application’s library. Brenda sent an email to the researcher informing him of the problem, and the researcher uploaded the pictures to the library. Initially, Brenda’s email did not seem to be a critical event for the researcher; however, when the researcher compared Brenda’s reaction to the reaction of another research participant (Caleb) who encountered a similar problem, the impact that Brenda had on the research became more apparent. Both Brenda and Caleb tried to use the application that was codesigned with 74 them. When Brenda encountered errors, she emailed the researcher, and the researcher uploaded the images that she could not upload herself. In contrast, when Caleb encountered errors, he did not email the researcher; instead, the researcher heard about the event from Caleb many months after the research project had ended in an informal conversation. It made the researcher realize that if Brenda had had the same reaction as Caleb, she may have dropped the application from use, as he did, instead of persisting. Brenda’s persistence is not to be expected from all participants, nor is Caleb’s reaction uncommon in the PD literature. For example, in one PD study conducted by Groussard et al. (2018), one of the four participants—like Caleb—dropped out of the study because he claimed that the application designed for him did not function properly. The three other participants used the same application and did not report any problems. The difference between Brenda and Caleb, and between the participants in Groussard et al.’s (2018) study, illustrates that different participants respond differently to technical difficulties. Participants’ reactions to technical difficulties are variables that can either facilitate or complicate the PD process. 4.2.1.3 Taking Pictures of Paper Visual Symbols and Uploading them Despite failing to correctly upload images the first time, Brenda and her educational assistants subsequently uploaded many images successfully to the web application’s library. However, some of the uploaded images needed editing to improve their visibility. The uploading of these pictures was a critical event because it revealed a gap in the process that the researcher followed. The design process did not include any instruction for Brenda and her educational assistants on where to find pictures for the visual supports, how to prepare the pictures for uploading, and how to upload the pictures. The researcher assumed that these tasks could be 75 completed by Brenda and her educational assistants without the need for instruction or assistance. Figure 4.4 illustrates the problem. Figure 4.4 Uploaded Pictures The two pictures in Figure 4.4 needed to be cropped so they would be displayed more clearly. The researcher, who was proficient at cropping, zooming, and uploading images after decades of using computer applications, assumed that the action of uploading pictures was an one that could be easily and efficiently performed. However, the task turned out to be more complicated than he assumed: prior to uploading an image, the educational assistants had to crop it, remove its background, zoom in to improve visibility, and then re-save it on their devices before finally uploading it. For individuals who do not perform these actions frequently, the task was both overly complicated and too time consuming. In fact, Brenda had mentioned in the first interview that taking pictures of paper-based visual symbols and uploading them to digital applications was a time-consuming and complex task: “we end up taking photos of all of the picture symbols and then import them into the schedule. But then that means we’re printing, right, making the pictures, printing the pictures, taking pictures and then importing it” (Brenda, meeting 1). It is clear that Brenda and her educational assistants would have benefited from support, scaffolding, and training from the researcher with the process of uploading pictures. 76 4.2.2 Critical Events from the Collaboration with Shannon and Ferreira Three types of critical events occurred during the collaboration with Shannon and Ferreira, as described in the sections that follow. 4.2.2.1 Tensions between Control and Usability After Shannon and Ferreira created a paper prototype of the web application, the researcher developed an electronic prototype, tested it with them, and then developed a final product. The final product was fully functional but had usability issues. Shannon and Ferreira wanted the application to allow them to upload (a) an image or a video, (b) a question about the image/video in audio format, (c) multiple images representing different answers to the question, (d) an image that would appear as feedback when Alex picked the right answer, and (e) another image that would appear when Alex picked on the wrong answer. By giving Shannon and Ferreira the option to upload all of these elements, the researcher provided them with full control over what appeared on the screen and what would happen when Alex clicked on a correct or incorrect answer. However, creating more options for Shannon and Ferreira also meant increasing the possibility of errors when creating the visual supports. For example, Shannon made an error when testing the final product; she uploaded several images, input a question, but forgot to specify which answer was the correct answer, and, as a result, her visual support did not function well. The error happened in a positive environment, and Shannon, Ferreira, and the researcher were amused by it; Shannon laughed and said, “I broke it.” This event illustrated the tension between control and usability that existed throughout the study with all three cases. The researcher tried to give participants control over the format and functionality of their visual supports, but he found that giving more control often created difficulty with creating the supports and with participants having to make too many choices. 77 4.2.2.2 Creating a Quick Booklet After the researcher created and tested the final product with Shannon and Ferreira, they scheduled a time with the researcher for an observation. On the day of the observation, Shannon was still preparing the visual support when the researcher arrived. She was uploading images of different parts of the human face to the web application’s library and using them to create an electronic booklet. Despite being in a hurry, Shannon was able to upload images, audio recordings of words representing the different parts of the human body, and images to appear when Alex would choose a correct or incorrect answer. But she forgot to upload images for the choices for some questions; for example, the question in Figure 4.5 did not include images for the answer choices. Next to “MOUTH” there was supposed to be an image of a mouth, and next to “EYES” there was supposed to be an image of eyes. Figure 4.5 Question from the booklet created by Shannon Given the many items that she had to upload, it was difficult for Shannon to create a complete visual support in a short time. And, unfortunately, the web application was not created 78 for a user with time constraints, as time was not considered as an important factor when the researcher developed the application. This experience showed that educators often do not have more than a few minutes to create a visual support, and that time constraints must be taken into consideration when designing technology with them. On a related note, Brenda told the researcher that she often updated her visual supports when her students were on a break: “If the student has earned a 3-minute break, well, no matter where you are, if the iPad’s there, you can sit there and update the schedule” (Brenda, meeting 1). Shannon’s experience and Brenda’s statement are supported by a teacher in another design study who claimed, “You’re 24 hours, 7 days a week working for and with your students… It doesn’t give us a lot of opportunity to play with technology” (Penuel et al., 2007, p. 67). 4.2.2.3 Alex was More Interested in Another Application During the observation, after Shannon finished preparing the booklet, we all waited for Alex to enter the room. Previously, Shannon and Ferreira had informed the researcher that Alex was quite stressed on that day. When he entered the room with an educational assistant, he was holding a tablet that allowed him to make requests by clicking on images. For example, when he clicked on the “tickling” image, he received a tickle from the educational assistant, and when he clicked on the “hug” image, he received a hug. Shannon tried to direct Alex’s attention to the web application on the device that she was holding. She did so by asking him about the pictures and manually prompting him to select the right answers; she also provided verbal praise for a correct response. However, when he was not prompted in this manner, Alex persisted in using the application on the tablet he was holding. At the end of the session, Shannon suggested that the researcher make spaces for images on the web 79 application bigger, so that they would be more prominent. The researcher agreed with this suggestion. This critical event revealed important factors that may affect the PD process in a special education classroom. First, when designing digital applications, it is important to consider students’ motor skills as well as their cognitive skills. Second, it is helpful to inquire about the history of a particular student with technology because it could allow one to design technology to be more engaging by incorporating elements of the familiar technology. Third, it is important to design a supportive context when introducing a new technology or a new activity to a student with ASD. On the day when the application was introduced to Alex, he was distressed and Shannon and Ferreira seemed somewhat distressed as well, as it was the last day of the school year. More importantly, Alex was engaging in a highly preferred activity (i.e., making requests and receiving them immediately) when the new application was introduced to him. Thus, engagement with the application required him to stop engaging in a preferred leisure activity in order to transition to a non-preferred academic activity -- answering questions. The abrupt shift from a preferred to a non-preferred activity appeared to result in additional distress and negatively impacted his perception of and engagement with the web application. 4.2.3 Critical Events from the Collaboration with Caleb and Educational Assistants Three types of critical events occurred during the collaboration with Caleb and his educational assistants, as described in the sections that follow. 4.2.3.1 Different Reactions from Different Educational Assistants The first event that the researcher considered critical happened when he introduced the electronic prototype to the educational assistants and encountered a mixed reaction from them. Two of them expressed interest in the application and believed it would be useful, two more 80 were initially open about trying the application but expressed concerns in the second meeting, and one had a neutral attitude and was very brief in her interactions with the researcher. The following are statements from the participants that illustrated their attitudes toward the web application developed with Caleb: • Positive reaction: “I feel like this might be like a schedule where it will allow him [name of student] to understand like you have to do this before we can do this. For him, he really likes to go home, so everyday he’d be like, when, is it home time yet … So, with that, hopefully, he’ll be able to focus” (Kathryn, interview 1) • Positive reaction: “I think that’s a very good idea to design this project, so that’s why I signed my name to support your project … Kids, they like to play some digital things, you know, with very many interesting photos and you know, that’s easier, very easy” (Sam, interview 1) • Negative reaction: “I’d have to work with it a little bit to figure out where the bugs are and how to use it and learning myself because right now it’s a little bit of mumbo-jumbo in my head” (Suzanne, interview 2) • Negative reaction: “I think [my student] will be confused with it … because I'll be the one doing all that, right? He'll just be watching. You know what I mean? Like I'll be the one scheduling, picking everything” (Harry, interview 2) Despite their resistance to the web application that was developed with Caleb, Harry and Suzanne were fairly forthcoming during the one-on-one interviews. Their attitudes contrasted with Caroline’s, who responded to the researcher’s questions very briefly; for example, when the researcher asked if she wanted to add any features to the web application, she said “some music” but did not specify where to add the music, what music to add, or for what purpose. 81 Given Harry’s and Suzanne’s engagement with the project, the researcher decided to create matching games for them, based on the insights they shared in the interviews indicating that a matching game would be more useful for their students. This was a turning point in the research; up to that point, participants had the role of “design partners” (Druin, 2002). At that point, the researcher decided to consider Harry and Suzanne as “informants” (Druin, 2002). Unlike design partners, informants do not make decisions at the design level; instead, they inform the researcher about their needs, and the researcher translates their needs into actual designs. When developing matching games for Harry and Suzanne, the researcher based some of his design decisions on what had already been shared in the interviews, and others—on his own judgment about usability. This strategy worked well for Harry, who used the matching game with a student one time during the researcher’s last visit to the classroom and, according to Caleb, many times after the end of the study. 4.2.3.2 From Low Perceived Competence to Adopting the Technology and Making Suggestions As the researcher collaborated with Harry on developing a matching game, he noticed a change in Harry’s attitude. Initially, Harry referred to himself as “not a computer whiz” and stated that he had no idea about what technology would best support his student (interview 1). In the second interview, he was apologetic after giving the researcher critical feedback on the scheduling application: Researcher: Okay. Okay, thank you very much. Harry: I'm sorry I'm not a super educated … [This utterance was made as the researcher was stopping the recording; the utterance is heard in the audio transcript but is cut. From 82 what the researcher remembers, the complete utterance was “I’m not super educated like you guys”] (Harry, interview 2) Despite his perceived low competence with technology, Harry did provide constructive suggestions in the second interview, which contained statements that demonstrated both his low self-efficacy for technology and his interest in technology. During the researcher’s last visit to Caleb’s classroom, he delivered the matching game to Harry and showed him how to use it. As Harry was using it with a student, the researcher was interviewing other educational assistants. Harry told the researcher that the student liked the matching game and then suggested that they add more questions to it. The researcher and Harry also thought that it would be a good idea to add Harry’s voice to the game, so that the student would receive feedback in Harry’s voice. The researcher recorded Harry’s voice and added it to the new version of the matching game. He then sent link to Caleb and asked him to forward it to Harry. This event showed that, for some participants, participation in a PD project could support them in developing a higher self-efficacy for technology and a more open mindset about using and creating technology. Harry’s experience showed that certain factors that have been shown to hinder technology adoption, such as low self-efficacy, low perceived usefulness, and low perceived ease of use of a technology, are not static; they can dynamically change during the PD process. This situation highlights the importance of planning a PD process that does not just deliver technology to solve challenges but also supports participants to develop a more confident and open attitude towards technology. 4.2.3.3 Kathryn Tried Creating a Schedule, But the Web Application Failed Kathryn was highly motivated about the project; this was clear in her comments during the first interview. During the researcher’s final visit to Caleb’s classroom, he introduced the 83 final version of the web application to Kathryn. She decided to create a schedule immediately; she took many pictures with her iPad’s camera, uploaded them to the web application’s library, and created a schedule. However, the schedule did not appear in the web application’s library of schedules, so it seemed that nothing had been created. The problem of uploading pictures from an iPad to a web application is a common issue and has been reported by many users on the internet1. The researcher asked Kathryn, Caleb, and Sam, to send him pictures for the app and said that he would upload them to the web application’s library from his computer. He also suggested that they send him some existing, paper-based activity schedules, so that he could search for relevant images and upload them himself to the web application’s library. The researcher only heard back from Sam, whose email was clear and specific and provided enough information for the researcher to create a complete visual activity schedule. The following was the activity schedule sent by Sam via email: Math (30-minute plan): Matching numbers with pictures (objects numbers) - 5 minutes Addition exercises (total within 20 at present) - 15 minutes Subtraction (basic) - 10 minutes English (30-minute plan): Identify and write alphabets -5 minutes Write words - 10 minutes Write sentences - 10 minutes Reading - 5 minutes 1 https://discussions.apple.com/thread/6540665 84 (Excerpt from email received from Sam) Unfortunately, by the time the researcher heard from Sam, the project had already been terminated because Caleb would not be teaching in the classroom in the following academic year, and Sam had been transferred to another classroom. As for Kathryn, despite her high motivation and her high perceived usefulness of the web application, she did not follow up with the researcher. Her perceived usefulness of the web application may have diminished after finding out that she had to upload pictures from a computer instead of from an iPad camera. Kathryn’s experience illustrates the impact that failure of technology can have on the engagement of research participants. The study conducted by Fage et al. (2016) exemplified a PD process that reduced technical failures. Fage et al.’s (2016) research was similar to the current study because it allowed educators to create visual supports through a codesign process. However, Fage et al. (2016) were somewhat more successful in fulfilling their goals because they created ready-to-use visual supports for their participants, and participants used them easily. 4.3 Common Themes Across Cases After coding the data, the researcher found many regularities across the cases and deduced common themes across cases. Some of the themes were ubiquitous across the three cases while others were common for two of the three cases. Many of the themes intersected with themes reported in other PD studies. One common theme that emerged across the three cases and had previously surfaced in other PD studies was the mismatch between the planned outcomes and the actual outcomes. Other themes that emerged from the analysis were related to the special education context and the methods used in the PD process. 85 4.3.1 Mismatch between Planned Outcomes and Actual Outcomes In all three cases, there was a mismatch between the actual outcomes and the planned outcomes. Brenda’s objective was to design an application that she could use to create visual schedules that would include images and videos. She ended up using the application to create only two visual supports: one consisted of a single image to signal the availability of a break, and the other consisted of two images that represented a break and “choice time.” When the researcher asked Brenda why she did not create more schedules, she said that on many occasions the application could not be accessed because of poor internet connectivity at the school, and, as a result, she decided not to use the application for creating and displaying schedules. Shannon’s and Ferreira’s objective was to create a digital application that would engage a student, Alex, during language lessons. Although Alex did not seem initially to be engaged by the application, his reaction helped to identify improvements that could be introduced. Caleb’s objective was to design a scheduling application that would allow him to create a schedule for all educational assistants to follow and that could be personalized by each educational assistant to fit individual student needs. Kathryn, one of the educational assistants, tried to use the application by uploading pictures from her iPad’s camera, and the application did not function. Several secondary objectives also emerged over the course of the project, including the development of two matching games, knowledge about the special education context and variables within the context that impact the PD process, recommendations for conducting PD in the special education classroom, and possibilities of future collaborations with the research participants. It is not uncommon for PD projects to fail in reaching their primary objective and to produce secondary outcomes instead. As Lundmark (2017) mentioned when describing her PD project, “the project at hand could be viewed as a failure in many respects. It has absorbed a lot 86 of work hours, a lot of time, and faced several challenges and setbacks, but it has also led to a number of unexpected outcomes” (Lundmark, 2017, pp. 92-93). Lundmark planned a PD process to support youth in accessing counseling services, but her PD project was also hampered by many factors, and she did not achieve the desired outcome. However, several secondary outcomes were produced, including the identity changes experienced by the participants. Another project that reported failure to reach primary outcomes and the emergence of secondary outcomes was Righi et al.’s (2018) project with participants at a seniors home. In this case, the objective was to codesign technology that would support participants in outdoor explorations, but the designed application was dropped from use. However, during the study, participants created a chat group to ask each other questions about technology and provide answers. The chat group, though it was unrelated to the study’s primary objectives, was consistently used by participants and continued to be used even after the study ended. The chat group became the “project’s legacy,” according to Righi et al. (2018). The abundance of PD studies that fail to reach their primary objectives does not mean that failure is inevitable. In fact, many PD studies have successfully reached their goals (e.g., Fage et al., 2016; Hirano et al., 2010). However, given the many reported failures, it is important for researchers in PD to acknowledge the possibility of failure as they plan for their study and to communicate this possibility to their participants. 4.3.2 Mismatch Between the Technology Created and the Technology Used by Participants One common challenge that surfaced across cases was the mismatch between the created technology and the technology that participants were already using. This mismatch made it difficult for participants to integrate the new applications into their practice, especially in 87 Brenda’s and Caleb’s classrooms. The researcher created web applications for participants; however, participants had already formed a strong habit of using iOS applications. Web applications and iOS applications differ in several ways. First, access to web applications usually requires internet connectivity, while iOS applications can be accessed offline once they are downloaded onto a device. In theory, web applications can also be accessed offline, but that requires changing some of the settings on devices, a practice that participants may find complicated and time-consuming. Second, the process of uploading pictures from a tablet to a web application was not reliable. When trying to upload pictures from their tablets to the web applications, several participants encountered technical errors, and when the researcher suggested that they upload the pictures from their computers instead , they did not seem motivated to do so. Brenda explained that uploading the pictures from the desktop computer in her classroom would break some of the habits and routines that she and her educational assistants had already developed and that functioned well for them. She clarified that, because of time constraints, they usually created visual supports during short breaks between tasks; thus, having to move to the desktop computer, turn it on, open the browser, upload the pictures, and then display the visual support on tablets, would reduce the efficiency of the process. Brenda and her educational assistants eventually managed to upload pictures to their web application directly from their tablets successfully; however, persistent failures of the application likely affected its perceived usefulness. Kathryn, one of the participants who was highly motivated at the beginning of the project and expressed high perceived usefulness of the web application that the researcher created, was also unable to create a visual support with the application. Kathryn walked around the classroom and the hallway and took pictures of different spaces and artefacts with her iPad’s built-in camera. She then uploaded the pictures to the web 88 application and followed all steps to create the schedule, only to find out that the pictures were not uploaded to the application. This was not just a technical failure; it was also a failure of the routine that she had hoped to follow. She had imagined taking pictures with the built-in camera and instantly creating the visual supports; however, incompatibility between the web application and the iOS the device forced her to give up the routine that she had envisioned. The researcher suggested that she upload the pictures from a desktop computer, then create the support and display it on her iPad, or send him the pictures and he would upload them to the web application’s library. This process, unfortunately, drastically contrasted the process that she had envisioned, to the point that Kathryn was no longer interested in using the application. The mismatch between the applications that were developed and the habits and expectations of participants created a gap in the research process. While the participants enjoyed creating the paper prototype and giving feedback on the electronic prototypes, the process of introducing the web application was less positive and included technical failures. The researcher had not previously observed the participants’ habits surrounding technology use and had not specifically considered the scenarios that they imagined in relation to the new web applications, nor did he recognize potential incompatibilities between the web applications and the available infrastructure (i.e., internet connectivity and iOS devices in common use). The researcher would have benefited from scheduling observations of the participants and their technology habits in the classroom before working on the final product. Having a strong understanding of the technology habits of participants and of how participants imagined using the applications would have assisted the researcher in determining whether a web application could be incorporated into participants’ practices. In understanding this in advance, the researcher might have decide to convert the web application to an iOS application prior to being introduced into practice. 89 4.3.3 The Importance of Time Constraints for Educators Time was an issue for almost all participants in the three cases. This was reflected in some of the participants’ requests and feedback on the prototypes, in the observations of the researcher, in the difficulties with scheduling meetings, and in reports from other PD studies. 4.3.3.1 Brenda The theme of time surfaced with Brenda when the researcher scheduled meetings with her; there were many times when she was too busy to meet. At the beginning of the study’s second year, Brenda told the researcher that she could not meet because she was busy training new educational assistants. The theme of time constraints also emerged when Brenda spoke about the time it took to upload images to the visual supports applications that she used. Time was also a factor in Brenda’s design requests; in her critique of one of the features in the electronic prototype she noted, “But then I feel like that won't be instantaneous enough when we need” (Brenda, meeting 4). 4.3.3.2 Shannon and Ferreira Scheduling meetings with Shannon and Ferreira was not difficult. However, the theme of time surfaced when the researcher observed Shannon create the visual support prior to the observation. The issue of time also came up when Ferreira expressed her concern about the difficulty of uploading pictures to their application’s library. Shannon was thinking about transferring pictures from another application into the new application. The process would have required them to download the images first and then upload them to the new application, a process that would take a long time. Ferreira reacted by saying: “Nice. Uh, but it’s going to be hard to transfer them [the images]” (meeting 3). It is not clear whether time was one of the factors that led Ferreira to state that the process of transferring images would be difficult. 90 Ferreira’s statement presented an opportunity for the researcher to inquire more about her perceived difficulties and her thoughts on the issue of time constraints; unfortunately, the researcher missed this opportunity. 4.3.3.3 Caleb and Educational Assistants Caleb had a busy schedule during the study. Scheduling meetings with Caleb was sometimes difficult, especially at the time of the year when report cards were due. In none of the two recorded interviews with Caleb or his educational assistants was time explicitly mentioned. However, through the researcher’s interactions with Caleb and his educational assistants, it was palpable that the time they had to participate in the study was limited. For example, interviews with the educational assistants were conducted during class time and could not last longer than 15 minutes because they had to get back to the students. Furthermore, one possible cause for dropping the scheduling application from use was that educational assistants did not have enough time to explore the app, as the following statement by Suzanne demonstrates: “Like, if I could play with it at home a little bit or on my time at school, you know just play around with it to figure out how to use it” (Suzanne, interview 2). Despite expressing resistance for using the application, Suzanne also showed she was willing to try it if she could get the time to play with it. However, she would either have to find that time at home, which would mean more unpaid time for her—or at school, which would mean that she would have to find someone else to supervise or work with her student while she played with the application. The issue of educators’ lack of time was noted by Könings et al. (2007) in a survey study with teachers and students to inquire about their impressions of PD. Teachers who had positive opinions on PD preferred to meet fairly infrequently (i.e., three times a year), while teachers with negative opinions of PD stated that PD would take too much of their free time. In both cases time 91 was a constraint: while teachers with positive opinions wanted to meet less frequently to avoid a greater commitment, those with negative opinions were not interested in PD precisely because of the high time costs they anticipated: Teachers emphasized that participatory design costs time. Investing even more time in school was not a welcome idea, because students and teachers already have a busy schedule. A solution, already put forward, includes scheduling time for participatory design during regular school hours. (Könings et al., 2007, p. 459) Given the importance of time for educators and their busy schedules, PD researchers working with educators should expect scheduling difficulties and a preference among participants for a low frequency of meetings. 4.3.4 Turnover among Educational assistants in the Special Education Classroom In both Caleb’s and Brenda’s classrooms, there was considerable turnover among educational assistants. In Brenda’s classroom, not all of the educational assistants that the researcher saw in the first year of the study remained with Brenda in the second year. Moreover, Brenda had explicitly stated that she had to train new educational assistants. In Caleb’s classroom, many of them either left the school or were transferred to a different classroom. When the objective of a PD study is to design a technology for a classroom and the educational assistants in the classroom are there on short-term appointments, it might negatively affect their interest and the perceived usefulness of the project. In a retrospective review of the literature on visual supports and PD, the researcher found only one reference to turnover among educational assistants and its impact. In an article on educators’ perspectives on the use of visual supports, one of the educators stated, “Staff changes are a big issue that get in the way, because some staff might be using it [the visual supports] in 92 one way, and then other staff come in and they’re not aware of the way other people are using it” (Donato, et al., 2014, p. 119). This statement supports some of the challenges that the current study faced, namely, that turnover led to difficulties in scheduling meetings, which made it difficult for the researcher to engage new educational assistants who joined Caleb and Brenda’s classroom in the second year of the study. Turnover also increased the workload of classroom teachers, and this is one of the challenges that PD can help overcome: codesigning artefacts and systems with special education teachers that would support them in training new educational assistants can help reduce their stress and heavy workload. 4.3.5 It is Not Always about Technology In the three PD cases that formed this study, the objective was to codesign a technology/web application. However, in the cases of Brenda, on the one hand, and Shannon and Ferreira, on the other, some of the challenges they reported about their work with visual supports did not necessarily require the creation of a new technology. For example, Brenda had an issue with copyright. Although she did not specifically use the word “copyright,” she reported that she was using multiple digital and nondigital tools to create visual supports, and each tool came with its own copyrighted sets of visual symbols. She created communication symbols, which consisted of single images used to communicate instructions, with a commercially produced set of image symbols. Simultaneously, she created her visual schedules with a digital application that came with its own library of images. One challenge was that the images in the digital application’s library did not coincide with the images she used for communication. This created a problem with consistency; her students were used to one type of images, and then they were seeing other types of images on the visual schedules: “So, am I going to take a student who spent five years learning to communicate using pictures and now start showing them a complete new 93 library of a totally different vocabulary [of images] Some kids can handle that; lots of kids can’t” (Brenda, meeting 1). Brenda’s problem was partly caused by issues of copyright— that is, the digital application that she used could not include commercially produced images unless the creators of the digital application paid for licenses that would allow them to use the commercially produced images. Because most sets of image symbols are commercially produced, it is difficult to find the same set of image symbols preinstalled on multiple digital applications. The solution for this challenge does not necessarily concern the design of a new technology. Instead, the solution would be to support Brenda in designing her own unified library of visual symbols and uploading it to the different applications that she uses. Several sources could be used to support Brenda in building her library: open-source visual symbols libraries (https://www.opensymbols.org/), her pre-existing image symbols, and pictures of objects that are meaningful to her own individual students (the student’s tools, the student’s parents/guardians, and other subjects and objects that the student regularly interacts with). In that case, the objective of the research would be to codesign an image library with Brenda, not a new technology, and to help Brenda upload the image library to the different applications that she uses. Shannon and Ferreira reported a different limitation with the technology that their student, Alex, was using. The available technology allowed Alex to make requests by clicking on icons, but they wanted the communication with Alex to go beyond making requests. They wanted him to be able to answer questions and make comments: So we have like different symbols… and then various folders like with classroom activities, places, home activities, and he gets to choose whatever he wants to do throughout the day. So it's just for him to request. But yeah, we want him to do 94 something else like moving to the next step, making comments, answering questions. (Shannon, meeting 1) While technology could help Shannon and Ferreira move to the next step and have the student make comments and answer questions, technology by itself is likely not enough. The digital application that Alex was using to make requests provided him with a rewarding experience (i.e., when he clicked on an image, he got the requested item or activity). What made requests appealing to him is not necessarily the medium used (paper vs. digital), but the rewarding experience itself. The researcher missed the importance of considering how to make the process of answering questions and making comments rewarding for Alex, focusing instead mostly on developing the web application. Yet, Alex’s negative reaction to the web application was a reminder that the issue may not have been a technology issue. Unlike the cases of Brenda and of Shannon and Ferreira, creating a web application seemed the most suitable solution for Caleb’s challenges. Caleb wanted a system that would allow him to show a common schedule for all his students and allow educational assistants to personalize the schedule based on their student’s developmental level. For example, he envisioned system in which, when the whole class was working on English, different students could be working on a variety of English activities; and when the class transitioned to Math, different students would be working on different math activities. Implementing such a system as a paper-based system would have created more work for Caleb: he and his educational assistants would have to print a schedule to show to the whole class as a well as individual schedules for each student. In contrast, a digital solution could make the process more efficient: Caleb would create a common schedule and the separate copies would be automatically created for each 95 educational assistant, and each educational assistant would log in to edit their own copy of the schedule. In two of the three cases, the challenges reported by participants could have been resolved without the need for the development of new technologies: in Brenda’s case, a set of visual symbols that she could use in all of her applications may have been sufficient, and, in Shannon and Ferreira’s case, a system for providing incentives to Alex to enjoy making comments and answering questions as much as he enjoyed making requests would have helped resolve the problem with Alex’s engagement. Only Caleb’s case seemed to require a new technology for the reported challenge to be resolved. 4.3.6 Hesitation with the Paper Prototype Activity In this study, participants were asked to draw on paper their vision of what the web applications would look like and how they would function. They drew, with the researcher, the paper prototypes from scratch, and they were not provided with any templates. For all participants, the researcher provided color pencils, sticky notes and other craft material, but these materials were rarely used. Brenda drew her prototype with a pen at the beginning of the second meeting; she had previously discussed the project with her educational assistants, and drawing the prototype did not take long as she had already decided what she wanted. Shannon and Ferreira drew the prototype on the first meeting by using their pens. Caleb’s prototype was created by the researcher as Caleb shared his ideas. It appeared that having participants create a paper prototype was not a straightforward task and some participants expressed hesitation, as illustrated in the following excerpts: Researcher: So now you have this. This [the white paper] is your screen. Caleb: Yeah. 96 Researcher: And these are colours, and you could actually – [gives Caleb some colored pencils and other material] Caleb: I am not an artist (laughter) Researcher: Yeah. I'm not an artist either (laughter). (Caleb, meeting 1) It's difficult for me to picture all this because I'm not used to drawing interfaces. So it's hard. (Ferreira, meeting 1) Brenda did not express hesitation when she drew her first prototype; however, in the third meeting, when the researcher showed her an electronic prototype and a paper representation of the electronic prototype, Brenda preferred not to write or draw on the paper representation of the prototype. She was worried that she might “mess it up”: “You write, I have terrible writing. I don’t want to mess up your beautiful… (laughter)” (Brenda, meeting 3). The reluctance of participants to engage in the paper prototype activity has also been reported by other researchers. For example, a textbook on interaction design reported the following issue: “Low-fidelity prototyping often relies on hand-drawn sketches, and many people find it difficult to engage in this activity because they are inhibited about the quality of their drawing” (Preece et al., 2011, p. 389). One suggestion made in the textbook was to provide participants with premade icons, sketched by the researcher, and ask them to use the premade icons when making their paper prototypes. This converts the activity from a sketching activity to a design activity and has been referred to as “prototyping with index cards” (Preece et al., 2011, p. 390). 97 Other researchers came up with additional ways of supporting the paper prototype activity. For example, Benton and Johnson (2014) used templates to provide scaffolding to their participants as they were designing; and Groussard et al. (2018) designed different interfaces on paper and asked participants to rate the interfaces and suggest changes to those that received lower rating. It is not entirely clear whether participants in the current study hesitated because of their perceived design skills (“I have terrible writing,” “I am not an artist”), or because of a lack of interest in the activity, or both. Both the educators’ perceived usefulness of the paper prototype activity and their perceived design skills should be taken into consideration when planning the activity. 4.4 Eliciting Feedback On many occasions, when participants asked for changes, they prefaced their requests by checking in with the researcher whether it was too much work for him: • “But maybe it only shows one day at a time, but the option of the whole week will be there. Will it be too complicated?” (Caleb, meeting 1) • “I mean, the fancier the better. How much work is that for you to do?” (Brenda, meeting 3) • “Is that hard to change, that close to submit?” (Brenda, meeting 4) Participants’ concerns about not creating additional work for the researcher, along with the positive rapport between the participants and the researcher, may have actually hindered the process of eliciting critical feedback on the prototypes and the final products. When the researcher asked the participants about their impressions of the project, he received positive answers in every case; however, those positive answers could have been due to the rapport and to the social desirability effect (Könings et al., 2007). One alternative way of evaluating a PD 98 project from the perspective of participants would be to articulate, with participants, clear criteria for success at the beginning of the project and evaluate the project based on these criteria. 99 Chapter 5: Interpretation, Implications, and Conclusions Interpretation is the phase when the researcher “takes the leap” (Wolcott, 1994) and transcends the findings by discussing their research implications and their theoretical significance. In this chapter, the researcher articulates research implications on conducting PD in the special education classroom; turns to his experiences as a PD researcher; and narrates the dilemmas, difficulties, unforeseen events, and lessons learned. To articulate the limitations of the study and evaluate its methodological rigor, the researcher used Frauenberger’s (2015) tool-to-think approach to assess the study’s internal reliability. 5.1 Research Implications for Conducting PD in the Special Education Classroom Based on the study findings, the researcher articulated several research recommendations for conducting PD in the special education classroom. The primary audience for these recommendations is PD researchers who plan to codesign visual support systems with educators; however, most of the recommendations also apply to the design of other technology systems in both special education and mainstream classrooms. For each general principle, a set of specific practices that exemplify how to apply the general principle are presented. The recommendations are presented in Table 5.1. The table is divided to four sections; each section corresponds to a phase in the participatory design process and includes recommendations for that phase. 100 Table 5.1 Recommendations for conducting PD Planning for and Introducing the Project General Principle Specific Practices Develop a plan for continuously assessing the internal reliability of the project Ensure alignment between values, knowledge, stakeholders, and outcomes from the beginning Determine if this is technical action research, practical action research, or emancipatory action research Identify and manage participant expectations of the project Explain to participants the iterative nature of participatory design Field Work General Principle Specific Practices Assess the visual support systems and other tech-based systems currently used by participants Study the current technology infrastructure at the research site and identify any constraints related to it Observe participants’ habits around technology use Examine the extent of staff turnover in the setting (e.g., transferring between classrooms, changing careers) Ask participants about their long-term plans Designing with Participants General Principle Specific Practices Support participants to translate their ideas and knowledge into a design prototype Give participants a choice of sketching an interface of the application from scratch or starting with an existing template 101 Give participants choice over their role and time commitment during the design process Discuss participants’ role with them and give them choice: do they want to be design partners, informants, testers, users, or some combination of these? (Druin, 2002) Create a design process that contributes positively to participants’ perceptions of technology and their self-efficacy for technology Create prototypes that minimize user error Conduct testing on devices and infrastructure at the research site prior to introducing prototypes to participants Introducing the New/Modified System General Principle Specific Practices Develop with participants a strategy for introducing the new system to students Decide when the system should be introduced to minimize disruption (e.g., avoid change from a preferred activity; choose a time when the student is calm and receptive) Decide who should introduce the system to minimize disruption (e.g., have a familiar, preferred person introduce the system) Support long-term and sustainable use of the system Create training material for current and future stakeholders 5.1.1 Planning for and Introducing the Project Two principles are presented here for researchers to consider as they prepare for a PD project. 102 5.1.1.1 Develop a plan for continuously assessing the internal reliability of the project Unexpected events are likely to happen in participatory design, and decisions often need to be made on the spot. Concern over the rigor of the research process is common among participatory designers because of how “messy” the process can become (Frauenberger et al., 2015, p. 93). To ensure the project rigor at all stages, researchers need a methodological tool that allows them to assess the consistency between decisions made during the project and the theoretical and methodological underpinnings of the project (i.e., internal reliability). An example of such a tool is Frauenberger et al.’s (2015) tool-to-think framework, which the researcher used retrospectively to assess the internal reliability of the current study, and which can also be used at earlier stages of PD projects to continuously assess internal reliability. Frauenberger et al.’s (2015) framework consists of four lenses: knowledge, values, stakeholders, and outcomes. Assessing the alignment and misalignment between these lenses helps researchers maintain the rigor of their project and the trustworthiness of the knowledge they produce. For example, as argued previously in this dissertation, if a research project is based on the values of empowerment and democratization, yet the outcome is a technology that reproduces power inequities between stakeholders, then the values and outcomes lenses are misaligned. When such a misalignment occurs, the researcher and participants might decide to drop the technology from use. In this case, dropping the technology from use becomes a defendable action because it supports the researcher and participants in realigning the outcomes of the research project with its original values. 103 Before starting a study, it is important for researchers to assess the alignment between the values that inform the research process, the knowledge that they plan to produce, the potential impact of the project on participants and stakeholders, and the outcomes they plan to deliver. In addition, it is important for researchers to integrate assessment of the alignment between the four lenses into the research process itself and plan to regularly reflect on the alignment between the four lenses. In the event of a misalignment, the researcher should also be prepared to modify the research process in order to restore alignment between the four lenses. A second framework that PD researchers can use to ensure internal reliability is Grundy’s (1982) three categories of action research: technical, practical, and emancipatory. This framework can be particularly useful when managing disagreements or differences in perspectives between researchers and participants. When such disagreements and differences occur, the researcher will have to decide whether to negotiate with participants or follow the participants’ preferences despite the disagreement. Such decisions must be made based on a theoretical rationale to ensure the rigor of the project; otherwise, decisions will be made arbitrarily, and the replicability of the study will be negatively affected. Grundy’s (1982) differentiation between technical, practical, and emancipatory action research can guide researchers around making decisions when encountering differences and disagreements with their participants. In a technical action research project, the researcher is the expert, and their objective is to support participants in developing evidence-based practices. Hence, when disagreements occur, it is expected that the researcher will negotiate and advocate for their perspective. On the other hand, in practical action and emancipatory action research, it 104 is expected that the researcher follows the preferences of participants, even when the researcher disagrees with these preferences. By positioning their research within the technical/practical/emancipatory research framework, researchers can go into the field with a clear awareness of their role in the design process, which will allow them to be better prepared for managing differences and disagreements. 5.1.1.2 Identify and Manage Participant Expectations of the Project It is important to manage the expectations of participants and other stakeholders as early in the project as possible. Educators working with students with special needs may expect the codesigned technology to help solve practical problems and to reduce some of the stressors that they encounter in their educational work. Unfortunately, making a promise but failing to deliver an anticipated outcome may leave participants and stakeholders disappointed. In a retrospective reflection of the work they did with parents of children with special needs, Frauenberger et al. (2011) argued that it is important to manage expectations, and not to overcommit: “overcommitting […] is bound to result in great disappointment that is less easily made up for than with other user groups. From the perspective of the parents, there may be a greater investment in the research from the start, and higher expectations, as the software could have a potential impact on their child’s well-being in a way that is perhaps not so direct in the case of typically developing children. From a practical perspective, a failure to manage the relationship in an appropriately caring and sensitive manner is likely to lead to a denial of access to an already restricted population” (pp. 24-25) Frauenberger et al. focused on designing technology with parents of children with special needs; however, these risks also apply to designing with educators. Caring for and 105 teaching students with special needs is very often a demanding and distressing process, and the promise of designing a technology that reduces the time demands and stressors may lead to both greater expectations from the research process. Consequently, sub-optimal results may lead to feelings of disappointment that are difficult to mitigate. To manage the expectations of participants and avoid disappointments that are difficult to mitigate, it is important to explain to participants the iterative nature of participatory design and of design in general. A new technology may unexpectedly fail at some point, hence the importance of continuous refinement through a feedback cycle that includes participants giving feedback and designers adjusting the technology. 5.1.2 Field Work Conducting field work before starting the design process can help PD researchers better understand the context in which the new system will be introduced. In the current study, the researcher did not carry any observations prior to starting to design with his participants, and this proved to be a limitation. As narrated in the descriptions of the three cases, several challenges surfaced when participants attempted to integrate the codesigned applications into the teaching setting. The challenges were secondary to contextual factors that were not discussed during the design process. Thus, when challenges surfaced, the researcher did not have sufficient knowledge of the context and consequently was unable to support participants or motivate them to persist in using the new applications. For example, he was unable to create visual supports for participants because he did not know the study routines that participants designed for their students and, as a result, he did not know what type of supports would be useful for them. In the special education context, conducting field work in advance can help researchers and participants save time during the design phase: the more the researcher knows about the 106 context, the fewer questions they will have for educators to answer. It is still important to examine educators’ perspectives on their own contexts; however, given educators’ limited time, studying the context in advance through field work can make the design process more efficient and less demanding for educators. Conducting field work prior to design, which includes observing in classrooms and interacting with participants as they work, can support researchers in gathering crucial information and using that information to inform design decisions. This subsection presents two general principles for PD researchers to follow while conducting field work in a special education classroom. 5.1.2.1 Observe the visual support systems and technologies used by participants Observing the systems that educators use to create their visual supports and the time they allocate for creating the supports helps researchers to understand educators’ habits around technology use. It also helps researchers to predict the constraints on the new system that will be created and assess the compatibility between the new system and participants’ context. As educators continuously use a system, they develop habits around technology use, so new technologies need to be compatible with these habits. A technology that requires educators to change their habits is less likely to be adopted. In this study, the researcher developed web applications with participants; however, all of the participants were using iOS applications, not web applications, in their educational work. Despite the differences between web applications and iOS applications, participants were willing to try using web applications. However, challenges surfaced when using the web applications required them to change existing habits because they were required to upload images from their desktop computers and then display them on their tablets, rather than both uploading and displaying them on the tablets from the outset. 107 In addition to compatibility with participants’ habits, the new system must be compatible with the available technology infrastructure. One common constraint that the researcher witnessed at both sites where the study was conducted was low WIFI connectivity. Another constraint that the researcher observed in Caleb’s classroom was that some students used outdated tablet devices. This constraint is common and is not an anomaly. Outdated devices are the reality in educational and in many non-educational contexts, operating systems are continuously enhanced, and it is difficult and impractical to be continuously updating devices and downloading the newest versions of operating systems. The constraints observed by the researcher confirmed reports in the literature on PD with educators. For example, Stacy et al. (2017) introduced online math web applications in an after-school program for elementary school students and found that youth benefited from the web applications; however, WIFI connectivity was too slow, which affected the possibility of long-term use. The authors who conducted the study recommended investing in infrastructure. An additional recommendation could be designing interventions that are closely aligned with the existing infrastructure. Common characteristics of school technology infrastructure include low-WIFI connectivity, outdated devices, and different types of devices used by different students. A PD intervention that aims to facilitate long term change and facilitate the integration of new technologies into educators’ practice should be planned based on the available infrastructure. For example, one gap in the current study is that the researcher was testing the application on his own up-to-date devices. In many cases, the applications did not work on some of less current devices in the classrooms. One modification to the design process would involve planning sessions in which the researcher tests the applications on the classroom’s devices prior to introducing applications to the educators. 108 Finally, field work should also include observations of pedagogical practices. Very often, the impact of the systems that PD researchers design depends on factors that are extrinsic to the systems themselves and related to participants’ pedagogical practices. In the case of visual supports, several factors have been identified as affecting effectiveness. For example, as outlined in Chapter 2, the type of instructional prompting that is used when presenting visual supports affects their pedagogical impact, with graduated guidance recommended over verbal prompting (McClannahan & Krantz, 2010). However, in the current study, the researcher observed that verbal prompting was used more often than graduated guidance. It is important to note that the number of observations in this study were quite limited, so the overall extent to which verbal prompting was used is not known. Yet, it appears that the approach that is most recommended in the research literature, graduated guidance, was not used consistently in Brenda’s and Shannon/Ferriera’s settings. Of course, the type of prompting that educators use is a variable that is outside of the researcher’s control and, as Penuel et al. (2007) noted, educators may feel uncomfortable when their practices are scrutinized. However, it is important to observe educators’ prompting practices and to ask them about the prompting strategies they use in advance. 5.1.2.2 Examine the extent of staff turnover in the context (transferring between classrooms, changing careers) In both Caleb’s and Brenda’s classrooms, there was considerable turnover of educational assistants. This phenomenon has several explanations: first, educational assistants are sometimes hired on short-term contracts, and second, many educational assistants seek to eventually change careers. Some of them plan to become teachers; for example, Brenda herself was an educational 109 assistant before becoming a teacher, and Kathryn, one of the educational assistants in Caleb’s classroom, terminated employment to enroll in a teacher education program. The turnover among educational assistants poses a challenge for PD researchers. If the aim of PD is to codesign classroom technologies with teachers and educational assistants, and if educational assistants are not likely to use the technologies when they leave the classroom, their motivation to participate in the project may be low. In addition, if the objective of a PD study is to empower participants, educational assistants may not feel empowered when they participate in the design of technology that they will not use in the long term. Classrooms, like other contexts, are in continuous change; while some of their properties tend to be stable and remain constant across time, other properties change at a relatively fast rate. This fact emphasizes the importance of examining, prior to the start of a PD project, which elements are likely to be stable and which elements are likely to change during and after the project is completed. To assess in advance the extent of staff turnover, researchers can ask participants about their long-term goals. Also, researchers can inquire about the hiring policies of the school districts where they conduct research, as these hiring policies are often publicly available. Public job postings can be useful in determining whether potential research sites hire educational assistants on short-term or long-term contracts. 5.1.3 Designing with Participants In the special education classroom, as well as in the mainstream classroom, time is a scarce resource. The researcher’s observations during the study suggest that educators’ schedules are extremely busy and they have multiple responsibilities. In addition to the responsibility of teaching, educators also have to prepare teaching resources; while they do get paid for teaching, they are rarely paid for the time they put into creating resources. Educators also have to interact 110 with parents on a regular basis and the time required for this is also uncompensated, for the most part. The researcher’s impressions support reports from the literature on educators’ significant time constraints and the scarcity of support they receive (Donato et al., 2014; et al., 2007). Consequently, it is important to consider educators’ limited time when planning PD studies with them. Three general principles are presented here to support researchers in conducting time efficient design processes and in give participants control over their involvement in the design process. 5.1.3.1 Support participants to translate their ideas and knowledge into a design prototype “I am not an artist,” was Caleb’s immediate reaction when the researcher put a blank piece of paper in front of him and asked him to draw an interface. Shannon, Ferreira, and Brenda were also hesitant to sketch their ideas on paper. A blank paper creates a free opportunity for participants to express their ideas, but a blank paper can also cause confusion. To reduce confusion and support the design process, researchers can give participants control over the paper prototyping process by giving them options: for example, participants can start with a blank paper or with preexistent templates. The paper prototyping process can include sketching combined with a selection of elements from various example templates that can then be merged. These templates can be developed by researchers in advance, based on field work observations. Another rationale for including template scaffolding in the paper prototype activity is related to participants’ limited expertise with design. Most participants in a PD process will have neither strong expertise in design nor strong expertise in expressing their ideas through design. From knowledge to design is a substantial leap, and participants will need support from design experts in the process of knowledge-to-design translation. 111 112 5.1.3.2 Give participants choice over their role in the design process and the required time commitment Initially, PD was developed to give workers control over the technologies that mediate the work process. In the case of educators, one major source of fatigue and frustration in their work process is time burden. Hence, technology designed with educators should, ideally, support them to transform their work process into one that is a less time consuming. This priority should inform the planning of PD projects with educators and should guide PD researchers to make decisions about time-related dilemmas that they may face during the project. One dilemma that the researcher continuously faced in this project was related to making decisions for participants, which meant giving them less control over the interfaces of their applications. For example, the interface of the matching game created for Harry was designed by the researcher based on two interviews with him, and Harry had little control over the layout, colors, and fonts – all features that he did not find important to control. He had only 15 minutes available for each interview and he was working with a student who required continuous support. Consequently, the researcher decided to make design decisions on behalf of Harry and Harry eventually used and enjoyed using the matching game that was created for his student. On the other hand, the scheduling application designed with Caleb took much longer to develop because it gave full control of the interface to Caleb; however, it also required Caleb to meet with the researcher several times, and these meetings were difficult to arrange because of Caleb’s busy schedule. 5.1.3.3 Create a Design Process that Positively Contributes to Participants’ Perceptions of Technology and their Self-Efficacy for Technology Research suggests that educators, especially those working in elementary schools, tend to have a low self-efficacy for technology (Ertmer, 1999; Shapka & Ferrari, 2003; Wang et al., 113 2004). This has certain implications for their involvement in a PD process. Most importantly, participation in a PD study should not contribute to reducing educators’ self-efficacy. Frequent failures in the process and numerous difficulties may further reduce educators’ trust in technology. In addition to co-designing artefacts with educators, researchers are in charge of designing experiences for their participants, and when participants have low self-efficacy, the priority must be to avoid experiences that further reduce technology self-efficacy. One element of PD that could negatively affect educators’ self-efficacy for technology is user errors. When educators test an application and the application does not function, two different attributions can be made: (1) educators may attribute the failure to themselves because they made the mistake or (2) educators may attribute the failure to the technology. As Norman (1988) argued, an application’s failure that follows a user’s input is not necessarily a user mistake; rather, it may be the result of a bad design. Interfaces that are well designed control for users’ mistake; for example, if a user misses a step in a process, a well-designed application will not let the user move to the next step. An important element of PD with educators, as with any population that may have low-self efficacy with technology, is integrating error control into the design process, a step that should be completed by both researchers and designers. 5.1.4 Introducing the New System Insistence on sameness is a primary characteristic of individuals with ASD (American Psychiatric Association, 2013), and this often causes resistance to change and to engaging in new activities. Correspondingly, students with ASD may resist engaging with a new visual support, especially if they perceive the activity as non-preferred and if the visual support is introduced by someone with whom they do not have strong rapport. Resistance to new visual supports could 114 also come from educational assistants who may find the task of learning how to use a new visual support system to be taxing and time consuming. Two general principles are presented here for introducing a new visual support system into a special education classroom. 5.1.4.1 Design a supportive context in which the new system is introduced To effectively introduce a new visual support system into a student’s classroom routine, it is important to first design a supportive context. Designing the context includes deciding who, when, and where to introduce the new system. As Jull and Mirenda (2016) noted, children with ASD are less likely to follow instructions when the instructions come from an individual with whom they do not have a “history of reinforcement” (p. 30). This insight points to the importance of considering who presents a visual support system to a student with ASD, as a student’s reaction to a system that is newly introduced may depend on both the system itself and on their rapport with the person introducing it. The case of Shannon and Ferreira from the current study illustrated the importance of considering this issue. On the day when Shannon and Ferreira introduced the new digital application to Alex, he was already experiencing distress and the researcher was present in the classroom and at a close proximity to him. This was the first time that the researcher and Alex met; furthermore, it was not clear how strong the rapport was between Alex and Shannon and Ferreira, as Alex did not see them as often as he saw his classroom teacher and the educational assistant with whom he worked regularly. One or more of these factors might have contributed to Alex’s resistance to the new technology. 5.1.4.2 Design teaching material and activities for stakeholders As stated previously, educational assistants are important stakeholders in the special education classroom and their perspectives will have a significant impact on the adoption process 115 of a new visual support system. Furthermore, the educational assistants who work with the teacher at the time when a new system is designed will not necessarily continue to work in the same classroom over time. This means that the teacher will have to introduce the visual support system to future educational assistants. Supporting teachers in this endeavor is essential and can facilitate the long-term adoption of the new system. PD researchers can support teachers by designing handouts, online activities, and other artefacts that the teacher can use when introducing the visual support system to new educational assistants. Supporting teachers in this way can also positively impact their perception of the PD research project and their engagement and motivation. 5.2 My Story with PD: Autoethnographic Reflections and Lessons Learned The insights shared thus far have been the results of a positivistic analysis: the researcher gathered data, identified critical events, generated themes, and interpreted their implications. Throughout the analysis and interpretation process, the researcher distanced himself from the data and analyzed his behavior and actions from a “fly on the wall” perspective. Nonetheless, as argued by proponents of auto-ethnography (Ellis et al., 2011), when researching social interactions, one has to take into account that researchers are part of the processes under study and cannot separate themselves from those processes -- we co-create the very processes that we research, and these processes also re-create us. Given the reciprocity between the researcher and the researched, narrative becomes an additional, valuable method for communicating the research “findings.” Through narratives about the researcher’s experiences, the impact of the research process on the researcher can be further discerned, and by making visible the impact of the research process on the researcher, a personal narrative may illustrate aspects of the research process that were missed in a positivistic objectivist analysis. Hence, this section has two 116 objectives: (1) to further contextualize the research process by sharing how the researcher’s commitments, understandings, and values changed throughout the process and (2) to produce more knowledge about the PD process through narrative. In this section, the researcher switches to writing in the first person and shares an auto-ethnographic narrative of how his knowledge and values evolved, the tensions that he encountered, and the lessons that he learned. The section is described as “auto-ethnographic” because it is based on the objectives of auto-ethnography: sharing narratives that contribute to understanding a cultural process (Ellis et al., 2011). Auto-ethnographic narratives are a particular type of stories; like autobiographies, they include narrations of personal epiphanies and lived experiences. However, unlike autobiographies, auto-ethnographic narratives are grounded in a theoretical framework and have a specific objective – that is, to understand the cultural and the social by examining the personal. The researcher shares here his narrative as he engaged in multiple intersecting cultural processes: the process of codesigning a technology with educators, the process of attempting to introduce the codesigned technology into educators’ practice, and the process of conducting participatory research. 5.2.1 Initial Objectives: Codesigning a Cultural Tool to Create Pedagogical Opportunities The theoretical framework for my study was influenced by the work of developmental psychologist Vygotsky, a theorist that I continuously read during my Master’s study and whom I have seen as genuinely committed to social justice and to developing educational theories and theories of development that advance social justice agendas. Following Vygotsky’s theory, I wanted to create a cultural tool that would support children and youth with ASD to develop intellectual and emotional competences such as self-regulation and planning. The following 117 statement written by Vygotsky (1993) reflects his commitment and the commitments that I share with him: The fundamental problem in [the development of a child with special needs] is the incongruence between his psychological structure and the structure of cultural forms. What remains is the necessity of creating special cultural tools suitable for the psychological make-up of such a child. (p. 47) Following Vygotsky’s lead, I believed that new ways of interactions would transform the development of individuals and that, to create new modes of interactions, new cultural tools would need to be created. As I reflect on my early Vygotskyan inspirations, I now see that my understanding of the relationship between cultural tools, interactions, and development was too linear. One of the many ways this study has affected my thinking about development is that the process is more complex than I once imagined it. Creating a new cultural tool is a design challenge, while designing a tool for a specific context requires understanding of that context and of the users. Introducing a cultural tool into a new context is also likely to cause resistance; as this study evolved, I discovered that resistance to change and resistance to new tools (especially technology tools) has been studied extensively and that an academic literature exists on the topic (Sven et al., 2016). 5.2.2 Reality Check The first reality check that I recall encountering was a committee meeting in which one of my committee members questioned my study’s objectives on an ethical basis. She reminded me that I did not have experience in working with students with ASD; hence, she wondered if it was ethically questionable to plan an intervention for individuals with whom I was not familiar. In that meeting, we decided to change the focus of my study from creating an intervention for 118 students with ASD to creating cultural tools for teachers and educators who work with students with ASD. In my study, the educators would be proxies for their students, and I would be designing digital applications based on educators’ perceived needs of their students. I am thankful that the focus of my study changed. As I read the academic studies on visual supports and on participatory design in special education contexts, I developed a strong awareness of the risks of developing interventions for students with ASD. An intervention typically involves introducing a change to the routines followed by students with ASD. When interventions involve changes to their routines, students with ASD are likely to resist change. This creates a situation in which the creator of the intervention may have to navigate this resistance in a variety of ways (Mirenda & Fossett, 2015), and decisions about how to do so should be based on expertise and knowledge in the field. For researchers who do not have the theoretical and practical experience on ASD, it may be unethical to introduce a new routine when there is not a clear rationale backed by evidence that the new routine is likely to lead to a positive outcome. 5.2.3 Changed Focus, Some Uncritical Optimism Even though my focus changed to working with teachers, my commitment to the sociocultural principles of development remained the same, and I was hoping that the tool that I would codesign with educators would facilitate new pedagogical interactions. This was reflected in the research question in my original proposal: “How does a technology designed with educators create new pedagogical opportunities for students with ASD?” Implicit in my research question were several optimistic assumptions: I assumed that the design process would be successful and that a web application would be developed with my research participants. I also 119 assumed that the designed application would be integrated into the educators’ practice. As discussed previously, neither assumption was accurate, in the end. 5.3 Conducting the Study: Unresolved Tensions, Turning to the Literature Throughout the study, several factors pulled me in different directions; these tensions led to many questions, some of which remain unanswered. 5.3.1 Tension 1: Solving Practical Problems or Creating Pedagogical Opportunities During the first meetings with my participants, I perceived a tension between my objectives and their objectives, and a tension between my theoretically driven objective and their practically driven requests. In the first meeting with Brenda, she explicitly stated that she wanted to design an application that would allow her to do what she was already doing but more conveniently. In the first meeting with Caleb, he stated that he was hoping to have an application that would allow him to display a unified schedule for all his educational assistants, and then each educational assistant would have a copy of the schedule on their tablets. In the first meeting with Shannon and Ferreira, the meeting in which they requested to join the research project, they stated that they were having difficulty engaging a student in their lessons, and because the student liked tablets, they were hoping that a tablet-based application would help them engage the student. The logistical objectives of my participants were different than the pedagogical objectives that I envisioned. I was still thinking in Vygotskyan terms -- “creating cultural tools” and “facilitating development” – but the requests of my participants were framed in pragmatic terms. This led to a dilemma: should I suggest to my participants the adoption of pedagogical goals or should I focus on their practical goals? It was difficult to navigate the dilemma as I was lacking a solid rationale based on which I could commit to clear and consistent approach. The 120 dilemma remained till I read Grundy’s (1982) work on the three types of action research: technical, practical, and emancipatory. Reading Grundy (1982) led to an epiphany, to determine whether I should fully accept my participants’ priorities or whether I should “nudge” them (as Penuel et al., 2007 did with their participants) to adopt different goals. I had to decide in advance whether my project was going to be a technical, practical, or emancipatory action research project. 5.3.2 Tension 2: Giving Choices to Participants or Making Choices for Them A second tension that occurred during the research process was between giving participants more choices and control over the interface and making choices for them. As I was designing the digital applications, I hesitated to make design decisions myself; for example, when I had an insight about improving design and usability, I hesitated to follow the insight. I started the three collaborations with blank pages and assumed that by doing so, I was giving participants more control over the process of creating and displaying visual supports. I also assumed that involving them in the initial design of the applications would make it easier for them to use the applications later; in other words, I assumed that participation in design would increase usability. This tension was again resolved by turning to relevant literature; in particular, I found Druin’s four categories of participation helpful in articulating the role to give to participants. As stated in a previous chapter, Druin (2002) articulated four categories of participation: design partner, informant, tester, and user. The role that I initially assigned to participants was that of design partner, but I realized that, given their significant time constraints and their preferences for reduced time commitments, the role of informants might have been a better fit. 121 As I resolved the tension over the participants’ role in the design process, I turned to a related tension, the tension related to my own role in the research process. Throughout the research process, I was hesitant to suggest ideas to my participants because I wanted to ensure that the designed technology reflected their ideas and preferences, not mine. Upon reflecting on my hesitance, I speculated that the hesitance that I experienced may have been caused by a tension in my epistemological beliefs. My hesitance may have been grounded in an objectivist epistemology, an epistemology that assumes that entities exist objectively and independently of the researcher observing them. From this perspective, the needs and preferences of my participants would be objective entities and my role as a design researcher would be to facilitate the expression of my participants’ needs as they are, without influencing them or misrepresenting them. Yet, the methodology of participatory design is based on a different epistemology, a constructionist epistemology in which entities such as needs and preferences are dynamic and continuously changing. From the constructionist perspective, the encounter between researcher and participant is bound to impact the perceived needs and preferences of both parties, and the objective is not to capture the needs and preferences of participants as they are, but to create a democratic and dialogic environment in which both researcher and participants learn from each other and develop interests and preferences that emerge from joint reflection and design. Hence, the object of inquiry in a constructionist epistemology is not necessarily the already existent interests and preferences of participants, but the interests and preferences that emerge from engagement in democratic and dialogic design. Correspondingly, the researcher should not hesitate in expressing their opinion and sharing their expertise, as long as this happens in a democratic and dialogic environment. 122 With a stronger commitment to the constructionist epistemology, and a stronger commitment to studying the becoming of my participants, as opposed to their being, I resolved the tension over my role as a researcher in the design process. I also want to note that even within an objectivist epistemology, there is an important role for the researcher in the design process. While we may suppose that participants are the experts about their needs and about the needs of their students, we cannot assume that participants are experts in translating their needs and the needs of their students into designs that work for them and for their students. Hence, the objectivist epistemologist who conducts design research may want to capture the needs and preferences of participants as they are, but still, they will need to use their expertise to support participants in translating their knowledge of their needs and preferences into designs that work for them. Participants are experts about their needs, they are not necessarily experts on translating their needs into usable designs. 5.3.3 Tension 3: Timing Introduction of the Application One memorable event from the study was the Alex’s reaction to the application I codesigned with Shannon and Ferreira. As mentioned previously, the observation was scheduled in the last week of the school year when Alex was already experiencing a high level of stress. He was holding another device when he was shown the new digital application and asked to identify the parts of the human body. He showed much more interest in the familiar application on his device than on the new one, and was resistant to exploring the new one as a result. One lesson learned from that observation was that in designing the study, I focused on the content and the interface of the application but did not plan for variables that could affect the student’s reaction to the application, such as the timing of introduction, the location of introduction, and the person who introduces the new application. In other words, I focused on the systems I was designing 123 more than on the extrinsic variables that may affect their adoption. In retrospect, the literature on visual supports helped me to identify some of the extrinsic factors that are important to consider, such as prompting and rapport with the person introducing the visual support. The literature on technology integration helped me to identify other relevant factors, such as perceived ease of use and perceived usefulness. While I was unable to identify a methodological framework that integrates these factors into the PD process and while I still do not have definitive answers, I have the curiosity to find out. 5.4 Perceived Failure and Renewed Commitment Data collection for this research project ended in June of the 2017-2018 academic year. In my original research proposal, I stated that I was going to conduct a thematic analysis of my data, yet by the end of the project I was seeing fundamental differences between my three cases, and articulating common themes seemed to be an impossible task. Furthermore, the research questions that I wrote in my initial proposal seemed difficult to answer with the data that I had. In addition, the failure to create systems that were effectively integrated into my participants’ practice caused me to perceive the project as a failure. In the following passage, Wolcott (1994), offers pragmatic advice on how to frame research questions that examine efforts to introduce change into a context -- advice that I read after finishing my study and that I will likely remember in the future: I have occasionally succeeded in encouraging doctoral students originally intending to study ‘change’ -- on what invariably proves too short a timetable ever to hope to see it -- to direct their attention instead to what can be learned about any social system from its efforts to change itself [...] In that way, a 124 ‘disappointing’ effort to study change may become a provocative analysis of the maintenance functions of a social system. (p. 178) The objective of my study was to codesign web applications that would lead to changes into the practices of my participants and one of my research questions was related to the changes that the web applications would cause. However, as Wolcott (1994) pointed out, effort to introduce change into a context does not always lead to change. Hence, articulating research questions that examine the elements in a context that resist change and the elements that facilitate change is a more pragmatic option. Wolcott (1994) was one among many readings that I completed after collecting my data that helped me critically reflect on the research process that I followed. Reading about PD studies that encountered similar challenges to the ones I encountered was reassuring. Other PD researchers have developed products that were dropped from use (Righi et al., 2018) and other PD researchers also reported failures (Lundmark, 2017). Reading the experiences of other researchers was both a learning experience and a source of emotional support and confirmation. As Steketee et al. (2020) noted in their autoethnography: “As participatory research has become more common, experts have uncovered a “dark side” to this collaborative model with practical, personal, and professional costs to researchers, as well as costs to stakeholders” (p. 2). Hence the importance of researchers sharing the “dark side” of their experiences and contributing knowledge on how to navigate the challenges, both intellectual and emotional, that surface when conducting participatory research. At several points in my research, I experienced regret and I thought about abandoning my project and conducting a new study in a context where designing and introducing a web application would be less challenging. But as I reflected retrospectively on my journey, I was 125 reminded that the primary objective of participatory design is democratization – that is, giving people control over the technologies that shape their work processes and their learning. If I and other researchers avoid contexts that we perceive as challenging, then these contexts will be deprived of opportunities of influencing the technologies that shape their work and their lives, and PD’s democratic ideal will be far from reach. The special educational classroom has many challenges, and these challenges make it even more important to work with teachers, educational assistants, speech language pathologists, and other stakeholders in order to design technology that support them in managing these challenges. 5.5 Assessing Internal Reliability: Alignment between Knowledge, Values, Stakeholders, and Outcomes The research process in this study is difficult to replicate because it was adapted to the particular classrooms in which it took place and was shaped by local policies and local infrastructures, which may also be unique. Furthermore, I myself influenced the process—my values, my attitude, and my rapport with participants. As Frauenberger et al. (2015) argued, any attempt to use positivistic practices to assess the reliability of knowledge claims made in PD is bound to fail; and the knowledge produced by PD, when assessed with positivistic criteria, is also bound to fail the assessment and to be perceived as weak and lacking rigor. Frauenberger et al. (2015) argued that assessment of the reliability of a PD project and its knowledge claims should be based on the epistemological grounds of the PD methodology and its commitments. In PD, there are four elements that shape the dynamics of a research project: the knowledge that the project aims to produce, the values that drive the project, the stakeholders that are affected by the project, and the outcomes of the project. Frauenberger et al. (2015) proposed assessing reliability based on alignment between these four elements. 126 An application of Frauenberger et al.’s (2015) framework led to identification of points of alignments and misalignments in the current study. I interpreted points of alignment as signs of a strong internal reliability and points of misalignment as weaknesses in the project; the latter were used to articulate improvements that may have benefited both the current project and future projects. I embraced the democratic PD values that focused on designing technology with participants in order to give them control over the work process and to translate their knowledge and goals into a technical artefact that would transform their work process and give them more agency. My aim in this study was to empower participants by co-designing a product that was based on their needs and preferences and that would allow them to achieve new pedagogical goals that would empower their students with ASD. The data that resulted from the study concerned the challenges and benefits of designing with educators, the peculiarities of the special education context, the way educators use visual supports in their practice, and what they hoped to achieve with visual supports. The knowledge that was produced in the study was aligned with my values. The findings helped me understand how to empower educators through design and they illustrate how, as the researcher, my conception of empowerment changed throughout the process. Knowledge about educational assistants and their unique situation was also presented and their needs were assessed. On the other hand, the impact of the research on stakeholders in general was minimal. The designed technologies were not incorporated into practice, for different reasons. It cannot be argued that the project introduced new practices to the teaching methods of participants. In this area, the project did not reach its goals, and this lack of impact was misaligned with my values. Another major issue of misalignment concerns the role stakeholders had in the process; this issue 127 was revealed during the data analysis. The products that were designed may have created more work that was neither educative nor empowering for one group of stakeholders: the educational assistants. The technology that was created failed on numerous occasions because of user-error; nonetheless, failure due to user error counts as bad design. Moreover, the process delivered a technology that may have made educators uncomfortable, and integration of the applications into practice may have created more tasks work for educational assistants: finding images, cropping images, uploading images, and creating supports. Educational assistants on short term appointments may not benefit from such work, especially if they end up moving into a classroom where other types of digital applications are used. From the perspective of educational assistants, the applications that were created for both Brenda and Caleb may have included the same amount of de-skilling work as the commercially available applications, and the learning curve may have been the same. My values and their impact on educational assistants became better aligned towards the end of the project when I created two matching games for two educational assistants based on their requests. The matching games did not require educational assistants to upload any images; I had already uploaded the images for them based on a previous conversation. The matching games were also created based on individual conversations between myself and the educational assistants; the teacher was not involved in these conversations. While the matching games that were created did not require de-skilling work from educational assistants, they also did not allow educational assistants to create more matching tasks for their students. These products could be improved by creating an interface that would allow them to create matching tasks with ease. 128 5.6 Summary and Conclusions Visual supports facilitate communication with students with ASD and can support students with ASD to complete routines and transition between activities independently. Due to the increased availability of tablet technologies, it has become possible for educators to create interactive digital visual supports and display them on tablets. Tablets have created opportunities for educators to create more advanced and complex visual supports and to conveniently create and store supports on their devices. To effectively integrate tablet-based visual supports into their instructions, educators need to develop systems that allow them to find images; use them to create support; and save, access, and display the supports to their students. The systems should be compatible with the available technology infrastructure in the context where educators work and compatible with educators’ pedagogical beliefs and habits with technology. Participatory Design (PD) has been utilized to support educators in developing visual support systems and other technology systems in instruction. By engaging educators in the design of the technology they use, PD aims to develop technologies that can be effectively integrated into teaching. Also, researchers found that when educators engage in design activities related to instructional technology, their relationship to technology can improve and their intention to use technology in instruction increases (Matuk et al., 2018). In this study, the researcher collaborated with educators to develop tablet-based visual supports applications that they could use in their teaching. The procedure that the researcher followed consisted of the following steps: (a) creating paper prototypes with participants; (b) developing electronic prototypes based on the paper prototypes; (c) getting feedback on the electronic prototypes and using it to develop the final product; and (d) teaching participants how 129 to use the final product and (e) observing them using it with their students. Despite some initial hesitation, participants successfully created paper prototypes of the applications they sought to develop. Seeing the electronic prototypes was a positive moment for them and they provided positive feedback and expressed high motivation at this point. This observation supports other researchers’ observations on the importance of starting design early in the research process and delivering functional prototypes early; otherwise, participants may perceive the design meetings as unsuccessful (Cviko et al., 2015). Despite the high motivation that participants demonstrated early in the research process, challenges surfaced and the products that the researcher developed were either minimally used or dropped from use. This has revealed a gap in the research process. The researcher developed the final products based on feedback received from participants on the electronic prototypes; however, this feedback was not sufficient to prevent the usability issues that surfaced when the final products were introduced. By solely relying on participants’ feedback on electronic prototypes, the researcher missed other sources of information that could have improved the design process. Planning observations in the classroom to observe participants’ use of technology, the applications they used, and the infrastructure that was available to them would have given the researcher crucial design information. Another gap in the research procedure was the limited interactions the researcher had with potential users of the final products. In the collaboration with Brenda, Brenda had the lead in the design process; however, the users of her application included educational assistants and students, and the usability of the applications depended on their reactions and perceptions. In the collaboration with Caleb, educational assistants were recruited as participants after the electronic prototype was developed; this allowed them to contribute to the project, but their contribution 130 was limited. In the collaboration with Shannon and Ferreira, the student who used the application was not involved in the design process. Despite his limited communicative abilities, the student could have been indirectly involved into the design process had the researcher planned observation sessions to observe how he used technology and the applications that he enjoyed using, in order to incorporate these observations in the design of the final product. Observing students and interviewing educational assistants prior to developing the final products may have provided crucial design information for designing highly usable applications. By solely relying on input from a limited number of users, the researcher unintentionally assigned too many responsibilities to these users: the responsibility of converting their needs into designs, the responsibility of communicating the needs of their students and their educational assistants, and the responsibility of translating the needs of their students and their educational assistants into design decisions. The study’s participants had limited time and busy schedules, assigning too many responsibilities to them did not help them engage more with the project. Observations and inferences from this study revealed the importance of supporting educators as they develop visual support systems. Visual support systems are an important area of research given their pedagogical effectiveness and the lack of support given to educators and the limited time they have to experiment with technology. It appears that educators are often on their own when developing visual support systems (Donato et al., 2014). At this point, it is important to remind scholars of Margaret Cassidy’s (1998) insights: it is easy to get excited about new advanced technologies and to hope that these advanced technologies will increase engagement of students; yet, if the barriers to technology integration are not addressed, no matter how sophisticated new technologies get, integrating them in teaching will still be difficult. The 131 challenges involved in using visual supports technologies in teaching is an important domain of inquiry, and further research is needed on how to support educators to address these challenges. 132 References American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders (5th. Ed.). Washington, DC: APA. Arthur-Kelly, M., Sigafoos, J., Green, V., Mathisen, B., & Arthur-Kelly, R. (2009). Issues in the use of visual supports to promote communication in individuals with autism spectrum disorder. Disability and Rehabilitation, 31(18), 1474–1486. Ashktorab, Z., & Vitak, J. (2016). Designing cyberbullying mitigation and prevention solutions through participatory design with teenagers. In J. Kay, & A. Druin (Eds.), Proceedings of the 2016 CHI conference on human factors in computing systems (pp. 3895–3905). New York, NY: Association for Computing Machinery. Benssassi, E. M., Gomez, J. C., Boyd, L. E., Hayes, G. R., & Ye, J. (2018). Wearable assistive technologies for autism: Opportunities and challenges. IEEE Pervasive Computing, 17(2), 11-21. Benton, L., & Johnson, H. (2014). Structured approaches to participatory design for children: Can targeting the needs of children with autism provide benefits for a broader child population? Instructional Science, 42(1), 47-65. Blum-Dimaya, A., Reeve, S. A., Reeve, K. F., & Hoch, H. (2010). Teaching children with autism to play a video game using activity schedules and game-embedded simultaneous video modeling. Education and Treatment of Children, 33(3), 351-370. Bossavit, B., & Parsons, S. (2016). Designing an educational game for and with teenagers with high functioning autism. In C. Bossen, R. Smith, A. Kanstrup, J. McDonnel, M. Teli, & K. Bødker (Eds.), Proceedings of the 14th participatory design conference, (pp. 11–20). New York, NY: Association for Computing Machinery. 133 Brodhead, M. T., Higbee, T. S., Pollard, J. S., Akers, J. S., & Gerencser, K. R. (2014). The use of linked activity schedules to teach children with autism to play hide‐and‐seek. Journal of Applied Behavior Analysis, 47(3), 645-650. Brown, K. E., & Mirenda, P. (2006). Contingency mapping: Use of a novel visual support strategy as an adjunct to functional equivalence training. Journal of Positive Behavior Interventions, 8(3), 155-164. Carlile, K. A., Reeve, S. A., Reeve, K. F., & DeBar, R. M. (2013). Using activity schedules on the iPod touch to teach leisure skills to children with autism. Education and Treatment of Children, 36(2), 33-57. Cassidy, M. (1998). Historical perspectives on teaching with technology in K‐12 schools. Atlantic Journal of Communication, 6(2), 170–184. Chang, H. (2013). Individual and collaborative autoethnography as method. In S. H. Jones, T. E. Adams, C. Ellis (Eds.), Handbook of autoethnography (pp. 107–122). New York, NY: Routledge. Chien, M. E., Jheng, C. M., Lin, N. M., Tang, H. H., Taele, P., Tseng, W. S., & Chen, M. Y. (2015). iCAN: A tablet-based pedagogical system for improving communication skills of children with autism. International Journal of Human-Computer Studies, 73, 79–90. Cihak, D. F. (2011). Comparing pictorial and video modeling activity schedules during transitions for students with autism spectrum disorders. Research in Autism Spectrum Disorders, 5(1), 433-441. Collin, P., & Swist, T. (2016). From products to publics? The potential of participatory design for research on youth, safety and well-being. Journal of Youth Studies, 19(3), 305–318. 134 Davies, D. K., Stock, S. E., & Wehmeyer, M. L. (2002). Enhancing independent task performance for individuals with mental retardation through use of a handheld self-directed visual and audio prompting system. Education and Training in Mental Retardation and Developmental Disabilities, 209-218. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management Science, 35(8), 982–1003. Dettmer, S., Simpson, R. L., Myles, B. S., & Ganz, J. B. (2000). The use of visual supports to facilitate transitions of students with autism. Focus on Autism and Other Developmental Disabilities, 15(3), 163-169. Donato, C., Shane, H. C., & Hemsley, B. (2014). Exploring the feasibility of the Visual Language in Autism program for children in an early intervention group setting: Views of parents, educators, and health professionals. Developmental Neurorehabilitation, 17(2), 115–124. Druin, A. (2002). The role of children in the design of new technology. Behavior and Information Technology, 21(1), 1–25. Ellis, C., Adams, T. E., & Bochner, A. P. (2011). Autoethnography: An overview. Historical Social Research, 36(4), 273-290. Ertmer, P. A. (1999). Addressing first-and second-order barriers to change: Strategies for technology integration. Educational Technology Research and Development, 47(4), 47–61. Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423–435. 135 Fage, C., Pommereau, L., Consel, C., Balland, E., & Sauzéon, H. (2016). Tablet-based activity schedule in mainstream environment for children with autism and children with ID. ACM Transactions on Accessible Computing, 8(3), 1-26. Fage, C., Consel, C., Etchegoyhen, K., Amestoy, A., Bouvard, M., Mazon, C., & Sauzéon, H. (2019). An emotion regulation app for school inclusion of children with ASD: Design principles and evaluation. Computers & Education, 131, 1–21. Frauenberger, C., Good, J., & Keay-Bright, W. (2011). Designing technology for children with special needs: bridging perspectives through participatory design. CoDesign, 7(1), 1–28. Frauenberger, C., Good, J., Fitzpatrick, G., & Iversen, O. S. (2015). In pursuit of rigour and accountability in participatory design. International Journal of Human-Computer Studies, 74, 93–106. Giles, A., & Markham, V. (2017). Comparing book-and tablet-based picture activity schedules: Acquisition and preference. Behavior Modification, 41(5), 647-664. Groussard, P. Y., Pigot, H., & Giroux, S. (2018). From conception to evaluation of mobile services for people with head injury: A participatory design perspective. Neuropsychological Rehabilitation, 28(5), 667–688. Grundy, S. (1982). Three modes of action research. Curriculum Perspectives, 2(3), 23–34. Hall, L. J., McClannahan, L. E., & Krantz, P. J. (1995). Promoting independence in integrated classrooms by teaching aides to use activity schedules and decreased prompts. Education and Training in Mental Retardation and Developmental Disabilities, 30(3), 208–217. Halskov, K., & Hansen, N. B. (2015). The diversity of participatory design research practice at PDC 2002–2012. International Journal of Human-Computer Studies, 74, 81–92. 136 Hayes, G. R. (2018). Design, action, and practice: Three branches of the same tree. In V. Wulf, V. Pipek, D. Randall, M. Rohde, K. Schmidt, & G. Stevens (Eds.), Socio-informatics (pp. 303-318). Oxford, UK: Oxford University Press. Hayes, G. R., Hirano, S., Marcu, G., Monibi, M., Nguyen, D. H., & Yeganyan, M. (2010). Interactive visual supports for children with autism. Personal and ubiquitous Computing, 14(7), 663–680. Hirano, S. H., Yeganyan, M. T., Marcu, G., Nguyen, D. H., Boyd, L. A., & Hayes, G. R. (2010). vSked: evaluation of a system to support classroom activities for children with autism. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1633–1642). New York, NY: Association for Computing Machinery. Hodgdon, L. (2016). Visual strategies for improving communication: Practical supports for autism spectrum disorders. Troy, MI: QuirkRoberts Publishing. Holloway, I., & Freshwater, D. (2007). Narrative research in nursing. Oxford, UK: Blackwell Publishing. Hume, K., Loftin, R., & Lantz, J. (2009). Increasing independence in autism spectrum disorders: A review of three focused interventions. Journal of Autism and Developmental Disorders, 39(9), 1329–1338. Jull, S., & Mirenda, P. (2016). Effects of a staff training program on community instructors’ ability to teach swimming skills to children with autism. Journal of Positive Behavior Interventions, 18(1), 29-40. Kidder, J. E., & McDonnell, A. P. (2017). Visual aids for positive behavior support of young children with autism spectrum disorders. Young Exceptional Children, 20(3), 103–116. 137 Kim, H. J., Choi, J., & Lee, S. (2019). Teacher experience of integrating tablets in one-to-one environments: Implications for orchestrating learning. Education Sciences, 9(2), 87-101. Knight, V., Sartini, E., & Spriggs, A. D. (2014). Evaluating visual activity schedules as evidence-based practice for individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 45(1), 157–178. Könings, K. D., van Zundert, M. J., Brand‐Gruwel, S., & van Merriënboer, J. J. (2007). Participatory design in secondary education: Is it a good idea? Students’ and teachers’ opinions on its desirability and feasibility. Educational Studies, 33(4), 445-465. Koumpouros, Y., & Kafazis, T. (2019). Wearables and mobile technologies in Autism Spectrum Disorder interventions: A systematic literature review. Research in Autism Spectrum Disorders, 66, Article 101405. Kuure, L., Molin-Juustila, T., Keisanen, T., Riekki, M., Iivari, N., & Kinnula, M. (2016). Switching perspectives: from a language teacher to a designer of language learning with new technologies. Computer Assisted Language Learning, 29(5), 925–941. Lundmark, S. (2018). Design project failures: Outcomes and gains of participation in design. Design Studies, 59, 77-94. MacDuff, G. S., Krantz, P. J., & McClannahan, L. E. (1993). Teaching children with autism to use photographic activity schedules: Maintenance and generalization of complex response chains. Journal of Applied Behavior Analysis, 26(1), 89–97. Makki, T. W., O'Neal, L. J., Cotten, S. R., & Rikard, R. V. (2018). When first-order barriers are high: A comparison of second-and third-order barriers to classroom computing integration. Computers & Education, 120, 90-97. 138 Matuk, C., Gerard, L., Lim-Breitbart, J., & Linn, M. (2016). Gathering requirements for teacher tools: Strategies for empowering teachers through co-design. Journal of Science Teacher Education, 27(1), 79–110. Massey, N. G., & Wheeler, J. J. (2000). Acquisition and generalization of activity schedules and their effects on task engagement in a young child with autism in an inclusive pre-school classroom. Education and Training in Mental Retardation and Developmental Disabilities, 35(3), 326–335. McClannahan, L. E., & Krantz, P. (2010). Activity schedules for children with autism: Teaching independent behavior. Bethesda, MD: Woodbine House. Mechling, L. C., Gast, D. L., & Seid, N. H. (2009). Using a personal digital assistant to increase independent task completion by students with autism spectrum disorder. Journal of Autism and Developmental Disorders, 39(10), 1420-1434. Mirenda, P., & Fossett, B. (2015). Visual supports as antecedent and teaching interventions. In F. Brown, J. Anderson, & R. De Pry (Eds.), Individual positive behavior supports: A standards-based guide to practices in school and community-based settings (pp. 347-360). Baltimore: Paul H. Brookes Publishing Co. Morrison, R. S., Sainato, D. M., Benchaaban, D., & Endo, S. (2002). Increasing play skills of children with autism using activity schedules and correspondence training. Journal of Early Intervention, 25(1), 58-72. Norman, D. A. (1988). The psychology of everyday things. New York, NY: Basic Books. Parsons, S., & Cobb, S. (2014). Reflections on the role of the “users”: Challenges in a multi-disciplinary context of learner-centred design for children on the autism spectrum. International Journal of Research & Method in Education, 37(4), 421–441. 139 Penuel, W. R., Roschelle, J., & Shechtman, N. (2007). Designing formative assessment software with teachers: An analysis of the co-design process. Research and Practice in Technology Enhanced Learning, 2(01), 51–74. Pierce, K. L., & Schreibman, L. (1994). Teaching daily living skills to children with autism in unsupervised settings through pictorial self‐management. Journal of Applied Behavior Analysis, 27(3), 471-481. Preece, J., Rogers, Y., & Sharp, H. (2011). Interaction design: Beyond human-computer interaction. New York, NY: J. Wiley & Sons. Prizant, B. M., & Fields-Meyer, T. (2015). Uniquely human: A different way of seeing autism. New York, NY: Simon and Schuster. Quill, K. A. (1997). Instructional considerations for young children with autism: The rationale for visually cued instruction. Journal of Autism and Developmental Disorders, 27(6), 697–714. Righi, V., Sayago, S., Rosales, A., Ferreira, S. M., & Blat, J. (2018). Co-designing with a community of older learners for over 10 years by moving user-driven participation from the margin to the centre. CoDesign, 14(1), 32-44. Shapka, J. D., & Ferrari, M. (2003). Computer-related attitudes and actions of teacher candidates. Computers in Human Behavior, 19(3), 319–334. Silva, M., & Breuleux, A. (1994). The use of participatory design in the implementation of internet-based collaborative learning activities in K-12 classrooms. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 2(3), 99-128. 140 Siozos, P., Palaigeorgiou, G., Triantafyllakos, G., & Despotakis, T. (2009). Computer based testing using “digital ink”: Participatory design of a Tablet PC-based assessment application for secondary education. Computers & Education, 52(4), 81–819. Smith, F., Wallengren, C., & Öhlén, J. (2017). Participatory design in education materials in a health care context. Action Research, 15(3), 310–336. Stacy, S. T., Cartwright, M., Arwood, Z., Canfield, J. P., & Kloos, H. (2017). Addressing the math-practice gap in elementary school: Are tablets a feasible tool for informal math practice? Frontiers in Psychology, 8, 1-12. Steketee, A. M., Archibald, T. G., & Harden, S. M. (2020). Adjust your own oxygen mask before helping those around you: An autoethnography of participatory research. Implementation Science, 15(1), 1-20. Stromer, R., Kimball, J. W., Kinney, E. M., & Taylor, B. A. (2006). Activity schedules, computer technology, and teaching children with autism spectrum disorders. Focus on Autism and Other Developmental Disabilities, 21(1), 14-24. Teo, T. (2015). Comparing pre-service and in-service teachers' acceptance of technology: Assessment of measurement invariance and latent mean differences. Computers & Education, 83, 22–31. Tsai, C. C., & Chai, C. S. (2012). The “third”-order barrier for technology-integration instruction: Implications for teacher education. Australasian Journal of Educational Technology, 28(6), 1057-1060. Van Laarhoven, T., Kraus, E., Karpman, K., Nizzi, R., & Valentino, J. (2010). A comparison of picture and video prompts to teach daily living skills to individuals with autism. Focus on Autism and Other Developmental Disabilities, 25(4), 195–208. 141 Vaughn, B., & Horner, R. (1995). Effects of concrete versus verbal choice systems on problem behavior. Augmentative and Alternative Communication, 11(2), 89–92. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: toward a unified view. MIS Quarterly, 27(3), 425–478. Vongkulluksn, V. W., Xie, K., & Bowman, M. A. (2018). The role of value on teachers' internalization of external barriers and externalization of personal beliefs for classroom technology integration. Computers & Education, 118, 70–81. Vygotsky, L.S. (1993). The collected works of L.S. Vygotsky Volume 2: The fundamentals of defectology (abnormal psychology and learning disabilities). New York, NY: Plenum. Wang, L., Ertmer, P. A., & Newby, T. J. (2004). Increasing preservice teachers’ self-efficacy beliefs for technology integration. Journal of Research on Technology in Education, 36(3), 231-250. Winters, N., & Mor, Y. (2008). IDR: A participatory methodology for interdisciplinary design in technology enhanced learning. Computers & Education, 50(2), 579–600. Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oakes, CA: Sage Publications.
Thesis/Dissertation
eng
Human Development, Learning, and Culture
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Shapka, Jennifer
Participatory design in the special education classroom : designing technology with educators for students with autism spectrum disorder
Text