Education, Faculty of
Educational and Counselling Psychology, and Special Education (ECPS), Department of
DSpace
UBCV
Molyneux, Tonje Mari
Master of Arts - MA
University of British Columbia
The children and youth of today will need social-emotional competence to help them navigate the challenges of an uncertain future. School-based social and emotional learning (SEL) programs have been shown to be effective in helping youth develop these vital, malleable skills, and that the positive outcomes endure. However, achieving these positive outcomes requires quality implementation by classroom teachers. There are many factors that contribute to teachers’ motivation, readiness, and ability to deliver an SEL program as intended. Using data collected in four experimental evaluations of SEL programs, two studies were conducted to explore the relationship between program implementation quality and teacher factors such as teaching efficacy, stress, job satisfaction, and beliefs about SEL. Study 1 explored relations of teachers’ (N = 32) pre-implementation factors including SEL experience, job satisfaction, and SEL beliefs, to program implementation quality. Correlational and multiple regression analyses revealed no significant relationships between teacher factors and implementation quality. However, the total number of lessons in the SEL programs surfaced as a significant predictor of implementation quality, where fewer lessons predicted greater implementation quality. Study 2 explored how changes in teacher factors including SEL beliefs, occupational stress, efficacy, and job satisfaction were related to teachers' program implementation quality. Teachers (N = 23) were grouped into low- and high-quality implementation categories based on their total implementation quality scores. Paired-samples t tests showed that teachers’ depersonalization, a symptom of occupational stress, and teaching efficacy scores significantly changed after implementing an SEL program; however, these changes were not in the hypothesized direction. Specifically, teaching efficacy scores decreased and depersonalization scores increased. Taken together, these findings indicate the need for further research to examine the relationship between teacher factors and SEL program implementation. Study contributions, limitations, and directions for future research are discussed, as well as practical implications for supporting teachers’ implementation.
THE RELATIONSHIP BETWEEN TEACHER FACTORS AND SOCIAL AND EMOTIONAL LEARNING PROGRAM IMPLEMENTATION by Tonje Mari Molyneux M.Ed., University of Calgary, 2008 B.Ed., University of Alberta, 2001 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Human Development, Learning, and Culture) THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) May 2021 © Tonje Mari Molyneux, 2021 ii The following individuals certify that they have read, and recommend to the Faculty of Graduate and Postdoctoral Studies for acceptance, a thesis entitled: The Relationship Between Teacher Factors and Social and Emotional Learning Program Implementation submitted by Tonje Mari Molyneux in partial fulfillment of the requirements for the degree of Master of Arts in Human Development, Learning, and Culture Examining Committee: Dr. Kimberly Schonert-Reichl, Professor, Department of Psychology, University of Illinois at Chicago (on leave, University of British Columbia) Supervisor Dr. Eva Oberle, Assistant Professor, School of Population and Public Health, University of British Columbia Supervisory Committee Member Dr. Deborah Butler, Professor, Educational and Counselling Psychology, and Special Education, University of British Columbia Supervisory Committee Member iii Abstract The children and youth of today will need social-emotional competence to help them navigate the challenges of an uncertain future. School-based social and emotional learning (SEL) programs have been shown to be effective in helping youth develop these vital, malleable skills, and that the positive outcomes endure. However, achieving these positive outcomes requires quality implementation by classroom teachers. There are many factors that contribute to teachers’ motivation, readiness, and ability to deliver an SEL program as intended. Using data collected in four experimental evaluations of SEL programs, two studies were conducted to explore the relationship between program implementation quality and teacher factors such as teaching efficacy, stress, job satisfaction, and beliefs about SEL. Study 1 explored relations of teachers’ (N = 32) pre-implementation factors including SEL experience, job satisfaction, and SEL beliefs, to program implementation quality. Correlational and multiple regression analyses revealed no significant relationships between teacher factors and implementation quality. However, the total number of lessons in the SEL programs surfaced as a significant predictor of implementation quality, where fewer lessons predicted greater implementation quality. Study 2 explored how changes in teacher factors including SEL beliefs, occupational stress, efficacy, and job satisfaction were related to teachers' program implementation quality. Teachers (N = 23) were grouped into low- and high-quality implementation categories based on their total implementation quality scores. Paired-samples t tests showed that teachers’ depersonalization, a symptom of occupational stress, and teaching efficacy scores significantly changed after implementing an SEL program; however, these changes were not in the hypothesized direction. Specifically, teaching efficacy scores decreased and depersonalization scores increased. Taken iv together, these findings indicate the need for further research to examine the relationship between teacher factors and SEL program implementation. Study contributions, limitations, and directions for future research are discussed, as well as practical implications for supporting teachers’ implementation. v Lay Summary Social and emotional learning (SEL) is a process that helps children and youth develop skills such as self-control, self-awareness, and interpersonal skills, that are important for success in school and life. Research shows that school-based programs can help develop these skills, but sometimes they work and sometimes they do not. Teachers most often deliver the programs, and researchers have started to study factors that might affect teachers’ ability to teach SEL programs well. To add to this research, two studies were conducted to explore the relationship between teacher factors and SEL program delivery quality. Study 1 found that the number of lessons was related to program delivery quality, with more lessons leading to poorer quality delivery. Study 2 found that teachers’ stress symptoms increased after delivering the SEL program, and their beliefs about their teaching ability decreased. Together, these findings suggest that teachers need support to deliver SEL programs well. vi Preface This thesis is an original intellectual product of the author, Tonje Molyneux. All research presented as part of this thesis was approved by the University of British Columbia’s Behavioural Research Ethics Board (certificate # H20-03161). The data used in this study are secondary data collected in four experimental evaluations of social and emotional learning programs led by Dr. Kimberly A. Schonert-Reichl. The evaluations were approved by the University of British Columbia’s Behavioural Research Ethics Board (certificate # H11-00391, H14-02370, H16-00155, H16-03240). vii Table of Contents Abstract ......................................................................................................................................... iii Lay Summary…………………………………………………………………………………….v Preface ........................................................................................................................................... vi Table of Contents ........................................................................................................................ vii List of Tables ............................................................................................................................... xii List of Figures ............................................................................................................................. xiii Acknowledgements .................................................................................................................... xiv Chapter 1: Introduction ................................................................................................................1 Introduction ..................................................................................................................... 1 Social and Emotional Learning ....................................................................................... 3 1.2.1 Benefits of Social and Emotional Learning Programs ................................................ 4 1.2.1.1 Healthy Development and Well-Being ............................................................... 5 1.2.1.2 Increased Academic Achievement ...................................................................... 5 1.2.1.3 Positive School Climate ...................................................................................... 6 1.2.1.4 Impact on Public Health...................................................................................... 7 School-Based SEL Programs .......................................................................................... 8 1.3.1 Features of Effective Programs ................................................................................... 8 1.3.2 Program Ratings.......................................................................................................... 9 Inconsistency in Program Outcomes ............................................................................ 10 Chapter 2: Literature Review .....................................................................................................12 Implementation Science ................................................................................................ 12 viii 2.1.1 Implementation Components .................................................................................... 13 2.1.2 Factors Affecting Implementation ............................................................................ 15 2.1.3 Implementation Frameworks .................................................................................... 16 2.1.4 Process Model of Sustained Implementation by Teachers ....................................... 17 2.1.5 Implementation Measurement .................................................................................. 19 SEL Program Implementation ...................................................................................... 21 2.2.1 Effective SEL Implementation ................................................................................. 22 2.2.2 SEL Program Evaluation and Implementation ......................................................... 24 Teacher Factors and SEL Implementation .................................................................... 27 2.3.1 Teacher Pre-Implementation Factors ........................................................................ 28 2.3.1.1 Efficacy ............................................................................................................. 28 2.3.1.2 Stress/Burnout ................................................................................................... 30 2.3.1.3 Beliefs ............................................................................................................... 32 2.3.2 Teacher Supported/Sustained Implementation Factors ............................................ 35 2.3.2.1 Initial Training and Ongoing Coaching ............................................................ 35 2.3.2.2 Student Engagement and Responsiveness ........................................................ 37 2.3.2.3 Change in Teacher Factors ................................................................................ 38 The Present Studies ....................................................................................................... 38 2.4.1 Study 1 ...................................................................................................................... 41 2.4.2 Study 2 ...................................................................................................................... 41 Chapter 3: Method .......................................................................................................................43 Participants .................................................................................................................... 43 Procedure ...................................................................................................................... 48 ix 3.2.1 Data Collection ......................................................................................................... 48 Measures ....................................................................................................................... 50 3.3.1 Teacher Demographics ............................................................................................. 51 3.3.2 Teacher Factors ......................................................................................................... 51 3.3.2.1 SEL Experience ................................................................................................ 51 3.3.2.2 Job Satisfaction ................................................................................................. 52 3.3.2.3 Teacher SEL Beliefs ......................................................................................... 52 3.3.2.4 Occupational Stress ........................................................................................... 54 3.3.2.5 Teaching Efficacy ............................................................................................. 55 3.3.3 Implementation ......................................................................................................... 55 Chapter 4: Results........................................................................................................................58 Study 1 .......................................................................................................................... 58 4.1.1 Preliminary Analyses ................................................................................................ 58 4.1.1.1 Teacher Pre-Implementation Factor Data ......................................................... 59 4.1.1.2 Implementation Data ......................................................................................... 60 4.1.2 Hypothesis 1.............................................................................................................. 62 4.1.3 Hypothesis 2.............................................................................................................. 63 Study 2 .......................................................................................................................... 66 4.2.1 Preliminary Analyses ................................................................................................ 66 4.2.1.1 Teacher Factor Data .......................................................................................... 67 4.2.1.2 Implementation Data ......................................................................................... 69 4.2.2 Hypothesis 1.............................................................................................................. 71 4.2.3 Hypothesis 2.............................................................................................................. 72 x Summary of Results ...................................................................................................... 79 Chapter 5: Discussion ..................................................................................................................80 Teacher Factors and SEL Program Implementation Quality ........................................ 80 SEL Program Implementation Quality and Changes in Teacher Factors ..................... 85 Strengths and Limitations ............................................................................................. 87 Implications and Conclusion......................................................................................... 91 References .....................................................................................................................................94 Appendices ..................................................................................................................................118 Appendix A Program Evaluation Descriptions....................................................................... 118 A.1 SMART and MindUP Study ................................................................................... 118 A.2 Kindness in the Classroom Study ........................................................................... 120 A.3 Taxi Dog RCT......................................................................................................... 121 A.4 SEL and Biology Feasibility Study......................................................................... 122 Appendix B Consent Forms .................................................................................................... 123 B.1 SMART and MindUP RCT Teacher Consent Form ............................................... 123 B.2 Kindness in the Classroom RCT Teacher Consent Form ....................................... 129 B.3 Taxi Dog RCT Teacher Consent Form ................................................................... 134 B.4 SEL and Biology Feasibility Study Teacher Consent Form ................................... 140 Appendix C Measures ............................................................................................................. 154 C.1 Teacher Demographic Questionnaires ......................................................................... 154 C.2 Teacher SEL Experience Surveys ................................................................................ 156 C.3 Job Satisfaction Items................................................................................................... 164 C.4 Teacher SEL Beliefs Scale ........................................................................................... 165 xi C.5 Maslach Burnout Inventory – Educator’s Survey ........................................................ 166 C.6 Personal Teaching Efficacy Subscale .......................................................................... 168 C.7 Implementation Diaries ................................................................................................ 170 Appendix D Summaries of Extension Activity Implementation……………………………….176 D.1 SMART and MindUP Study: Lesson Extension Implementation Summary ............... 176 D.2 Kindness in the Classroom Study: Lesson Extension Implementation Summary ....... 177 D.3 Taxi Dog RCT: Lesson Extension Implementation Summary .................................... 178 D.4 SEL and Biology Feasibility Study: Lesson Extension Implementation Summary .... 179 xii List of Tables Table 1.1 Summary of implementation components…………………………………………….14 Table 1.2 Summary of two implementation frameworks………………………………………..16 Table 3.1 Study 1: Teacher demographics and SEL program information……………………...46 Table 3.2 Study 2: Teacher demographics and SEL program information……………………...48 Table 3.3 Summary of measures used in Study 1 and Study 2……………………………….....51 Table 3.4 Summary of total number of lessons in each SEL program…………………………..57 Table 4.1 Study 1: Summary of teacher pre-implementation factor data………………………..60 Table 4.2 Study 1: Summary of implementation data…………………………………………...61 Table 4.3 Summary of implementation composite scores by SEL program…………………….62 Table 4.4 Intercorrelations between teacher pre-implementation factors and total number of lessons, and implementation quality……………………………………………………………..63 Table 4.5 Summary of hierarchical regression model of teacher pre-implementation factors as predictors of implementation quality…………………………………………………………….65 Table 4.6 Study 2: Summary of teacher factor data……………………………………………..68 Table 4.7 Study 2: Summary of implementation data…………………………………………...70 Table 4.8 Study 2: Summary of paired-samples t test results between pre- and post-implementation mean scores ………………….............................................................................72 Table 4.9 Bootstrapping ANCOVA: Teacher factor adjusted mean post-implementation scores and difference for low- and high-implementation quality groups……………………………….74 Table 4.10 Mixed design repeated measures ANCOVA: Teacher factor adjusted mean post-implementation scores and difference for low- and high-implementation quality groups………76 xiii List of Figures Figure 2.1 Process model of sustained implementation by teachers…………………………….18 Figure 4.1 Change in estimated marginal means for teaching efficacy for low- and high-quality implementation groups…………………………………………………………………………..77 xiv Acknowledgements With humble gratitude, I acknowledge the traditional, ancestral, unceded territories of the Musqueam and Coast Salish Nations on which I have been able to learn at UBC’s Point Grey Campus. I extend heartfelt gratitude to my supervisor, Dr. Kimberly A. Schonert-Reichl, who has served as a guide and mentor throughout this process. She gives freely of her time, expertise, data, and words of encouragement and support. Thank you to my thesis committee members, Dr. Eva Oberle and Dr. Deborah Butler, for offering their guidance, support, and wisdom. Thank you to my fellow graduate students, especially Melvin, Misha, and Amanda, who helped me stay focused and committed to my work despite the separation and distance introduced by the global pandemic. Our weekly chats helped me stay connected to a community of scholars and friends while working and learning from home. And many thanks go to my family—my husband, Wilder, and daughter, Maja—who have offered me their unwavering support throughout this process, including moving to Canada so that I could pursue this course of study and achieve my goals. Their support buoyed my spirits, and their confidence in me inspired my resolve to persist. 1 Chapter 1: Introduction Introduction Children growing up in the 21st century are faced with unique challenges not experienced by previous generations. Firmly situated in the digital age, today’s youth must be critical consumers of information and savvy users of web applications like social media. They must develop competencies that will prepare them to be successful collaborators and creative problem solvers in occupations that do not yet exist. Moreover, with nationalist fervor rising, civil discourse declining, and the climate crisis looming, children and youth must be prepared with the knowledge, skills, and attitudes necessary to solve challenging societal and environmental problems. The current global COVID-19 pandemic has exacerbated these conditions and surfaced others including an economic crisis, worldwide movements for racial justice, and extended school closures, all of which are putting at risk the foundational safety and stability children and youth require for positive development. This generation will not find adequate preparation for the current crisis and uncertain future in academic textbooks alone; rather, children and youth need the opportunity to develop social and emotional competencies that can help them successfully navigate the challenges that lie ahead. Research suggests that quality implementation of school-based social and emotional learning (SEL) programs can help children and youth develop social-emotional competencies that will help them be successful in school and life (e.g., Durlak et al., 2011; Jones & Bouffard, 2012; Sklad et al., 2012; Taylor et al., 2017). Ensuring that these promising programs lead to positive outcomes for children and youth is of critical importance given the challenging future they are preparing for coupled with the rising prevalence in mental health disorders evident 2 among the current generation (Choi, 2018) which is projected to rise due to COVID-19 and physical distancing (Galea et al., 2020). It is troubling, then, that the SEL field has been plagued by inconsistency in program outcomes (Jones & Doolittle, 2017; Jones et al., 2019; Weare & Nind, 2011). For example, SEL program evaluations have found null or mixed effects between treatment and intervention groups (Evans et al., 2015; Jones et al., 2019; McCormick et al., 2015; Rowe & Trickett, 2018; Shriver & Weissberg, 2020; Social and Character Development Research Consortium, 2010; Weare & Nind, 2011). To understand the inconsistencies, researchers have turned to the field of implementation science, which involves the examination of factors related to successful delivery of school-based SEL programs (Durlak & DuPre, 2008), to illuminate factors affecting SEL program implementation. A meta-analysis of SEL interventions found that program effectiveness was significantly related to implementation fidelity (Weare & Nind, 2011), the extent to which a program is delivered as intended by its developers (Durlak & DuPre, 2008). Thus, the success of school-based SEL programs hinges on how well they are delivered, and research suggests that teachers, as the primary deliverers of SEL programs, play an integral role in quality implementation (Beets et al., 2008; Low et al., 2016). Preliminary research suggests that educators’ sense of professional efficacy, their attitudes and beliefs toward SEL, and their own well-being and social-emotional competence may influence the quality of their SEL implementation (Brackett et al., 2012; Collie et al., 2015; Jennings & Greenberg, 2009; Ransford et al., 2009; Reyes et al., 2012; Wanless & Domitrovich, 2015), but further research is needed to elucidate if and how these factors affect teachers’ ability to successfully implement SEL programs. To address this gap, the aim of the present research was to explore the association between teacher factors and SEL program implementation quality. The remainder of this chapter 3 describes the current context in which this research is situated. First, SEL is defined, and its significant, positive outcomes are described. Then a brief review of research on school-based SEL programs is presented to illustrate their promise to positively affect students’ school and life success. This is followed by an expanded discussion regarding the inconsistency in program outcomes. In Chapter 2 the extant literature related to the research in two areas is put forth: 1) implementation science and 2) factors related to teacher performance in general and successful SEL program implementation specifically. Chapter 3 describes the method used to conduct the two studies that comprise the research, then Chapter 4 presents the results. In Chapter 5, the results from Study 1 and Study 2 are discussed, including their strengths and limitations. Chapter 5 then concludes with a discussion of the significance of this research and directions for future research. Social and Emotional Learning Social and emotional learning (SEL) is defined as the processes through which children and youth acquire, practice, and apply social-emotional competencies in five key areas: self-awareness, self-management, social awareness, relationship skills, and responsible decision-making (CASEL, 2019). People with social-emotional competence can effectively recognize and manage emotions, feel and show care and concern for others, establish and maintain positive relationships, make responsible decisions, and set and achieve positive goals. The extant literature positions SEL as central to the healthy development and well-being of children and youth and suggests that school-based programs are effective in helping them develop these malleable skills (e.g., Durlak et al., 2011; Jones & Bouffard, 2012; Sklad et al., 2012; Taylor et al., 2017; Weare & Nind, 2011; Wiglesworth et al., 2016). After reviewing and summarizing the relevant evidence, members of the Aspen Institute’s National Commission on Social, Emotional, 4 and Academic Development (NCSEAD)—a group of leaders from education, research, business, policy, and the military convened to make recommendations for integrating social, emotional, and cognitive dimensions of learning into schools and communities based on research and promising practices—concluded that social-emotional competencies (SECs) are essential to learning, positive development, and success in school (Jones & Kahn, 2017). More recently, Shriver and Weissberg (2020) argued that we “…move beyond debate about whether schools should address students’ social and emotional development to how we can integrate effective SEL programming into the mission and daily work of all schools,” (p. 54, emphasis in original). This section discusses the research that supports assertions regarding the benefits of SEL and the effectiveness of school-based SEL programs. 1.2.1 Benefits of Social and Emotional Learning Programs SEL is recognized as a key ingredient for school and life success (Durlak et al., 2011; Jones & Bouffard, 2012). Durlak et al.’s (2011) meta-analysis included the important finding that students participating in SEL programs showed significant gains in SEL skills, attitudes, and behaviors, as well as academic achievement compared to those students who did not receive SEL programs. Educators acknowledge the benefits of SEL in schools and agree that teaching students social and emotional skills is a necessary and valuable component of their education (Atwell & Bridgeland, 2019; Bridgeland et al., 2013). Furthermore, gains from SEL programs are durable (Taylor et al., 2017) and related to economic benefits to both individuals and society at large (Belfield et al., 2015; Jones et al., 2015; Moffit et al., 2011). In the following section, research on key areas positively affected by SEL including children’s healthy development and well-being, and academic achievement; school and classroom climate; and public health is reviewed. 5 1.2.1.1 Healthy Development and Well-Being SEL has been associated with a host of outcomes related to the healthy development and well-being of children and youth, and these positive effects continue into adulthood (Hawkins et al., 2005; Jones et al., 2015; Moffitt et al., 2011). Children and youth with greater social-emotional competence can develop healthier relationships with peers and adults, behave more appropriately, and are more likely to avoid risky behaviours (Trentacosta & Fine, 2010). SEL is related to the development of a sense of purpose and other positive virtues (Elias, 2014), and to character development and decreased misconduct (Duncan et al., 2017). SEL programs have been shown to positively impact students’ general mental, emotional, and social health and well-being with small to moderate effect sizes (Durlak et al., 2011; Weare & Nind, 2011). Moderate to strong effect sizes have been found for SEL program impacts on students’ social-emotional skills and competence (Durlak et al., 2011; Weare & Nind, 2011). SEL programs also have been shown to positively affect students’ self-image and behavioral adjustment, including reducing or preventing antisocial behaviour, and preventing substance use and mental disorders (Sklad et al., 2012). These effects carry into adulthood, positively affecting adults’ workplace success and marital status, and reducing risk for incarceration and depression (Heckman et al., 2006; Moffitt et al., 2011). Recent evidence also shows that these effects endure across generations with significant differences found for offspring of individuals who received the SEL intervention as children (Hill et al., 2020). 1.2.1.2 Increased Academic Achievement Denham and Brown (2010) described numerous ways in which SEL skills (e.g., relational/prosocial skills and emotional competence skills) may share a bidirectional relationship with academic success. Indeed, a key finding of Durlak et al.’s 2011 meta-analysis 6 of universal, school-based SEL programs was an 11-percentile gain in academic achievement for participating students versus nonparticipating students. This is a welcome benefit for teachers under pressure to produce gains in students’ academic performance and provides some justification for implementing these programs in school. In addition to research demonstrating the academic benefits of SEL programs, correlational research suggests a link between social-emotional competence and academic achievement. For example, Oberle and colleagues (2014) found that students’ social-emotional competence in grade 6 significantly predicted higher scores on standardized measures of reading and math in grade 7. Other correlational research relates responsible decision-making, one domain of social-emotional competence, to academic achievement in reading, math, and science (Corcoran et al., 2018). Even before formal schooling, there is evidence that a focus on SEL can affect students’ academic achievement. Children who participated in an SEL program in an early learning setting demonstrated increased school readiness and a smoother transition to kindergarten (Wenz-Gross et al., 2018). This is an important outcome because both the transition to and performance in kindergarten form the foundation for future academic success (Schulting et al., 2005). 1.2.1.3 Positive School Climate A positive learning environment in which students feel safe, respected, connected, engaged, and supported form the basis of the conditions for learning (Garibaldi et al., 2015). These conditions are reflected in the school climate, the interactions between members of the school and wider community; school features such as structure, culture, and resources; and individuals’ experience of these interactions and features. Although typically studied separately, school climate and SEL are becoming recognized as influencing each other (Osher & Berg, 2017). SEL program implementation has been found to be associated with positive student 7 perceptions of school climate (Brown et al., 2011), higher student ratings of school bonding (Kumpfer et al., 2002), and more positive attitudes towards school (Durlak et al., 2011). Bear and colleagues (2017) sought to understand how students’ perceptions of school climate were associated with their perception of the frequency of use of schoolwide management practices including SEL approaches, punitive consequences, and praise and rewards. Using data from a large sample of students in grade 3-12 (N = 30,071), the researchers found that praise and rewards for good behaviour and teaching SEL were related positively with school climate, and punitive consequences were related negatively. Importantly, the effect size of teaching SEL was nearly twice that of using praise and rewards at the school and student level, and twice that of negative consequences at the student level (Bear et al., 2017). 1.2.1.4 Impact on Public Health The effects of SEL on students’ healthy development, well-being, and academic achievement, and school climate have been more broadly associated with benefits to public health and society-at-large. In their study of the economic value of SEL, Belfield and colleagues (2015) hypothesized “that much of the effect of education on economic outcomes and personal well-being is based on non-cognitive or social and emotional development such as effort, motivation, curiosity, empathy, caring, and the many other dimensions that can be encompassed by social and emotional skills,” (p. 8). Indeed, they found a US $11 return on investment for every dollar spent on SEL interventions demonstrating that SEL interventions are a sound investment for schools and society. Moreover, Hill and colleagues (2020) found that the cost benefits of an SEL intervention extend to the offspring of the target generation. That SEL may be related to long-term health outcomes is another exciting area of research (Greenberg et al., 2015). Researchers have only just begun to include measures of students’ salivary cortisol levels, 8 a biomarker of stress, in evaluations of SEL programs (e.g., Schonert-Reichl et al., 2015). SEL researchers advocate for its inclusion in a public health approach to education (Greenberg et al., 2015) and this is surely warranted given that SEL is touted as “a key ingredient in health, wealth, and public safety as well as a sensible policy target,” (Moffitt et al., 2011, p. 2698). School-Based SEL Programs The research reviewed above suggests that social and emotional learning (SEL) is beneficial for children’s healthy development and well-being with positive impacts enduring across the lifespan and into future generations. Given that most children and youth attend school on a regular basis, it is typically through school-based SEL programs that children learn and develop social-emotional competencies (Jones & Kahn, 2017). These programs occur in preschool, elementary, and secondary settings, as well as out-of-school time settings (Durlak et al., 2015). The evidence of effectiveness for SEL programs is based on rigorous research (Mahoney et al., 2018) with the greatest effects occurring for children with the greatest need (Jones et al., 2019; Low et al., 2015; Weare & Nind, 2011). Further evidence suggests that classroom teachers can be successful in supporting students’ SEL skill development through multiyear SEL programs (Hawkins et al., 2005), eschewing the need for specially trained staff. 1.3.1 Features of Effective Programs Effective SEL programs vary on what specific topics are covered and how they are conveyed to students, but they do share certain elements that contribute to their effectiveness. For example, successful programs demonstrate evidence of the following elements which together comprise the acronym SAFE: Sequenced activities, Active learning, Focus specific time on developing SEL skills, and Explicit defining and targeting of social-emotional skills (CASEL, 2019). Jones et al. (2018) identified the following as additional elements of effective SEL 9 programs: 1) occur in supportive contexts; 2) build adult competencies; 3) engage family and community; 4) target key behaviours and skills; and, 5) set reasonable proximal and distal goals for program outcomes. Zins and colleagues (2007) provided further guidance, identifying seven essential characteristics of effective, comprehensive SEL programming: 1) carefully planned, theory- and research-based; 2) teaches SEL skills for application to daily life; 3) addresses affective and social dimensions of learning; 4) leads to coordinated, integrated, unified programming tied to academic outcomes; 5) addresses key implementation factors to support effective SEL and development; 6) involves family and community partnerships; and, 7) includes continuous improvement, outcome evaluation, and dissemination components. 1.3.2 Program Ratings SEL programs which have undergone evaluation through randomized-controlled trials have typically found effects in the areas targeted by the program (Jones et al., 2019). These trials have helped advance the science and practice of SEL in that they identify effective strategies for developing students’ social-emotional competencies that can be implemented broadly. Results from program evaluations and other factors are considered in program ratings. For example, the Collaborative for Academic, Social, and Emotional Learning (CASEL) maintains guides which provide reviews of the vast array of SEL programs to help schools and other providers make informed decisions when selecting a program to implement in their setting. The 2015 CASEL Guide (CASEL, 2015) rates middle and high school SEL programs based on the quality of their design, implementation, and program evaluations. If rated highly, programs are labeled SELect. Notably, CASEL has identified far fewer SELect programs for middle and high school than for preschool and elementary school (CASEL, 2013; CASEL, 2015). Jones and colleagues (2017) conducted a thorough review of 25 leading SEL programs for elementary students culminating in 10 a collection of snapshots detailing each program’s evidence of effectiveness, components, curricular content, and how it compares with other programs. It is evident that effective SEL programs are plentiful and, due to increasing demand, will continue to grow in number to serve a diversity of learners in different contexts. Inconsistency in Program Outcomes Despite the mounting evidence supporting SEL program effectiveness and their role in promoting the healthy development of children and youth, there is the lingering issue of inconsistent outcomes which demands the attention of researchers, program developers, and practitioners. The Social and Character Development Research Consortium (2010) reviewed implementation and outcome data from randomized controlled trials of seven interventions and found no differences between treatment and control groups. Null or mixed effects have been reported elsewhere in the literature (Evans et al., 2015; Jones et al., 2019; McCormick et al., 2015; Rowe & Trickett, 2018; Shriver & Weissberg, 2020; Weare & Nind, 2011). Speculation regarding what accounts for these differences or absence of effects include lack of alignment between program content and outcome measures (Jones et al., 2019), insufficient applicability across diverse groups of children and adolescents (Rowe & Trickett, 2018), and school-level social processes (McCormick et al., 2015). Importantly, an overarching explanation for mixed outcomes reported in the literature is variability in implementation (Dane & Schneider, 1998; Durlak & DuPre, 2008; Wandersman et al., 2008). For example, in a meta-analysis of SEL intervention evaluations, Weare and Nind (2011) found that program effectiveness was significantly related to implementation fidelity. They stated strongly, “Interventions were only effective if they were completely and accurately implemented: this applied particularly to whole-school interventions which could be ineffective if not implemented with clarity, intensity, and 11 fidelity,” (Weare & Nind, 2011, p. i29). As the persons most often responsible for SEL program implementation, teachers have recently garnered attention in studies exploring factors that affect implementation quality, but this inquiry is in a nascent stage (Han & Weiss, 2005; Wanless & Domitrovich, 2015). The research described herein aimed to address this gap and add to the science and practice of SEL by shedding light on the nature of the relation between teacher factors and SEL implementation quality. 12 Chapter 2: Literature Review Implementation quality is significantly related to SEL program effectiveness, and teachers as the primary program deliverers play a key role in successful program implementation. Given their central importance to program effectiveness, implementation and teacher factors related to implementation are the two main foci for the literature review presented in this chapter. The chapter is comprised of four main sections. In the first section, extant literature from implementation science is reviewed including the following: implementation components and factors that affect them, implementation models, and approaches to implementation measurement. Characteristics of effective SEL implementation and recent research that considers the role of implementation in SEL program evaluations is reviewed in the second section. In the third section, the literature concerning teacher factors that affect successful implementation is reviewed. These include pre-existing factors such as teacher efficacy, stress and burnout, and beliefs; and factors affecting sustained implementation including training and coaching; student engagement and responsiveness; and changes in teachers related to program implementation. Finally, in the last section, the research questions and hypotheses for the current studies are presented. Implementation Science The success of school-based SEL programs hinges on how well they are delivered. Durlak and DuPre (2008) define implementation as: “what a program consists of when it is delivered in a particular setting,” (p. 329). Fixsen and colleagues (2005) offer a similar definition of implementation as: “a specified set of activities designed to put into practice an activity or program of known dimensions,” (p.5). Both definitions reflect the active nature of implementation, that it entails actively providing a practice or program to a group of participants. 13 Research shows that high-quality implementation is significantly associated with positive outcomes (Durlak & Dupre, 2008; Weare & Nind, 2011). Indeed, the recognized importance of implementation has elevated its study to a scientific discipline employed in intervention and program evaluations across a growing number of fields including applied social sciences, education, and health sciences (Durlak, 2015). For more than a decade, implementation science has helped researchers across disciplines understand more about how the components of implementation and factors affecting implementation quality are related to program delivery and outcomes (Durlak & Dupre, 2008). The increased interest in implementation is driven by researchers’ and practitioners’ recognition that transferring interventions proven effective in controlled trials to real world settings is a lengthy, complicated process that introduces myriad issues affecting program integrity and outcomes (Durlak & DuPre, 2008; Wandersman et al., 2008). Implementation science can help bridge the gap between science and practice—between what we know works and what we actually do—and ensure that evidence-based programs are disseminated and maintained in ways that ensure they realize their potential positive impact. 2.1.1 Implementation Components Implementation is comprised of several components. Dane and Schneider (1998) identified five components of implementation: 1) adherence: also known as fidelity, compliance, or integrity, this entails the extent to which the program is delivered as intended by the developers; 2) exposure: also known as dosage, this means how much of the program is delivered; 3) quality of delivery refers to how well program elements are delivered; 4) responsiveness, also known as participant engagement, concerns the degree to which participants are engaged in and respond to the program; and, 5) program differentiation which refers to how unique the program content and strategies are in relation to other programs. Ten years later, 14 Durlak and Dupre (2008) expanded on this list to include three more components: 1) monitoring control/comparison conditions: this refers to the attention paid to the type and quantity of the program different groups receive; 2) program reach: also known as participation rates or program scope, this concerns rates of involvement and overall representation of participants; and 3) adaptation, also known as program modification or reinvention, this refers to changes made to the original program during implementation, often to make it more relevant to a particular group of participants in a given setting. Table 1.1 summarizes the eight components of implementation. Table 1.1 Summary of Implementation Components (based on Dane & Schneider, 1998; Durlak & DuPre, 2008) Component Related term(s) Description 1. Adherence Fidelity, compliance, integrity Extent to which the program is delivered as intended by the developers (e.g., all parts of the lesson, transfer activities) 2. Exposure Dosage How much of the program is delivered (e.g., number, length, and/or frequency of lessons) 3. Delivery quality Quality How well the program elements are delivered (e.g., implementer enthusiasm, preparedness, and attitudes towards the program) 4. Responsiveness Participant engagement, participant responsiveness The degree to which participants are engaged in and respond to the program (e.g., observed level of student participation and enthusiasm) 5. Program differentiation N/A How unique the program content and strategies are in relation to other programs (e.g., check if other, similar interventions also being employed in the same context) 15 Component Related term(s) Description 6. Monitoring control/ comparison conditions N/A Attention paid to the type and quantity of the program different groups receive (e.g., check for treatment contamination or alternative services) 7. Program reach Participation rates, program scope Rates of involvement and overall representation of participants (e.g., check if the program is reaching the intended people, organization, community) 8. Adaptation Program modification, reinvention Changes made to the original program during implementation (e.g., adapting the content to be more relevant for a given audience) 2.1.2 Factors Affecting Implementation When a program or innovation is delivered in a real-world context, multiple factors can influence the eight components of implementation reviewed previously (Domitrovich et al., 2008; Durlak & Dupre, 2008; Fixsen et al., 2005). Durlak and DuPre (2008) identified 23 such factors and organized them into five categories: 1) community-level factors, including program theory or research, contextual politics, funding, and local policies and mandates; 2) staff characteristics, including their perceived need for the program, their perceived benefits of the program, their self-efficacy or level of confidence to deliver the program, and their skill base; 3) program features, including fit with the context and adaptability; 4) school/system features, including work climate, openness to change, shared vision, shared decision-making, partnering agencies, effective communication, clear roles and responsibilities, strong leadership, a program champion, and administrator support; and 5) professional development, including pre-implementation training and ongoing technical assistance. These factors, both individually and in 16 interaction, can either help or hinder implementation components, thereby affecting implementation quality. 2.1.3 Implementation Frameworks Researchers have advanced several models or frameworks for implementation that organize complex implementation components and the factors that can influence them across different stages or phases. Importantly, in their synthesis of the implementation research, Fixsen and colleagues (2005) assert that “implementation is a process, not an event,” (p. 15) and the frameworks reviewed in this section reflect this assertion. Although they differ in many respects, they all capture the nature of implementation as a collection of activities that occur over time and feature discernible stages or phases. A brief summary of two prominent frameworks is presented in Table 1.2: 1) Implementation Stages based on Fixsen et al.’s (2005) extensive review of the literature, and 2) Quality Implementation Framework (QIF), based on Meyers, Durlak, and Wandersman’s (2012) synthesis of 25 implementation frameworks. A third framework, the Process Model of Sustained Implementation by Teachers (Han & Weiss, 2005), is described in more depth as it presents as a useful model for this research. Table 1.2 Summary of Two Implementation Frameworks Framework Implementation Phases/Stages Implementation stages (Fixsen et al., 2005) 1. Exploration and adoption: survey options to find one that meets the needs of the community 2. Program installation: prepare to implement; secure staffing and resources 3. Initial implementation: start formal delivery 4. Full implementation: program implemented completely and producing forecasted outcomes 5. Innovation: adapt and change program to meet community’s needs 6. Sustainability: long-term program usage with ongoing effectiveness; responsive to changes in staffing, resources, etc. 17 Framework Implementation Phases/Stages Quality Implementation Framework (QIF; 2012) Phase one: initial considerations • Assessment: conduct assessment of needs and resources; program fit; readiness • Adaptation decisions: make decisions about if/how to adapt • Capacity-building: secure stakeholder buy-in; build capacity; recruit staff; train staff Phase two: create a structure for implementation • Create implementation teams • Develop implementation plan Phase three: ongoing implementation support structure • Ongoing implementation support strategies: technical assistance, coaching, supervision • Implementation process evaluation • Supportive feedback mechanism Phase four: improve future implementations • Reflect on the process and apply the learnings to future implementations 2.1.4 Process Model of Sustained Implementation by Teachers Hann and Weiss (2005) offer an implementation model (see Figure 1) which reflects the specific nature of teachers’ program implementation with special attention to teacher-level factors. That the model focuses on the relationship between teacher-level factors and the implementation of classroom-based mental health programs makes it a particularly useful model for this research. Similar to other implementation frameworks, this model divides implementation into phases. First is the Preimplementation Phase in which teachers and school leaders learn about the program and plan for implementation. During this phase, Han and Weiss identify school- (e.g., principal support for the program) and teacher-level factors (e.g., teachers’ self-efficacy beliefs, professional burnout, and beliefs about the program such as its compatibility with students and potential effectiveness) as preimplementation factors that can influence teachers’ motivation and readiness to implement a program. 18 Figure 2.1 Process Model of Sustained Implementation by Teachers (Han & Weiss, 2005) 19 The second phase of the Process Model of Sustained Implementation by Teachers is the Supported Implementation Phase. In this phase, teachers are supported in their initial implementation by training, consultation, and performance feedback. To increase program effectiveness, training and feedback in this phase are designed to increase teachers’ knowledge and skills, assist with appropriate adaptation, and help teachers connect their quality delivery of the program’s components with changes in student behavior. In the ongoing process of implementation, teachers’ motivation to implement is supported by a feedback cycle that features experiences of success and attribution of positive changes in students to teachers’ own quality implementation. Finally, skilled, confident teachers move into the Sustainability Phase when implementation support is not needed as teachers have the requisite ability and motivation to continue implementing the program well. Importantly, Han and Weiss (2005) emphasize that teachers’ ability to continue to produce the desired outcomes for students may be necessary to sustain their motivation to implement the program. Considering how teacher factors in the pre-implementation, supported, and sustained phases influence implementation quality, the Process model of Sustained Implementation by Teachers is a useful frame for this research. 2.1.5 Implementation Measurement As illustrated above, implementation is a complex process with many components that can be influenced by a variety of individual and contextual factors. Organizing the factors and components into frameworks featuring defined stages or phases can lend some order to the complexity, but to truly understand how implementation affects program delivery and outcomes, it needs to be measured. Durlak (2015) asserted that this adds yet another layer of complexity because “it is not yet clear what methods should be used in different circumstances to measure 20 implementation most reliably and validly, and how often such measurements should be taken because levels of achieved implementation can vary over time,” (p. 1124-1125). However, Durlak (1998) also maintained that measuring implementation is a necessary step for studying implementation. Self-reports and independent behavioral observations are cited as the two main methods used to measure implementation (Begeny et al., 2013; Durlak, 1998; Durlak & DuPre, 2008). Notably, observational methods are reported to yield more valid and reliable data (Durlak, 1998; Durlak & DuPre, 2008; Dusenbury et al., 2003), especially if inter-rater reliability is evident (Begeny et al., 2013), but they are more time-intensive (Breitenstein et al., 2010). Self-reports, on the other hand, are recognized as more practical (Breitenstein et al., 2010; Durlak, 1998; Martinez et al., 2014) and are used more often (Breitenstein et al., 2010; Dane & Schneider, 1998; Martinez et al., 2014). For example, Dane and Schneider (1998) found that 61% of the studies examining implementation effects on program outcomes they reviewed used implementer reports to measure implementation integrity. Self-report measures typically entail having implementers detail if and how much of an intervention they implemented (Breitenstein et al., 2010). Although this method is reportedly prone to biased or inflated reports of implementation integrity (Begeny et al., 2013; Breitenstein et al., 2010; Koziol & Burns, 1986; Martinez et al., 2014), research suggests that teachers’ repeated use of self-report measures improves the accuracy of their reporting (Koziol & Burns, 1986). In a meta-analysis of teacher survey methods, Desimone, Smith, and Frisvold (2010) found teacher self-reports to be strongly correlated with classroom observations of teaching practices. Similarly, Sanetti and Kratochwill (2011) found that teacher reports of treatment integrity were highly accurate when using a standardized protocol completed weekly. Together, these findings suggest that teacher self- 21 reports may be both efficient and effective means of measuring implementation, but more research is needed. While the “how” of efficient and effective implementation measurement is developing, defining what to measure is an area where there is more agreement in the field. Put simply, the quantity and quality of implementation is recognized as what needs to be measured (Durlak, 1998). In practice, however, quantity—adherence and dosage— is typically what gets measured in evaluation studies (Durlak & DuPre, 2008; Fixsen et al., 2005). Although higher levels of dosage and adherence are typically related to better outcomes, this is not universally true for every situation or outcome being assessed (Durlak, 2015). More comprehensive measures of implementation that assess quality and participant responsiveness can help yield a richer, more nuanced understanding of how implementation is related to program outcomes (Durlak, 2015). Notably, Berkel and colleagues (2011) include facilitator behaviors (fidelity, quality of delivery, and adaptation) and participant behaviors (engagement, participation, and responsiveness) in the implementation model they developed with the stated goal to support research on program implementation and outcomes. SEL Program Implementation The preceding review of the implementation science research, including its components, factors affecting the components, frameworks, and measurement, provides a useful foundation from which to explore SEL program implementation specifically. Similar themes existing in the broader implementation science literature are seen in SEL implementation literature as well. For example, the acknowledged complexity of installing and implementing programs in real-world settings is widely recognized by SEL researchers (e.g., Durlak, 2015) and an increasing number of program evaluation studies are investigating how implementation is related to program 22 outcomes. In this section, current conceptions regarding the nature of effective SEL implementation and factors influencing it are discussed, followed by a brief review of SEL program evaluations that include measures of implementation and consider its influence on program outcomes and effectiveness. 2.2.1 Effective SEL Implementation Jones, Bailey, Brush, and Kahn (2018) offer six key recommendations for effective SEL implementation which together reflect the need to go beyond mere program installation to attain the promised outcomes of evidence-based programs. Their first recommendation is to explicitly identify when to implement the program and how it fits with existing structures and routines. This seems an obvious task, but time is a common barrier to sufficient implementation especially if SEL instruction is not granted equal weight with academic subjects. Their second recommendation is to provide opportunities for students to practice SEL skills outside of lesson time which acknowledges that developing social-emotional competence requires sufficient opportunities to apply SEL skills. Related to the second recommendation, Jones and colleagues’ third recommendation is that teachers and other school staff find ways to scaffold students’ application of their skills both in and outside of the classroom. The remaining three recommendations concern teacher- and school-level factors that support effective implementation. They recommend that that school staff receive ongoing training and support to build their own knowledge and skills, as well as their social-emotional competence. The need for staff and the broader school community to feel ownership over the program is reflected in their recommendation to ensure that the program aligns with the needs of their students and school community. Finally, Jones and colleagues recommend using data to inform decisions regarding program adoption, and further to monitor implementation and outcomes. These data can include 23 data schools already collect, such as school climate data or behavior referrals. In this set of recommendations, one can find echoes of, for example, the QIF phases in that they advise tasks related to buy-in and training (phase 1), establishing structures to support implementation (phase 2), enacting support strategies like training during implementation (phase 3), and then reflecting on the process by examining the data (phase 4). In recent years, schoolwide or systemic implementation have surfaced as key approaches for effective SEL implementation. These approaches acknowledge the effect school- and system-level features can have on overall implementation quality. Oberle, Domitrovich, Meyers, and Weissberg (2016) argue that systemic implementation approaches serve to maximize the potential benefits of SEL programming, thus allowing students and schools to reap the greatest benefits of SEL initiatives. They advance a Theory of Action (ToA) as a guide for systemic SEL in schools. The School ToA specifies six activities as integral to advancing schools to systemic SEL implementation. Like Jones et al. (2018), they first identify establishing a shared vision for SEL among school stakeholders as a key activity to foster buy-in and program ownership. Second, also in line with QIF’s phase 1, they cite needs and resource assessment as an important step towards systemic SEL so that it can be efficiently aligned and coordinated with existing activities. The third activity reflects the need to focus on staff training and development similar to Jones et al. (2018) and phase 3 of QIF. Their fourth activity highlights the need to adopt and implement an evidence-based SEL program which is an important call-out in this era of a near flooding of the market with SEL programs, most of which have yet to be rigorously evaluated. The fifth activity, which entails integrating SEL into wider school practices, shares a similar goal to Jones et al.’s (2018) recommendations to enable skill transfer and application outside of lesson time. Finally, Oberle et al. (2016) underscore what others do as well, the importance of 24 collecting and monitoring implementation process and outcome data in service of continuous improvement. 2.2.2 SEL Program Evaluation and Implementation It follows that effective SEL implementation as conceived by prominent SEL researchers (e.g., Jones et al., 2018; Oberle et al., 2016) echoes broader implementation frameworks found in the implementation science literature. So, too, is the recent focus on implementation in SEL program evaluation research an extension of the direction being taken in the wider prevention and intervention evaluation research. The following briefly reviews how recent SEL program evaluations have measured implementation and analyzed it when explaining program outcomes. In an evaluation of the RULER Approach to Social and Emotional Learning, Reyes and colleagues (2012) sought to understand how the interaction between training, dosage, and implementation quality affected student outcomes of social and emotional competence. The sample for this study included teachers (N = 28) and their students (N = 812) who were assigned to the intervention condition in a larger randomized controlled trial of RULER. Both student and teacher data were collected in three waves. Wave 1 occurred prior to random assignment, Wave 2 occurred in the fall of the first year of programming, and Wave 3 occurred at the end of the first year of programming. Teacher measures included the following: teaching efficacy; training, which was measured as the total number of training and coaching sessions attended by teachers; dosage, which was captured by the teacher-reported total number of lessons taught; and implementation quality, which was assessed by coaches at the beginning and end of implementation in two areas: the extent to which teachers demonstrated an open attitude towards the program and the extent to which they delivered the lessons with high quality. Student measures included emotional literacy, social problem solving, and social competence. Through 25 their analyses, Reyes and colleagues (2012) were able to identify profiles for low-, moderate-, and high-quality implementers. They found that high-quality implementers who taught more of the lessons and attended more training had students with higher scores on all three outcome measures. Similar patterns were found for moderate-quality implementers with students achieving higher scores on some outcome measures. Conversely, low-quality implementers who attended more training and taught more lessons had students with lower scores compared to students whose teachers were moderate- or high-quality implementers. Using post-hoc analyses to investigate the findings, teaching efficacy emerged as a key factor explaining the results. There were significant differences in teaching efficacy scores between low-, moderate-, and high-quality clusters at Wave 3, and low-quality implementers scored lower in teaching efficacy than high-quality implementers. Thus, teaching efficacy is an important factor to consider when analyzing the relationship between teacher factors and SEL implementation and was included as a variable in this research. In a recent evaluation of another SEL program, implementation quality also surfaced as an important factor affecting student outcomes. Humphrey and colleagues (2018) included a comprehensive implementation and process evaluation as part of a randomized controlled trial of the Promoting Alternative Thinking Strategies (PATHS) curriculum. The sample included 23 schools (student N = 2294) in the intervention condition and 22 schools (student N = 2106) in the control condition. Student data were collected at multiple timepoints across two years of program implementation. Student outcomes measured included social skills, mental health difficulties, well-being, social support, school environment, attendance, and academic achievement. Implementation measures included objective indicators of dosage and reach, and observer-rated items measuring fidelity, quality, and participant responsiveness. Similar to Reyes et al. (2012), 26 each class/teacher was classified as low, moderate, or high for each aspect of implementation. In their analyses of data from the intervention group, they found significant associations between implementation quality and student outcomes. For example, after the first year of implementation, they found that moderate or higher levels of implementation quality and participant responsiveness were associated with significant improvements in students’ well-being compared with low levels. After the second year of the trial, they found moderate levels of dosage associated with significant increases in students’ perceptions of social support and peers compared with lower levels. Similarly, in their evaluation of the MindOut program, Dowling and Barry (2020b) found that implementation quality had a significant effect on program outcomes. In comparing student outcomes in social-emotional skills, mental health and well-being, and academic achievement between high-implementation, low-implementation, and control groups at post-intervention, they found only high levels of implementation were associated with positive program effects (e.g., reduced emotion suppression, stress, depressive symptoms, and avoidance coping, and increased social support coping). Interestingly, of the dimensions of implementation they included in their analyses—adherence, dosage, quality of delivery, and participant responsiveness—only quality of delivery was associated with a significant effect on all student outcomes. These findings lend further support to the claim that implementation can affect students’ outcomes and highlights the need to look at multiple dimensions of implementation to gain a greater understanding of what aspects of implementation may be affecting student outcomes. Participant responsiveness—participants’ response to or engagement with an intervention—has also surfaced as a dimension of implementation worthy of attention. In both Humphrey et al.’s (2018) and Dowling and Barry’s (2020b) SEL program evaluations, 27 participant responsiveness was found to be associated with effects on some student outcomes. Indeed, Dowling and Barry (2020b) found that it was the second most influential dimension behind quality of delivery. The relationship between participant responsiveness and program outcomes has been found in other SEL program evaluations as well. For example, in a recent randomized controlled trial of the Second Step SEL program, Low and colleagues (2015) found that student engagement was a stronger predictor of outcomes than program dosage which they measured as the number of lessons completed. Together with the findings reviewed above, this lends further importance to the quality of teachers’ program delivery as well as students’ response to the program itself on overall student outcomes. These factors are explored more thoroughly in the next section. Teacher Factors and SEL Implementation As discussed above, there are several research studies suggesting that high quality SEL program implementation is associated with more positive outcomes for students. Teachers are most often responsible for delivering SEL program components, thus examining how they affect implementation quality is important (Reyes et al., 2012). That teachers play an integral role in quality implementation has long been recognized (Beets, et al., 2008; Durlak & DuPre, 2008), but researchers only recently have begun to explore how teacher factors, such as motivation and readiness to deliver SEL programs, affect delivery (Wanless & Domitrovich, 2015). Perfect implementation – for example, one hundred percent adherence and dosage – is neither necessary for positive outcomes nor realistic (Durlak & Dupre, 2008). In fact, positive results have occurred at levels of about 60% and few studies have noted levels beyond 80% (Durlak & DuPre, 2008). However, as the primary program deliverers, teachers do need to be prepared to deliver SEL programs with some degree of quality. In this section, factors related to teachers’ 28 capacity for quality implementation are reviewed. The Process Model of Sustained Implementation by Teachers (Hann & Weiss, 2005) is used as an organizing framework, thus pre-implementation factors including teacher efficacy, stress and burnout, and beliefs are reviewed first. Then factors related to supported and sustained implementation are reviewed, including training and coaching; student engagement and responsiveness; and changes in teachers. 2.3.1 Teacher Pre-Implementation Factors Teachers are the intermediaries between the written lesson and what is delivered to students. Because the content flows through the teacher, it is necessarily shaped by who and how the teacher is in that moment. Thus, key factors to consider when investigating what influences implementation quality are those that are present before implementation begins. Han and Weiss (2005) identified teachers’ self-efficacy beliefs, teachers’ level of professional burnout, and teachers’ beliefs about the program as factors that can affect teachers’ motivation, readiness, and ultimately their ability to deliver a program with fidelity. Each factor is reviewed in turn below, first with discussion regarding its general influence on teachers’ performance or practice, then its specific potential influence on prevention program implementation, including SEL programs. 2.3.1.1 Efficacy The construct of teacher efficacy is rooted in Bandura’s (1977; 1997) social cognitive theory and his conception of self-efficacy which he defines as “beliefs in one’s capabilities to organize and execute the courses of action required to produce given attainments” (Bandura, 1977, p. 3). Tschannen-Moran and Woolfolk Hoy (2001) tailored the definition of self-efficacy for teachers: “A teacher’s efficacy belief is a judgment of his or her capabilities to bring about desired outcomes of student engagement and learning, even among those students who may be 29 difficult or unmotivated,” (p. 783). Teacher efficacy is a widely studied variable in education research, with a substantial body of research relating it to a variety of educational outcomes. For example, teacher efficacy has been related to student outcomes such as student motivation (e.g., Midgley et al., 1989) and students’ efficacy beliefs (e.g., Anderson et al., 1988). It has also been found to be predictive of students’ performance on achievement tests (e.g., Moore & Esselman, 1992; Ross, 1992). Teachers’ sense of efficacy is related to their classroom behaviour. For example, teachers with greater efficacy persist longer with struggling students and are less likely to criticize incorrect answers (Gibson & Dembo, 1984) whereas those with lower efficacy report more conflictual relationships with students (Hamre et al., 2012). Teachers with greater efficacy are more likely to be organized and well-planned, and demonstrate enthusiasm for teaching (Allinder, 1994). They experiment with new teaching methods and instructional materials, and look for ways to improve their practice (Guskey, 1988). Importantly, efficacious teachers tend to remain in the profession longer (Burley et al., 1991). There exist some studies that have considered teacher efficacy and SEL implementation quality. For example, research on behavioral interventions found higher efficacy to be associated with higher quality implementation (Kallestad & Olweus, 2003; Rohrbach et al., 1993). In their research examining how teachers’ psychological factors affected SEL program implementation, Ransford et al. (2009) found that teachers’ efficacy had a strong, positive correlation with implementation quality, and that among teachers with reports of high burnout, only those with high efficacy also reported higher quality implementation. Post-hoc analyses of data from Reyes et al.’s (2012) study investigating relations between SEL program implementation quality, training, and dosage on student outcomes revealed that low-quality implementers had a lower sense of teacher efficacy than high-quality implementers. Taken together, these findings suggest 30 that teachers’ sense of efficacy can affect their capacity to deliver a program as intended thus marking this construct as important to consider when analyzing the relationship between teacher factors and implementation quality. 2.3.1.2 Stress/Burnout Teaching is recognized as one of the most stressful occupations even surpassing stress levels found in the nursing profession (Greenberg et al., 2016). Greenberg, Brown, and Abenavoli (2016) identified four main sources of teachers’ stress: school organization (e.g., leadership, climate, culture), job demands, work resources (e.g., support and autonomy), personal resources, and teachers’ social-emotional competence. Periods of prolonged stress can lead to a form of extreme stress known as burnout (Maslach et al., 2001). The teaching profession has high rates of burnout (Chang, 2009) and it is recognized as a leading factor in teacher attrition (Kyriacou, 2001; Maslach, 1999). Aside from leaving the profession, teacher stress and burnout are associated with low performance, ill health/well-being, increased absenteeism; and for students, lower achievement, lower continuity for students and parents, and higher educational costs (Greenberg et al., 2016). Teacher stress and burnout can negatively affect teachers’ relationships with students (Braun et al., 2019; Herman et al., 2018; Jennings & Greenberg, 2009). For example, Geving (2007) found that negative teacher behaviours due to stress (e.g., interrupting students while they were talking or showing a disregard for school policy) prompted negative student behaviours including school property damage, peer criticism, and disrespectful communication with the teacher. The influence of teacher stress on students is also found at biological levels. Oberle and Schonert-Reichl (2016) identified a stress contagion effect, noting higher levels of the stress hormone cortisol in students whose teachers reported higher levels of burnout. Indeed, recent research suggests that students notice their teachers’ 31 stress levels and relate it to lower levels of social-emotional competence (Oberle et al., 2020). Also noteworthy is the negative association between teacher efficacy and stress and burnout (Brouwers & Tomic, 1999) which suggests that both are related to teachers’ ability to deliver programs or practices effectively (Han & Weiss, 2005). The relationship between teacher stress and burnout, and effective SEL implementation has been both theorized and demonstrated in experimental research. For example, the prosocial classroom model (Jennings & Greenberg, 2009) suggests that teachers’ social-emotional competence is related to their ability to effectively implement SEL programs. Jennings and Greenberg (2009) posit that teachers with higher levels of stress and burnout are less able to demonstrate social-emotional competence and support the social-emotional development of their students through effective modeling and reinforcement, thus compromising effective SEL implementation. Experimental evaluations of SEL interventions support this claim, with teachers’ social-emotional competence found to be associated with (Solomon et al., 1997) or predictive of SEL implementation quality (Buss & Hughes, 2007). Indeed, Collie, Shapka, Perry, and Martin (2015) found that teachers with SEL-thriver profile characteristics (high comfort with and commitment to SEL, and highly supportive SEL school culture) had the lowest stress and highest job satisfaction. Further, Ransford et al. (2009) found that teachers with higher levels of stress were less willing and able to use evidence-based SEL practices. Given its relationship with teacher performance in general, and SEL implementation quality specifically, teacher stress surfaces as another important construct to consider in analyses of teacher factors and SEL implementation quality. 32 2.3.1.3 Beliefs Teachers’ beliefs are widely acknowledged as significantly influencing their practice (Buehl & Beck, 2015; Pajares, 1992). They are regarded as one of the most important constructs for teacher education (Kagan, 1992; Pajares, 1992). Beliefs are defined simply as “an individual’s judgment of the truth or falsity of a proposition,” (Pajares, 1992, p. 316). For teachers, they can include implicit or explicit beliefs about the self, knowledge, pedagogy, and students (Buehl & Beck, 2015; Fives & Buehl, 2012). Fives and Buehl (2012) identify three functions of teachers’ beliefs: 1) filters for interpreting experiences and content; 2) frames to conceptualize or define a problem or task; and, 3) guides for action. Research findings suggest that teachers’ beliefs significantly influence their practice and are difficult to change (Buehl & Beck, 2015; Kagan, 1992; Lumpe et al., 2000) and that they are not always consistent with literature about best practices in teaching (Battista, 1994; Fetters et al., 2002; Haney et al., 1996). The extant literature examining the relationship between teachers’ beliefs and their practice looks mainly at teachers’ beliefs and math instruction (e.g., Stipek et al., 2001), science instruction (e.g., Milner et al., 2012), and technology integration (e.g., Kim et al., 2013). However, teachers’ beliefs are gaining attention as an important factor affecting SEL program adoption and implementation. For example, teachers who do not believe SEL to be important or that they should be responsible for SEL instruction may not be motivated to implement a program well (Kallestad & Olweus, 2003). Moreover, if teachers believe that students’ SEL skills are inborne and unalterable, they may be less likely to support SEL program implementation (Kress & Elias, 2006). Jennings and Frank (2015) expand on the importance of teachers’ beliefs for quality SEL program implementation: 33 Within the domain of SEL, these implicit beliefs may be more powerful than is the case with traditional content-based instruction. It is critical that effective SEL PD anticipate and challenge common implicit beliefs that may interfere with the delivery of SEL programs. For example if a teacher believes that talking about emotions in the classroom setting is inappropriate, it will be very difficult for him or her to deliver the curriculum effectively and model the naming of specific emotions. (p. 429) Schonert-Reichl and Weissberg (2014) also emphasize the importance of paying attention to teachers’ beliefs in their discussion about findings regarding characteristics of high-quality versus low-quality implementers of an SEL program which showed the latter group as significantly affected by their beliefs: These findings underline the importance of not only considering training and program fidelity when examining effective SEL program implementation, it is also critical to take into account teachers’ beliefs and attitudes about an SEL program and their teaching efficacy when assessing the influence of implementation on students’ outcomes. (p. 944) Teachers’ SEL beliefs are related to SEL implementation in a number of ways. They have been found to be related to teachers’ confidence to implement SEL programs, as well as their openness to SEL programs in general (Brackett et al., 2012). Teachers who believe SEL to be effective and valuable are more likely to deliver more of the curriculum (Beets et al., 2008) and to model, coach, and scaffold SEL outside of formal lesson time (Zins et al., 2014). For example, Ransford et al. (2009) found teachers’ SEL beliefs to be related to their use of supplementary SEL materials beyond the lesson, and to overall implementation dosage and quality. In research with preschool teachers, Zinsser and colleagues (2014) found that teachers who were highly supportive of SEL paid more attention to integrating SEL into daily classroom 34 activities and interactions with students. Indeed, they found that teachers who were highly supportive of SEL were more likely to and more skilled at going beyond specific SEL curricula by modeling, scaffolding, and coaching SEL skills whereas moderately supportive teachers relied heavily on pre-packaged lessons and activities. To better understand the construct of teachers’ SEL beliefs, Brackett and colleagues (2012) developed the Teacher SEL Beliefs Scale, a measure that examines the construct across three domains: comfort (sense of confidence in teaching SEL), commitment (desire to participate in SEL training and teaching), and culture (perceptions of schoolwide support for SEL). They found that teachers with more positive SEL beliefs rated an SEL program as more effective and valuable. Collie et al. (2015) employed the measure in an effort to group teachers based on their SEL beliefs. They identified three distinct profiles: the SEL-thriver with high comfort, commitment, and culture; the SEL-advocate with high comfort and commitment, and low culture; and the SEL-striver with high commitment, and low comfort and culture. Among the three profiles, the SEL-thrivers had the most positive SEL beliefs overall. Teachers’ beliefs are clearly related to their practice in general and to SEL implementation specifically. Jennings, Frank, and Montgomery (2020) summarize its importance: “teachers’ dispositions and beliefs may have a more powerful impact on SEL than on other, more content-based instruction,” (p. 132). As discussed previously, teacher efficacy and stress also affect teachers’ capacity to deliver SEL programs with fidelity. It is noteworthy that these pre-implementation factors can also affect each other. For example, Collie, Shapka, and Perry (2012) investigated the relationship between teachers’ SEL commitment and comfort and their experiences of stress, efficacy, and job satisfaction. They found that these variables significantly predicted teachers’ sense of stress, efficacy, and job satisfaction, with SEL comfort 35 being the strongest predictor. Their findings together with the research reviewed above point to the importance of considering teachers’ SEL beliefs, stress, and efficacy as important pre-implementation factors that can influence teachers’ effective SEL implementation. 2.3.2 Teacher Supported/Sustained Implementation Factors In addition to teacher factors that exist prior to program implementation, the presence or absence of factors during implementation can affect implementation quality and sustainability. Han and Weiss (2005) identify these as supported and sustained implementation factors, with the former supporting initial quality implementation and the latter supporting sustained implementation over the long term. The following reviews literature related to both types of implementation factors: initial training and ongoing coaching; student engagement and responsiveness; and changes in teacher factors. 2.3.2.1 Initial Training and Ongoing Coaching The body of research pertaining to teacher training and development is vast, therefore the following review focuses on how training and coaching are related to SEL implementation and outcomes. In general, is it widely recognized that high-quality teacher training and professional development are related to teachers’ ability to deliver SEL programs with fidelity (Durlak, 2015; Fixsen et al., 2005; Jennings & Frank, 2015). Although teacher readiness is influenced by pre-implementation factors as reviewed above, research suggests that these factors are malleable and responsive to training and coaching (Wanless et al., 2015). For example, Wanless et al. (2015) analyzed the relationship between teacher pre-implementation factors (teachers’ observed emotional support, teacher-rated use of intervention practices, teacher-rated self-efficacy, teacher-rated collective responsibility, education level, and years of experience) and observed fidelity of implementation of the Responsive Classroom approach two years following initial 36 training. While they found no direct relationship between teachers’ pre-implementation factors and implementation fidelity, they did find that teachers’ engagement in initial training mediated the relation between observed emotional support and later observed implementation fidelity. This suggests that effective training can help change teachers’ readiness to deliver programs with fidelity. Moreover, teachers’ uptake of training and implementation quality are related to more positive outcomes for their students (Reyes et al., 2012). Beyond initial training, ongoing coaching is often employed to ensure teachers’ quality implementation (Joyce & Showers, 1995). Wehby, Maggin, Partin, and Robertson (2011) studied associations between three teacher-factors on their level of implementation of the Good Behavior Game, an evidence-based classroom management strategy. The predictors included teacher-coach alliance (perceptions of the quality of the teacher-coach relationship), teaching ratings of the intervention’s social validity, and teacher burnout. In the model with all predictors, the teacher-coach alliance was found to have the largest relation with implementation. Interestingly, the teacher-coach alliance was found to mitigate the effects of teacher burnout. In other research examining teachers and implementation of the Good Behavior Game and PATHS, an SEL program, the researchers theorized that higher levels of coaching support accounted for teachers’ increased usage of program strategies (Domitrovich et al., 2015b). These findings are in line with the Process Model of Sustained Implementation (Han & Weiss, 2005) which emphasizes teaching training and ongoing support as integral to implementation sustainability, and suggest that these factors warrant attention when analyzing teachers and SEL program implementation quality. 37 2.3.2.2 Student Engagement and Responsiveness An intriguing factor that can affect teachers’ implementation quality is to what extent their students are engaged in and responsive to the intervention. Participant responsiveness is a recognized component of implementation (Dane & Schneider, 1998; Durlak & DuPre, 2008) although it has received less attention in the research compared to other components such as dosage and adherence (Durlak & DuPre, 2008). Berkel et al. (2011) proposed that participant responsiveness mediates the effect of teacher implementation on program outcomes. Taken further, Han and Weiss (2005) proposed that the extent to which teachers attribute positive changes in their students to the program and their efficacious delivery of it affects their motivation to sustain quality implementation. When examined in relation to SEL program implementation and outcomes, student engagement surfaces as a key factor (e.g., Low et al., 2014; Ringwalt et al., 2009). For example, Low, Smolkowski, and Cook (2016) performed latent class analysis of implementation and outcome data from a randomized controlled trial of the Second Step SEL program to identify latent classes of implementation and predict student outcomes. They found three classes that best fit the data including high-quality, low-engagement, and low-adherence classes. Analyses revealed only the low-engagement class as significantly associated with poor student outcomes when compared with the high-quality class. In their evaluation of the impact of implementation fidelity on student outcomes for the Life Skills Training program, Vroom, Massey, Yampolskaya, and Levin (2020) included participant responsiveness as one of three core elements of implementation. Multilevel modeling found significant associations between participant responsiveness at the classroom level and students’ SEL skills at posttest. This research suggests that participant responsiveness can affect program implementation and 38 outcomes, thus in analyses of teachers and program implementation, it is important to include participant responsiveness as an element of implementation quality. 2.3.2.3 Change in Teacher Factors The research reviewed above focused on factors related to teachers’ capacity to deliver SEL programs well, but researchers are also starting to explore how SEL program implementation affects teachers directly, and whether these changes are related to SEL implementation quality and sustainability. SEL program implementation as a pathway to improved teacher outcomes is suggested by the Prosocial Classroom Model (Jennings & Greenberg, 2009). Indeed, teachers who implement SEL programs report increased efficacy and reduced burnout (e.g., Ransford et al., 2009) and improved relationships with students, colleagues, and parents (Cain & Carnellor, 2008). They also have lower job-related anxiety and depression (Tyson et al., 2009) and higher quality interactions with students (Castillo et al., 2013). Domitrovich and colleagues (2016) reported similar findings regarding teachers’ reduced experience of burnout and improved efficacy following SEL program implementation. Importantly, they found that social-emotional content and practices were necessary to attain these results. However, it is not yet known whether it is simply due to exposure to the intervention content, its impact on students, related coaching, or a combination of any of these that explains the associated impact on teachers. Therefore, considering the effects of SEL program implementation on teachers and how that relates to implementation quality is an important area for further research. The Present Studies The preceding provided a review of the literature in two main areas: implementation science and teacher factors related to implementation quality. The literature from the field of 39 implementation science revealed the complexity of implementation with its myriad components (e.g., adherence, dosage, participant responsiveness, delivery quality) and factors that can affect them (e.g., contextual factors, staff characteristics, program features); however, implementation models or frameworks have been advanced that help order the many implementation components and factors affecting them into phases or stages to guide implementation efforts. Extant literature exploring the “what” and “how” of implementation measurement exposed a gap between the science and practice. Researchers agree that multiple dimensions of implementation should be considered (Durlak, 1998), but fidelity and dosage are typically what is reported in the literature (Durlak & DuPre, 2008; Fixsen et al., 2005). Moreover, although self-reports are the method most frequently used to measure implementation (Dane & Schneider, 1998), their validity and reliability are in question (Begeny et al., 2013). However, some evidence suggests repeated use by implementers can bolster their effectiveness (Koziol & Burns, 1986). Using the Process Model of Sustained Implementation by Teachers (Han & Weiss, 2005) as a guiding framework, the literature regarding teacher factors that can affect SEL program implementation, including pre-existing factors such as efficacy, stress/burnout, and beliefs, and factors affecting initial and sustained implementation such as training, coaching, student engagement, and changes in teachers related to program implementation was explored. The literature reviewed suggests that the above teacher factors are related to teachers’ ability to implement school-based SEL programs effectively. Indeed, higher quality implementation is related to increased outcomes compared to lower quality implementation. Thus, further exploration of these factors and their relationship with multiple dimensions of implementation is warranted, and this is the aim of the research described below. 40 As the primary deliverers of SEL interventions, teachers play a pivotal role in effective and successful implementation (Beets et al., 2008; Low et al., 2016) and, therefore, the potential for positive outcomes for children and youth. However, there is a dearth of research investigating how teacher factors are related to their ability to deliver SEL programs well despite research suggesting that teacher factors affect implementation quality more than organizational factors (Domitrovich et al., 2015b). The extant literature reviewed above points to the following teacher factors as warranting further investigation pertaining to their relationship with quality SEL implementation: teacher efficacy (Ransford et al., 2009; Reyes et al., 2012), job satisfaction (Collie et al., 2012), stress and social-emotional competence (Jennings & Greenberg, 2009; Ransford et al., 2009), and SEL beliefs and experience (Brackett et al., 2012; Collie et al., 2015). The research described below addressed this gap by exploring how teacher factors including stress, efficacy, job satisfaction, and SEL beliefs and experience are related to SEL implementation quality, and how SEL implementation is related to changes in teacher factors. The present study also examined changes in teacher factors in relation to SEL program implementation. Findings from this research will further elucidate whether and how teacher factors are related to successful SEL program implementation and help direct further research and the development of interventions that target teachers’ readiness to implement SEL programs and support sustained implementation. It may also help advance the practice of SEL program implementation as it highlights aspects of readiness that schools could address prior to implementation to increase the likelihood of positive, durable outcomes for students. Data from four experimental evaluations of SEL programs were used to conduct two studies. The following presents the research questions and related hypotheses for each study in turn. 41 2.4.1 Study 1 Using secondary data from four SEL program evaluations, Study 1 explored how teachers’ SEL beliefs, SEL experience, and job satisfaction were related to SEL program implementation quality. The following questions/hypotheses were put forth: What is the nature of the relationship between teachers’ pre-implementation factors (SEL Beliefs: Comfort, Commitment, Culture; job satisfaction; and SEL experience) to overall SEL program implementation quality? In Study 1, it was hypothesized that 1) teachers’ SEL Beliefs and experience, and job satisfaction would be positively correlated with implementation quality; and 2) teachers’ SEL Beliefs, SEL experience, and job satisfaction would predict greater implementation quality, after taking into account the total number of lessons in SEL programs. 2.4.2 Study 2 Using a sub-set of data from three of the four SEL program evaluations used in Study 1, Study 2 investigated changes in teacher factors, including SEL Beliefs, occupational stress, teaching efficacy, and job satisfaction following SEL program implementation in relation to implementation quality. The following research question was addressed: What is the relationship between changes in teachers’ pre-implementation factors (job satisfaction, efficacy, occupational stress, SEL Beliefs) following SEL program implementation to levels of implementation quality (low- versus high-quality implementation)? It was hypothesized that 1) SEL implementation would be associated with increases in teachers’ SEL Beliefs, efficacy, and job satisfaction, and decreases in occupational stress (i.e., decreased emotional exhaustion and depersonalization, and increased personal accomplishment); and, 2) there would be significant differences in teachers’ post-implementation teacher factor scores between low- and high-quality implementation groups, 42 with teachers in the high-quality SEL implementation group experiencing more positive changes than teachers in the low-quality SEL implementation group. 43 Chapter 3: Method Participants Study 1 and Study 2 used secondary data collected in four experimental evaluations of social and emotional learning (SEL) programs. Data for these studies were drawn from surveys completed by teachers who were assigned to the intervention condition and who implemented the SEL program in each evaluation. For all participating teachers, this was the first time that they had implemented the SEL program. Study 1 used demographic and pre-implementation teacher data, and program implementation data collected in all four SEL program evaluations. Study 2 used a sub-set of Study 1 data including demographic, pre- and post-implementation teacher data, and program implementation data collected in three of the four SEL program evaluations. A brief description of each program evaluation is provided below. A detailed description of each evaluation study is provided in Appendix A: Program Evaluation Descriptions. • Data set A: SMART (Stress Management and Relaxation Techniques) in Education and MindUP Randomized Controlled Trial (RCT): The goal of this study was to evaluate the single and joint effects of the MindUp program for children and the SMART in Education program for teachers on student and teacher outcomes. Data collected from teachers (N = 8) who delivered the MindUP program were used in Study 1 and Study 2. Pre-implementation data were collected in the fall of 2011. This was followed by a one-day training for teachers on the MindUP program provided by a certified MindUP trainer. Program implementation data were collected between February and June 2012. Post-implementation data were collected in June 2012. 44 • Data set B: Kindness in the Classroom Study: The aim of this study was to evaluate the effectiveness of the Kindness in the Classroom (KitC) Curriculum, an SEL program for children that focuses on promoting kindness and well-being. Data collected from teachers (N = 14) who delivered the KitC program were used in Study 1 and Study 2. Pre-implementation data were collected between February to March 2014. Prior to implementing the KitC program, teachers received a one-day training session in which they learned about program implementation from certified KitC program trainers. They were also provided with weekly support for program implementation from a certified KitC program provider. Program implementation data were collected between January and June 2015. Post-implementation data were collected in June 2015. • Data set C: Taxi Dog RCT: This study examined the effectiveness and feasibility of the Taxi Dog Educational Curriculum, an SEL program for kindergarten to grade 3 students. The program focuses on promoting social-emotional competence, resilience, and school success. Data collected from teachers (N = 9) who delivered the Taxi Dog program were used in Study 1. Pre-implementation data were collected between December 2015 to January 2016. Teachers completed online training prior to implementation that provided them with an overview of the Taxi Dog program, the links between SEL and the Taxi Dog curriculum, an overview of each of the Taxi Dog Lessons, a step-by-step guide for delivery of each of the lesson plans, and lesson plans, videos, and materials. In addition, teachers received implementation support from a certified Taxi Dog program provider during program implementation. Program implementation data were collected from April to June 2016. Post-implementation data were collected in June 2016. These data were unavailable for analysis due to COVID-19 related pandemic closures in the building in which the data were located. 45 • Data set D: SEL and Biology Feasibility Study: The goal of this study was to evaluate the effectiveness of the MindUP Program for middle school students, an SEL program that focuses on promoting social and emotional skills such as empathy and optimism. Data collected from middle school teachers (N = 2) who delivered the MindUP program were used in Study 1 and Study 2. Pre-implementation data were collected in March 2017. This was followed by a one-day training on the MindUP program provided by a certified MindUP program trainer. Program implementation data were collected between April and June 2017. Post-implementation data were collected in June 2017. All teachers who consented to participate in the above studies completed baseline assessments which asked them questions about their demographics, teaching background, stress, teaching beliefs, perceived efficacy, SEL experience, and well-being. Following, teachers were randomly assigned to either a control or intervention condition. All teachers in the intervention conditions completed implementation diaries detailing their use of the SEL program being evaluated. After program completion, teachers answered the questions in the baseline assessment again. In each of the studies, teachers also provided ratings of their students’ social, emotional, and behavioural competencies. Data regarding student-level outcomes were not used in Study 1 or Study 2. Study 1 included 33 teachers who participated in one of the four original studies, but one participant was removed because they only implemented one third of the SEL program (N = 32). The average age of participants was 43.48 years (SD = 10.21) with a range from 27.42 to 66.93. The average years of teaching experience was 15.6 years (SD = 10.34) with a range from 1.5 to 43 years of teaching experience. Table 3.1 provides further demographic and SEL program 46 information including participant gender, race/ethnicity, education level, program taught, and grade level taught. Table 3.1 Study 1: Teacher Demographics and SEL Program Information Variable n Percentage Gender Female 26 81.3 Male 6 17.4 Race/Ethnicity European 19 59.4 Asian 10 31.3 Other: Canadian 1 3.1 Missing 2 6.3 Education level Bachelor’s degree 10 31.3 Post-Baccalaureate degree 8 25 Graduate degree 8 25 Other 3 9.4 Missing 3 9.4 SEL program Kindness in the Classroom 14 43.75 MindUP 2012 8 25 MindUP 2017 2 6.25 Taxi Dog 8 25 Grade level taught Primary (K-3) 8 25 Intermediate (4-7) 24 75 47 Study 2 included 24 teachers who participated in one of three original studies; however, one participant had too much missing data and so was removed (N = 23). Thus, this sample is a subset of participants included in Study 1. The average age of participants was 42.82 years (SD = 8.02) with a range from 30.71 to 61.13. The average years of teaching experience was 14.74 years (SD = 7.51) with a range from 6 to 39 years of teaching experience. Table 3.2 provides further demographic and SEL program information including participant gender, race/ethnicity, education level, program taught, and grade level taught. 48 Table 3.2 Study 2: Teacher Demographics and SEL Program Information Procedure 3.2.1 Data Collection Prior to data collection in each of the aforementioned studies, ethics approval was obtained from The University of British Columbia’s Behavioural Research Ethics Board Variable n Percentage Gender Female 19 82.6 Male 4 17.4 Race/Ethnicity European 17 77.3 Asian 4 18.2 Other: Canadian 1 4.5 Missing 1 4.5 Education level Bachelor’s degree 8 34.8 Post-Baccalaureate degree 4 17.4 Graduate degree 7 30.4 Other 2 8.7 Missing 2 8.7 SEL program Kindness in the Classroom 13 56.5 MindUP 2012 8 34.8 MindUP 2017 2 8.7 Grade level taught Primary (K-3) 0 0 Intermediate (4-7) 23 100 49 (BREB), and then from school districts invited to participate in the research. Next, teachers were recruited via various methods such as flyers or principal invitation. Consent forms were also completed by each participant. Appendix B: Consent Forms includes copies of consent forms used in each study. All teachers who consented to participate completed baseline assessments which asked them questions about their demographics, teaching background, stress, teaching beliefs, perceived efficacy, SEL experience, and well-being. The amount of time taken to complete the questionnaire varied between studies from 20 to 40 minutes. Then teachers were randomly assigned to control or intervention conditions. All teachers in intervention conditions completed implementation diaries detailing their use of the program being evaluated. The implementation diaries required between five and 15 minutes per lesson to complete. After program completion, teachers answered the questions in the baseline assessment again and rated their students’ social, emotional, and behavioural competencies. The following briefly describes the program evaluation context and remuneration made to participants in each study. • Data set A: SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT: Participants were recruited from one large, urban school district in Western Canada. Teachers received free curriculum and training, a CD with guided mindfulness practices, and a small honorarium ($25). They also received half-day release with a teacher-on-call (TOC) during data collection periods. • Data set B: Kindness in the Classroom Study: Participants were recruited from one large, urban school district and one medium, regional district in Western Canada. Teachers received free curriculum and training, and a small honorarium ($50) for their participation. They also received half-day release with a teacher-on-call (TOC) during data collection periods, and for training and booster training sessions. 50 • Data set C: Taxi Dog RCT: Participants were recruited from one public school located in a large urban school district in Western Canada. Teachers received free curriculum and training, and a small honorarium ($50) for their participation. They also received half-day release with a teacher-on-call (TOC) during data collection periods. • Data set D: SEL and Biology Feasibility Study: Participants were recruited from one middle school located in a medium regional school district in Western Canada. Teachers received free curriculum and training, and a small honorarium ($50) for their participation. They also received half-day release with a teacher-on-call (TOC) during data collection periods. Measures Teachers in all four studies completed a battery of measures with regard to their demographics, teaching experience, experience with SEL, beliefs about teaching and SEL, perceptions of teaching efficacy, stress, job satisfaction, life satisfaction, and well-being. Data for Study 1 were collected from measures used across all four of the teacher questionnaires in the original studies that pertained to the research questions, and data for Study 2 were collected via three of the teacher questionnaires. Specifically, Study 1 used data regarding teachers’ demographics, SEL experience, job satisfaction, SEL beliefs, and implementation. Study 2 used data regarding teachers’ demographics, SEL beliefs, occupational stress, teaching efficacy, job satisfaction, and implementation. Table 3.3 provides a summary of the measures used in each study. This is followed by descriptions of each measure including sample items, scoring, and reliability statistics. 51 Table 3.3 Summary of Measures Used in Study 1 and Study 2 Study 1 Measures Study 2 Measures Demographics questionnaire Demographics questionnaire Implementation Diaries Implementation Diaries Teacher SEL Beliefs Scale Teacher SEL Beliefs Scale Job Satisfaction (1 item) Job Satisfaction (1 item) SEL Experience (composite of 2 items) Maslach Burnout Inventory-Educators’ Survey Personal Teaching Efficacy Subscale 3.3.1 Teacher Demographics The teacher demographic data employed in Studies 1 and 2 included age, gender, race/ethnicity, years of teaching experience, and highest level of education obtained. Examples of questions asked on the questionnaire included: What is your birthdate? What is your gender? What is your ethnic background? How many years have you worked as a teacher? What is your highest level of education? See Appendix C.1: Teacher Demographic Questionnaires for examples from each study. 3.3.2 Teacher Factors 3.3.2.1 SEL Experience Information regarding teachers’ experience with SEL was collected via surveys (see Appendix C.2: Teacher SEL Experience Surveys). Items addressed how much training teachers have had in a specific program or in SEL generally, if teachers had experienced a program themselves, if teachers had any special skills that promote SEL, and if they employed any stress management techniques. Previous research (Dusenbury et al., 2005) suggests that teachers with 52 greater understanding of the underlying concepts of a program and more years teaching a program are more adherent, thus item 2, “Have you ever been trained in/facilitated a program designed to promote children’s social and emotional learning, mental health or mindfulness?” and item 4 (item 5 in data set D), “Have you participated in or attended a training in any special coursework/workshops in social and emotional development or mindfulness education that applies to your current teaching position?” were used in this research to capture SEL experience. Data from data sets A, B, C, and D were used in Study 1 and converted to a scale from 0-2 where 0 = no on both items, 1 = one yes and one no, and 2 = yes on both items. 3.3.2.2 Job Satisfaction Participants responded to two items from a modified Teacher Stress Inventory (TSI) measure (Fimian, 1984; see Appendix C.3) in order to ascertain their level of satisfaction with their teaching job. The first item asked teachers to rate how satisfied they are with their present teaching job using a 4-point Likert-type scale (1= not at all satisfied, 2 = not very satisfied, 3 = satisfied, 4 = very satisfied). The second item asked teachers to respond with “no,” “yes,” or “not sure” to the question, “Knowing what you know now about the job of being a teacher, if you had to decide all over again to take this job, would you?” Data collected from item one in data sets A, B, C, and D were analyzed in Study 1 and Study 2. This is in line with previous research that measured job satisfaction with one item for portions of the analyses (Collie et al., 2015). 3.3.2.3 Teacher SEL Beliefs This measure includes 12 items that assess teachers’ beliefs about social and emotional learning (see Appendix C.4: Teacher SEL Beliefs Scale). The measure was developed by Brackett and colleagues (2012) in a two-phase study. In the first phase, exploratory and confirmatory factor analyses yielded three scales related to teachers’ comfort with teaching SEL, 53 their commitment to learning about SEL, and their perceptions regarding if their school culture supports SEL. Evidence of concurrent and predictive validity of the scales was secured in phase two. The scales use a 5-point Likert scale to assess the extent to which participants agree or disagree with each item (1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree). Items include statements such as “All teachers should receive training on how to teach social and emotional skills to students,” “I feel confident in my ability to provide instruction on social and emotional learning,” and “The culture in my school supports the development of children’s social and emotional skills.” The same measure was employed across all four program evaluations. Data collected from data sets A, B, C, D were analyzed in Study 1. Data collected from data sets A, B, and D were analyzed in Study 2. Items belonging to each scale – Comfort, Commitment, Culture – were considered separately in analyses in both Study 1 and Study 2 which is in line with the measure developers (Brackett et al., 2012) and previous research (Collie et al., 2011). Scoring for the Study 1 included averaging all items on each subscale to produce overall averages for SEL Comfort, Commitment, and Culture. Reliability for the subscales in Study 1 were satisfactory: Comfort (α = .840); Commitment (α = .867); Culture (α = .656). These alphas are comparable to those reported by the measure developers (Brackett et al., 2012) and in previous research (Collie et al., 2011). Scoring for Study 2 was similar to Study 1 but overall averages scores needed to be calculated for both before and after implementation (i.e., pre-and post-implementation). Reliability for the subscales in Study 2 ranged from satisfactory to excellent at pre- and post-implementation: Comfort (α = .83; .793); Commitment (α = .918; .660); Culture (α = .643; .695). These alphas are comparable to those reported by the measure developers (Brackett et al., 2012) and in previous research (Collie et al., 2011). 54 3.3.2.4 Occupational Stress The Maslach Burnout Inventory (see Appendix C.5: Maslach Burnout Inventory – Educator’s Survey) assesses the frequency of burnout symptoms experienced by teachers over the past few months in three areas: emotional exhaustion, depersonalization, and reduced personal accomplishment (Maslach et al., 2001). Participants use a 7-point Likert-type scale to rate the frequency of their burnout symptoms (1 = never, 2 = a few times, 3 = once a month or less, 4 = a few times a month, 5 = once a week, 6 = a few times a week, 7 = every day). Participants are prompted with the following question: “Using the rating scale above think about the school year and answer the following questions. How often do you…” then respond to 23 items using the scale described above. Data collected from data sets A, B, and D were used in analyses for Study 2. This measure was not used in the Taxi Dog RCT (data set C); therefore it was not included in Study 2. Items belonging to each scale – emotional exhaustion, depersonalization, and personal accomplishment – were considered separately in analyses per recommendations by one of the measure developers (Maslach, 1999). Scoring for Study 2 included averaging all items on each subscale to produce overall averages for emotional exhaustion, depersonalization, and personal accomplishment before and after implementation. Reliability for the subscales in Study 2 ranged from acceptable to excellent at pre- and post-implementation: emotional exhaustion (α = .845; .917); depersonalization (α = .435; .882); personal accomplishment (α = .959; .970). These alphas are comparable to those reported by the measure developers (Maslach et al., 1997) and in previous research (e.g., Brackett et al., 2012). 55 3.3.2.5 Teaching Efficacy Midgley and colleagues (2000) created the 8-item measure Personal Teaching Efficacy Subscale (PALS) (see Appendix C.6: Personal Teaching Efficacy Subscale) modeled after other scales (see Gibson & Dembo, 1984; Woolfolk et al., 1990) to assess two aspects of teacher efficacy: 1) personal teaching efficacy: a teacher’s belief about his or her ability to make a difference in a student’s life; and, 2) general teaching efficacy: a teacher’s general belief about teachers’ ability to make a difference in the lives of students. Participants use a 5-point Likert-scale to rate to what extent they agree or disagree with the statements provided (1 = strongly disagree, 2 = disagree, 3 = unsure, 4 = agree, 5 = strongly agree). Statements include: “I am certain that I am making a difference in the lives of my students,” and “There is little I can do to ensure that all my students make significant progress this year.” Data collected from this measure in data sets A, B, and D were used in Study 2. This measure was not used in the Taxi Dog RCT (data set C); therefore it was not included in Study 2. Scoring for Study 2 included averaging all items to produce overall averages for teaching efficacy before and after program implementation. Reliability was in the acceptable range at pre- (α = .694) and post-implementation (α = .445), but lower than what has been reported by the measure developers (α = .71; Midgley et al., 2000). 3.3.3 Implementation Teachers completed implementation diaries after teaching every lesson in all four program evaluations. Specific information recorded in the diaries varied across the four data sets (see Appendix C.7: Implementation Diaries). All four evaluations collected data regarding the following four indicators of implementation quality (Durlak & DuPre, 2008): 1) dosage (e.g., how many lessons were taught), 2) adherence (e.g., to what extent all lesson components were 56 delivered), 3) transfer (e.g., to what extent extension or additional material were used outside of lesson time), and 4) student engagement (teacher ratings of students’ engagement on a 5-point Likert-scale ranging from 1 = not engaged to 5 = very engaged). Data from data sets A, B, C, and D were used for Study 1, and from sets A, B, and D for Study 2. Scoring for both Study 1 and Study 2 included calculating a mean score for each implementation component. Although data regarding teachers’ use of extension or additional materials outside of lesson time were collected in each evaluation, due to concerns about overburdening teachers with program implementation, teachers were told that they were not required to include these in their implementation of the program. This resulted in an overall lack of consistent data for this implementation component. For example, the average implementation of transfer activities among the eight participants in the SMART and MindUP RCT (data set A) across 15 lessons was 31%. Thus, data for this component were not included in preliminary or other analyses. See Appendix D: Summaries of Extension Activity Implementation for information regarding the proportion of extension activities delivered by lesson for each SEL program. During preliminary analyses, a procedure was used to create a composite Total Implementation Quality score. This score’s reliability was in the acceptable range (α = .536), but lower than what has been reported in other research (Dowling & Barry, 2020a). A full description of the procedure used to create the Total Implementation Quality score is provided in Chapter 4. Due to variation in the number of lessons in each SEL program, a new variable was created to reflect these data: Total number of lessons. The total number of lessons for each program are summarized in Table 3.4. 57 Table 3.4 Summary of Total Number of Lessons in Each SEL Program SEL Program Total Lessons Kindness in the Classroom 16 MindUP 2012 15 MindUP 2017 9 Taxi Dog 9 58 Chapter 4: Results In this chapter, the results for Studies 1 and 2 are presented in turn. For each study, the results of preliminary analyses are presented first, including the descriptive statistics for all variables used in the main analyses. This is followed by a presentation of results for the main analyses used to test hypotheses for each study. Study 1 hypotheses were tested with bivariate correlations and hierarchical regression analysis. Study 2 hypotheses were tested with paired-samples t tests and bootstrapped one-way between-subjects analysis of covariance (ANCOVA). This chapter ends with an overall summary of the findings. Study 1 Using secondary data from four SEL program evaluations (N = 32), Study 1 sought to explore the following research question: How are teachers’ pre-implementation factors (SEL Beliefs: Comfort, Commitment, Culture; job satisfaction; SEL experience) related to SEL program implementation quality? The following section presents the results of analyses performed to answer this research question and test the related hypotheses. Preliminary analyses are presented first. These are followed by a presentation of the analyses used to test each hypothesis. 4.1.1 Preliminary Analyses Before performing any of the main analyses, several steps were undertaken to prepare the data. Because this study included data from four different studies, the data were first merged. Data from Study A (N =8), Study B (N = 14), Study C (N = 9), and Study D (N = 2) were merged into one dataset (N = 33). As mentioned in the Method section, one of the teachers delivered only one third of the SEL program (three out of nine lessons), so that participant was removed from subsequent analyses, resulting in a total of 32 participants for analyses. The combined data were 59 checked for missingness and errors, and outliers were evaluated. This is discussed in more detail below. 4.1.1.1 Teacher Pre-Implementation Factor Data Due to a photocopying error, six teachers from one study (Study B) were missing answers for several of the questionnaires, including job satisfaction, SEL Comfort, SEL Commitment, and SEL Culture. Because of this, the data from these teachers were deleted pairwise from the sample for preliminary analyses and were not included in the regression analyses. Mean composite scores were computed for each SEL Beliefs subscale. SEL experience was scored as a 0, 1, or 2 based on responses to two questionnaire items that asked about teachers’ experience being trained in a specific SEL program or training in SEL more broadly (0 = no on both items, 1 = one yes and one no, and 2 = yes on both items). Descriptive analyses and visualizations of the data were used to determine if these variables met the assumptions for linear regression analysis, including testing linearity, independence, normality, homoscedasticity, skewness, and kurtosis of the data. One teacher’s data point for SEL Commitment fell outside the upper fence of the box plot. According to Kwak and Kim (2017), this is an outlier and should be removed from the data. This left the sample fit for regression analysis at N = 25. Table 4.1 summarizes the means, standard deviations, minimum, maximum, skewness, and kurtosis for the sample data. 60 Table 4.1 Study 1: Summary of Teacher Pre-Implementation Factor Data Teacher Factor N M SD Minimum Maximum Skewness Kurtosis Job Satisfaction 26 3.69 .47 3 4 -.89 -1.33 SEL Comfort 26 4.13 .66 2.75 5 -.37 -.52 SEL Commitment 26 4.38 .78 1.50 5 -.29 6.99 SEL Culture 26 4.34 .55 2.75 5 -.72 1.01 SEL Experience 32 .81 .74 0 2 .32 .81 4.1.1.2 Implementation Data Prior to computing mean scores for each implementation component (dosage, adherence, and engagement), 11 cases of missing implementation engagement data were resolved through person mean substitution. Recommended for use with Likert-type ratings, this method substitutes the mean of each person’s completed items for those items that the person is missing resulting in different substitutions for each participant with missing data (Downey & King, 1998). This method yields a good representation of the data when the missing data are less than 20%, which was the case in this study. The Total Implementation Quality score was computed using methods described by Dowling and Barry (2020a, 2020b). First, each implementation indicator was averaged separately to yield mean scores for total implementation dosage, adherence, and engagement. Intercorrelations among these variables were analyzed. Implementation dosage and adherence were significantly and positively correlated with one other. However, neither implementation dosage nor adherence were significantly correlated with reports of student engagement. The weak, positive relation of implementation engagement to dosage and adherence suggests that increases in implementation dosage and adherence are not accompanied by 61 increases in student engagement, and thus they do not have a linear relationship. Therefore, implementation engagement was not included in creating the Total Implementation Quality composite. This resulted in a 2-component Total Implementation Quality score that was computed by summing the mean scores for implementation dosage and adherence, and then computing an average. Descriptive analyses and visualizations of the data were used to determine if these variables met the assumptions for linear regression analysis, including testing linearity, independence, normality, homoscedasticity, skewness, and kurtosis of the data. Upon evaluation of the data, implementation dosage was identified as substantially skewed (> 1). This is an expected distribution because implementation dosage was scored as either 0 or 1, and because high dosage rates are typical in most SEL program implementations (Durlak & DuPre, 2008). Table 4.2 summarizes the means, standard deviations, minimum, maximum, skewness, and kurtosis of these data (N = 32). Because four programs were implemented, Table 4.3 presents the means and standard deviations for all implementation composite scores for each SEL program separately. Table 4.2 Study 1: Summary of Implementation Data Composite Scores M SD Minimum Maximum Skewness Kurtosis Total dosage .95 .08 .67 1 -1.94 3.6 Total adherence .78 .16 .37 1 -.63 .33 Total engagement .80 .11 .58 1 -.29 -.30 Total quality (2-Component Score) .87 .10 .52 1 -1.23 2.74 Note. N = 32 62 Table 4.3 Study 1: Summary of Implementation Composite Scores by SEL Program SEL Program Total Number of Lessons Total Dosage Total Adherence Total Engagement Total Implementation Quality Kindness in the Classroom 16 .960(.058) .689(.333) .798(.085) .824(.05) MindUP 2012 15 .867(.118) .783(.196) .853(.111) .823(.05) MindUP 2017 9 1.00(.00) .878(.081) .800(.00) .939(.027) Taxi Dog 9 1.00(.00) .923(.087) .746(.138) .961(.058) Note. The numbers in parentheses are standard deviations. 4.1.2 Hypothesis 1 It was hypothesized that teachers’ SEL Beliefs, SEL experience, and job satisfaction would be positively correlated with implementation quality. As mentioned above, the Total Implementation Quality score for this analysis consisted of the mean of two indicators—adherence and dosage. Table 4.4 reports the results of bivariate Pearson correlations of teacher pre-implementation factors to total number of SEL lessons, implementation quality indicators (dosage, adherence), and Total Implementation Quality score (combined dosage and adherence). 63 Table 4.4 Intercorrelations Between Teacher Pre-Implementation Factors and Total Number of Lessons, and Implementation Quality Variable N Total Dosage Total Adherence Total Implementation Quality (2-Component) 1. Job Satisfaction 26 -.354 .241 .034 2. SEL Experience 32 .218 -.098 .016 3. SEL Comfort 26 -.185 -.011 -.084 4. SEL Commitment 25 .017 -.158 -.110 5. SEL Culture 26 .021 -.316 -.226 6. Total Lessons 32 -.350* -.605** -.598** Note. * p < .05, ** p < .01. As can be seen in Table 4.4, there were no significant correlations between teachers’ pre-implementation factors to implementation dosage, adherence, or total implementation quality. The total number of lessons in a program was significantly and negatively correlated with implementation dosage, adherence, and total implementation quality. 4.1.3 Hypothesis 2 It was next hypothesized that teachers’ SEL Beliefs, SEL experience, and job satisfaction would predict greater implementation quality when controlling for the number of lessons in the SEL program. Hierarchical regression analysis was conducted to explore teacher pre-implementation factors as predictors of SEL program implementation quality (dosage and adherence) when controlling for the number of lessons. Allison (1999) suggests a minimum of five cases per variable as necessary for multiple regression thus the sample size of N = 25 and five predictor variables in this analysis met this requirement. The control variable was entered in Step 1, followed by job satisfaction and SEL Culture in Step 2, and SEL Comfort, Commitment, 64 and experience in Step 3. The variables entered in this way reflect three factors affecting teachers’ sustained implementation: program-related factors, school context factors, and teacher factors (Han & Weiss, 2005). As shown in Table 4.5, the final model predicted 37.7% of the variance in the outcome variable of total implementation quality, R2 =.377, F(6, 24) = 1.814, p > .05. The R2 in Step 1 was statistically significant indicating that the total number of lessons in an SEL program was a significant predictor of total implementation quality. This predictor explained 30% of the variance in implementation quality and was negatively associated with implementation quality. That is, as the total number of lessons increased, total implementation quality decreased. The R2 change in neither Step 2 nor Step 3 was statistically significant. Although not significant, teacher pre-implementation factors predicted 7.6% of the variance in implementation quality after taking into account the total number of lessons. 65 Table 4.5 Summary of Hierarchical Regression Model of Teacher Pre-Implementation Factors as Predictors of Implementation Quality Variable B 95% CI for B SE B β R2 ∆R2 LL UL Step 1 .301 .301** Constant 1.115 .952 1.278 .079 Total lessons -.018 -.030 -.006 .006 -.549** Step 2 .341 .040 Constant 1.196 .752 1.640 .213 -.546** Total lessons -.018 -.031 -.006 .006 .082 Job satisfaction .018 -.065 .102 .040 -.182 SEL Culture -.035 -.105 .036 .034 Step 3 .377 .036 Constant 1.333 .738 1.929 .83 Total lessons -.019 -.033 -.006 .007 -.581** Job satisfaction .023 -.074 .121 .046 .103 SEL Culture -.037 -.118 .043 .038 -.196 SEL experience .025 -.037 .087 .029 .184 SEL Comfort -.011 -.084 .061 .034 -.070 SEL Commitment -.024 -.084 .037 .029 -.172 Note. CI = Confidence interval; LL = lower limit; UL = upper limit N = 25, ** p < .01 Model diagnostics suggest that the final model met the independence assumption (Durbin-Watson = 1.365 which is neither < 1 nor > 3). The assumptions of normality, linearity, and homoscedasticity were examined using the normal probability plots and residuals scatterplots and examinations suggested these assumptions were met. Multicollinearity was 66 assessed using the tolerance statistic. Allison (1999) suggests that tolerance < .40 is cause for concern, but all tolerance statistics in this model were > .75 suggesting little multicollinearity. In sum, teacher pre-implementation factors were not significantly related to implementation dosage, adherence, or total implementation quality. Furthermore, variation in implementation quality was not explained by teacher pre-implementation factors when controlling for the number of lessons in the SEL program. However, the total number of SEL lessons was significantly and negatively related to total implementation quality. Study 2 Using secondary data from a sub-set of Study 1 that included teacher data from three SEL program evaluations (N = 23), Study 2 study sought to answer the following research question: How are changes in teachers’ pre-implementation factors (job satisfaction, efficacy, occupational stress, SEL Beliefs) following SEL program implementation related to implementation quality (low- versus high-quality implementation)? The following section presents the results of analyses performed to answer this research question and to test the related hypotheses. First, preliminary analyses are presented. These are then followed by a presentation of the analyses used to test each hypothesis. 4.2.1 Preliminary Analyses Before performing any analyses, several steps were undertaken to prepare the data. Because this study included data from three different studies, the data were first merged. Data from Study A (N =8), Study B (N = 14), and Study D (N = 2) were merged into one dataset (N = 24). As mentioned in the Method section, one of the teachers delivered only one third of the SEL program (three out of nine lessons), so that participant was removed from subsequent analyses, 67 resulting in a total of 23 participants. The combined data were checked for missingness and errors, and outliers were evaluated. This is discussed in more detail below. 4.2.1.1 Teacher Factor Data Due to a photocopying error, six teachers from one study (Study B) were missing answers for several of the questionnaires, including teaching efficacy, job satisfaction, and all SEL Beliefs subscales. There were also missing post-implementation job satisfaction data for four teachers, and post-implementation teaching efficacy, occupational stress, and all SEL Beliefs subscales for one teacher. These missing data were deleted pairwise from the sample for preliminary analyses and were not included in the main analyses. Mean composite scores were computed for each variable at two timepoints: pre-implementation and post-implementation. Descriptive analyses and visualizations of the data were used to determine their normality including skewness and kurtosis. One teacher’s data point for SEL Commitment fell outside the upper fence of the box plot. According to Kwak and Kim (2017), this is an outlier and should be removed from the data. Four of the eight variables were highly skewed suggesting the use of non-parametric approaches for hypothesis testing with these data since non-parametric tests do not assume normality of the data (Siegel & Castellan, 1988). In fact, it has been argued that it is inappropriate to evaluate the normality assumption with small sample sizes (Siegel, 1957). Table 4.6 summarizes the means, standard deviations, minimum, maximum, skewness, and kurtosis of these data. 68 Table 4.6 Study 2: Summary of Teacher Factor Data Teacher Factor N M SD Min. Max. Skewness Kurtosis SEL Comfort Pre-implementation 18 4.24 .644 3 5 -.557 -.432 Post-implementation 23 4.41 .572 3.25 5 -.427 -.976 SEL Commitment Pre-implementation 18 4.44 .885 1.5 5 -2.48 7.048 Post-implementation 23 4.41 .486 3.25 5 -.313 -.263 SEL Culture Pre-implementation 18 4.43 .617 2.75 5 -1.22 1.667 Post-implementation 23 4.37 .538 3.5 5 -.118 -1.448 Emotional Exhaustion Pre-implementation 21 3.188 .992 1.56 4.83 -.108 -.864 Post-implementation 23 3.155 1.248 1.33 5.56 .181 -1.014 Depersonalization Pre-implementation 23 1.40 .367 1 2.4 1.065 .989 Post-implementation 23 1.661 .764 1 3.4 1.138 -.098 Personal Accomplishment Pre-implementation 23 3.717 2.01 1.75 7 .713 -1.402 Post-implementation 23 3.685 1.98 1.75 6.88 .662 -1.456 Teaching Efficacy Pre-implementation 17 3.823 .482 3.13 4.75 .822 -.103 Post-implementation 23 3.76 .376 3.25 4.50 .622 -.740 Job Satisfaction Pre-implementation 18 3.778 .4278 3 4 -1.461 .137 Post-implementation 20 3.750 .444 3 4 -1.251 -.497 69 4.2.1.2 Implementation Data Prior to computing mean scores for each implementation component (dosage, adherence, and engagement), 10 cases of missing implementation engagement data were resolved through person mean substitution. Recommended for use with Likert-type ratings, this method substitutes the mean of each person’s completed items for those items that the person is missing resulting in different substitutions for each participant with missing data (Downey & King, 1998). This method yields a good representation of the data when the missing data are less than 20% which was the case in this study. Total Implementation Quality was computed by using methods described by Dowling and Barry (2020a, 2020b). First, each indicator was averaged separately to yield mean scores for total implementation dosage, adherence, and engagement. Intercorrelations among these variables were analyzed which revealed significant correlations between all three implementation indicators. Thus, the decision was made to proceed with a 3-component Total Implementation Quality score which was computed by summing the mean scores for implementation dosage, adherence, and engagement and then computing an average.1 Further analysis found the composite scale including all three indicators of implementation to have acceptable reliability (α = .644). Descriptive analyses and visualizations of the data were used to evaluate normality of data including skewness and kurtosis. Upon evaluation of the data, implementation dosage was identified as substantially skewed (> 1). This is an expected distribution because implementation dosage was scored as either 0 or 1, and because high dosage rates are typical of SEL program implementations (Durlak & DuPre, 2008). Table 4.7 1 This is in contrast to Study 1 which used a 2-component Total Implementation Quality score because engagement was not significantly correlated with either dosage or adherence. 70 summarizes the means, standard deviations, minimum, maximum, skewness, and kurtosis of these data (N = 23). See Table 4.3 above for a summary of the means and standard deviations for all implementation composite scores for each SEL program separately. Table 4.7 Study 2: Summary of Implementation Data Composite Scores M SD Minimum Maximum Skewness Kurtosis Total dosage .932 .094 .667 1 -1.507 1.844 Total adherence .735 .149 .365 1 -.562 .586 Total engagement .815 .094 .60 1 -.155 .130 Total quality (3-Component Score) .827 .882 .564 .96 -1.054 2.316 Note. N = 23 Further preparation of the implementation data included sorting the Total Implementation Quality scores into low- and high-quality implementation groups. Following procedures used in previous research (Dix et al., 2012; Dowling & Barry, 2020a, 2020b), the visual binning procedure was performed using the statistical software SPSS. First, cut-points were applied at the mean and ±1 standard deviation level. This yielded four binned categories (1 = low, 2 = moderately low, 3 = moderately high, and 4 = high). To create categories of low- and high-quality implementation, binned scores of 1 or 2 were allocated into the low-quality category, and scores of 3 or 4 allocated to the high-quality category. The groups were fairly even, with 12 scores falling in the high-quality implementation group and 11 scores in the low-quality implementation group. 71 4.2.2 Hypothesis 1 For hypothesis 1, it was expected that SEL implementation would be associated with increases in teachers’ SEL Beliefs, efficacy, and job satisfaction, and decreases in occupational stress (i.e., decreased emotional exhaustion and depersonalization, and increased personal accomplishment). A series of paired-samples t tests were performed to determine whether the mean pre-implementation scores were significantly different from the mean post-implementation scores. Although normal distribution is preferred for conducting t tests, they are robust and can handle violations of this assumption (Cronk, 2018). As shown in Table 4.8, significant differences from pre- to post-implementation were found for depersonalization (t(23) = -2.389, p < .05) and teaching efficacy (t(18) = 4.373, p < .001), but neither difference was in the hypothesized direction. That is, depersonalization increased from pre- to post-implementation and teaching efficacy decreased. Cohen (1992) suggests that effect sizes of .2, .5, and .8 are small, medium, and large, respectively. Thus, the effect size for the change in teachers’ depersonalization is a medium effect size and the change in teachers’ teaching efficacy is large. 72 Table 4.8 Study 2: Summary of Paired-Samples t Test Results Between Pre- and Post-Implementation Mean Scores Note. Numbers in parentheses are standard deviations. * p < .05, ** p < .01. 4.2.3 Hypothesis 2 It was hypothesized that teachers’ post-implementation teacher factor scores would differ between low- and high-quality implementation groups and evidence more change in the hypothesized direction in the high-quality implementation group. ANCOVA is the analysis typically used to determine if post-test scores differ significantly between two groups while statistically controlling for pre-test scores (Dugard & Todman, 2006). However, because statistical guidelines recommend using non-parametric tests with small samples (e.g., Altman et al., 1983), the approach used in this study was a bootstrapped one-way between-subjects Teacher Factor N Pre- Implementation Post-Implementation r t p Cohen’s d SEL Beliefs SEL Comfort 18 4.236 (.644) 4.389 (.602) .679** -1.294 .213 .305 SEL Commitment 18 4.444 (.885) 4.430 (.468) .407 .072 .943 .017 SEL Culture 18 4.431 (.617) 4.431 (.499) .449 .000 1.00 .000 Occupational Stress Emotional Exhaustion 21 3.188 (.992) 3.106 (1.297) .691** .399 .694 .087 Depersonalization 23 1.400 (.367) 1.661 (.764) .792** -2.389 .026 .498 Personal Accomplishment 23 3.717 (2.014) 3.685 (1.981) .984** .439 .665 .092 Teaching Efficacy 18 4.167 (.788) 3.167 (1.043) .466* 4.373 .000 1.031 Job Satisfaction 16 3.750 (.447) 3.813 (.403) .462 -.565 .580 .141 73 ANCOVA to assess differences in teachers’ post-implementation scores on each teacher factor variable between low- and high-quality implementation groups. The dependent variable was the post-implementation score for each teacher factor. To statistically control for the influence of pre-implementation scores and the total number of lessons in each SEL program, these variables were used as covariates in the analyses. Non-parametric bootstrapped tests assume that the sample is a reasonable approximation of the population, and do not require that any other assumptions about the population are met (Efron, 1979; Siegel, 1957). Therefore, no assumptions were checked prior to analyses. To perform the bootstrapping ANCOVA, the data were resampled 10,000 times which created an empirical distribution of resampled mean differences (Efron, 1979). Significance is determined by examining the confidence intervals associated with the mean differences. If they include zero, then there is no significant difference between the groups. When controlling for the total number of lessons and pre-implementation factor scores, there were no significant differences detected for post-implementation scores based on membership in the low- or high-quality implementation group as zero was in the confidence interval range for every teacher factor. Table 4.9 reports the adjusted mean post-implementation scores for each implementation group, the mean difference, their respective bootstrapped confidence intervals, the F-statistic, p-value, and partial eta squared. Partial eta squared (ηp2) captures the proportion of the variance in the mean differences explained by membership in either the low- or high-quality implementation group. Cohen (1969) suggests that values of .0099, .0588, and .1379 can be interpreted as small, medium, or large, respectively. Thus, teaching efficacy was associated with a large effect size. 74 Table 4.9 Bootstrapping ANCOVA: Teacher Factor Adjusted Mean Post-Implementation Scores and Difference for Low- and High-Implementation Quality Groups Note. CI = Confidence interval Teacher Factor Mean SE 95% CI F p ηp2 SEL Comfort .023 .880 .002 Low 4.409 .168 [4.149, 4.718] High 4.373 .149 [3.922, 4.791] Difference .036 .257 [-.420, .573] SEL Commitment .426 .524 .030 Low 4.516 .171 [3.963, 5.075] High 4.362 .151 [4.239, 4.444] Difference .154 .251 [-.518, .813] SEL Culture .181 .677 .013 Low 4.369 .184 [3.936, 4.815] High 4.480 .163 [4.193, 4.829] Difference -.111 .280 [-.718, .463] Emotional Exhaustion .890 .359 .050 Low 2.884 .319 [2.107, 3.683] High 3.308 .304 [2.650, 4.024] Difference -.424 .401 [-1.289, .391] Depersonalization .587 .453 .030 Low 1.576 .150 [1.227, 1.946] High 1.739 .143 [1.421, 2.108] Difference -.163 .217 [-.605, .205] Personal Accomplishment .010 .920 .001 Low 3.693 .110 [2.949, 4.491] High 3.677 .105 [2.886, 4.507] Difference .016 .176 [-.286, .336] Teaching Efficacy 2.487 .139 .161 Low 3.889 .090 [3.676, 4.137] High 3.682 .085 [3.431, 3.902] Difference .207 .140 [-.005, .492] Job Satisfaction .353 .563 .029 Low 3.884 .155 [3.668, 4.042] High 3.741 .155 [3.281, 4.000] Difference .143 .156 [-.381, .500] 75 In order to evaluate the estimates and predictive power of a model, it is recommended to perform a sensitivity analysis using a conceptually similar test (Carpenter, 2017). One-way ANCOVA and repeated-measures (RM) ANCOVA employ different statistical models to accomplish similar objectives, in this case testing whether teachers with higher implementation quality had better outcomes over time compared to teachers with lower implementation quality. Thus, a mixed design RM ANCOVA2 was conducted where time (pre-implementation vs. post-implementation) was the within-subjects factor, implementation quality (low vs. high) was the between-subjects factor, controlling for the total number of lessons. Table 4.10 shows this analysis yielded similar results to the bootstrapping ANCOVA. That is, no significant differences between low- and high-quality implementation groups were found, but there was a large effect size present for teaching efficacy. This provided further evidence that there were no significant differences in teachers’ post-implementation scores between low- and high-implementation quality groups in this sample. Table 4.11 reports the adjusted mean post-implementation scores for each implementation group, the mean difference, their respective confidence intervals, the F-statistic, p-value, and partial eta squared.. 2 Bootstrapping repeated-measures ANCOVAs are not offered in the statistical software employed for these analyses. 76 Table 4.10 Mixed Design Repeated Measures ANCOVA: Teacher Factor Adjusted Mean Post-Implementation Scores and Difference for Low- and High-Implementation Quality Groups Note. CI = Confidence interval Teacher Factor Mean SE 95% CI F p ηp2 SEL Comfort .060 .810 .004 Low 4.293 .204 [3.859, 4.727] High 4.328 .181 [3.943, 4.713] Difference -.035 .282 [-.635, .565] SEL Commitment .001 .979 .000 Low 4.541 .224 [4.064, 5.018] High 4.355 .199 [3.931, 4.778] Difference .187 .309 [-.473, .846] SEL Culture 1.445 .248 .088 Low 4.554 .174 [4.184, 4.924] High 4.331 .154 [4.003, 4.660] Difference -.111 .280 [-.718, .463] Emotional Exhaustion 1.052 .319 .055 Low 3.135 .319 [2.465, 3.805] High 3.158 .304 [2.520, 3.796] Difference -.023 .448 [-.965, .920] Depersonalization .533 .466 .027 Low 1.479 .169 [1.127, 1.831] High 1.577 .161 [1.241, 1.914 Difference -.098 .240 [-.600, .403] Personal Accomplishment .074 .789 .004 Low 3.244 .636 [1.918, 4.570] High 4.120 .607 [2.854, 5.386] Difference -.876 .906 [-2.765, 1.013] Teaching Efficacy 4.546 .051 .245 Low 3.707 .160 [3.365, 4.049] High 3.885 .150 [3.564, 4.206] Difference -.178 .227 [-.665, .309] Job Satisfaction 2.153 .166 .142 Low 3.656 .116 [3.405, 3.907] High 3.906 .116 [3.655, 4.157] Difference -.250 .167 [-.611, .111] 77 Although these analyses did not yield significant findings, there were near-significant findings for the teacher factor teaching efficacy. Specifically, in the bootstrapped one-way between-subjects ANCOVA analysis, the lower bound for the confidence interval for the mean difference between teachers’ pre- and post-implementation teaching efficacy nearly did not include zero (-.005), and in the RM ANCOVA analysis, the p-value was nearly significant (.051). In both of these analyses, the effect size was large. Thus, further exploration of this teacher factor was warranted. Therefore, a plot that visualized the change in the estimated marginal means of the low- and high-quality implementation groups was produced. As shown in Figure 4.1, teachers in the high-quality implementation group experienced a decrease in teaching efficacy over the course of SEL program implementation whereas teachers in the low-quality implementation group experienced an increase. Figure 4.1 Change in Estimated Marginal Means for Teaching Efficacy for Low- and High-Quality Implementation Groups Teacher Factor: Teaching Efficacy 78 In sum, Study 2 explored how changes in teacher factors were related to teachers’ SEL program implementation quality. First, the hypothesis that SEL implementation would be associated with increases in teachers’ SEL Beliefs, efficacy, and job satisfaction, and decreases in occupational stress (i.e., decreased emotional exhaustion and depersonalization, and increased personal accomplishment) was tested with paired-samples t tests. The results showed that teachers’ depersonalization and teaching efficacy scores changed significantly after implementing an SEL program; however, these changes were not in the hypothesized direction. In other words, teachers’ experience of depersonalization and perception of teaching efficacy worsened. Next, one-way between-subjects bootstrapping ANCOVAs were performed to test the hypothesis that teachers’ post-implementation teacher factor scores would differ between low- and high-quality implementation groups while statistically controlling for the number of lessons in each SEL program. Results of the bootstrapping ANCOVA indicated that changes in teacher factors did not differ significantly between low- and high-quality implementation groups. A sensitivity analysis employing RM ANCOVAs yielded similar results. Thus, differences in teacher factor change could not be attributed to teachers’ SEL program implementation quality. However, further analysis of near-significant findings for the teacher factor teaching efficacy suggested that teachers in the low- and high-quality implementation groups did differ and, in line with the t-test results, this difference was not in the hypothesized direction. Specifically, high-quality implementers experienced a decrease in teaching efficacy over the course of SEL program implementation whereas low-quality implementers experienced an increase. 79 Summary of Results Study 1 explored associations of teachers’ (N = 32) pre-implementation factors including SEL experience, job satisfaction, and SEL beliefs, to program implementation quality. Correlational and multiple regression analyses revealed no significant relationships between teacher factors and implementation quality. However, the total number of lessons in the SEL programs surfaced as a significant predictor of implementation quality, wherein fewer lessons predicted greater implementation quality. Study 2 explored how changes in teacher factors including SEL beliefs, occupational stress, efficacy, and job satisfaction were related to teachers' SEL program implementation quality. Teachers (N = 23) were grouped into low- and high-quality implementation categories based on their total implementation quality scores. Paired-samples t tests showed that teachers’ depersonalization and teaching efficacy scores changed significantly after implementing an SEL program, with depersonalization scores increasing and teaching efficacy scores decreasing. Further analyses using one-way bootstrapping ANCOVAs suggested that these changes in teacher factors could not be attributed to teachers’ SEL program implementation quality. However, analysis of near-significant findings for the teacher factor teaching efficacy suggested that teachers in the low- and high-quality implementation groups did differ, with high-quality implementers experiencing a decrease in teaching efficacy and low-quality implementers experiencing an increase. 80 Chapter 5: Discussion Broadly, the purpose of the current research was to explore the relationship between teacher factors and the implementation of SEL programs. Using data from four experimental evaluations of SEL programs, this research represents one of the first attempts to examine this relationship across more than one SEL program. Two studies were conducted with these data to examine a number of research questions. Specifically, Study 1 examined the association of pre-implementation teacher factors to overall implementation quality in relation to the implementation of four SEL programs. The teachers factors included in this study were job satisfaction, experience with SEL, and beliefs about SEL. The latter factor includes teachers’ comfort with and commitment to SEL, and their perception of school culture supports for SEL. Study 2 explored if and how changes in teacher factors were related to implementation quality across three SEL programs. The teacher factors examined in this study included measures of SEL beliefs, occupational stress, teaching efficacy, and job satisfaction. Because this study was focused on changes in teacher factors, both pre- and post-implementation scores were considered and analyzed in relation to teachers’ overall implementation quality. Additionally, changes in these factors were further examined by analyzing the level of low- versus high-quality implementation. This chapter first offers discussion of the results of these studies and possible directions for future research. This is followed by a discussion of the strengths and limitations of the studies. The chapter concludes with discussion of the implications of this research. 5.1. Teacher Factors and SEL Program Implementation Quality Analyses for Study 1 revealed no significant correlations between teacher factors and implementation quality. Furthermore, findings revealed that teachers’ SEL Beliefs, including SEL Comfort, Commitment, and Culture, prior to implementation were not significantly 81 correlated with teachers’ self-reported implementation dosage or adherence, or the total implementation quality score that reflected the combined average of these implementation components. Brackett et al. (2012) found similar non-significant relationships between SEL Comfort, Commitment, and Culture and implementation quality in their evaluation of the RULER program. Interestingly, they did find that teachers’ SEL Beliefs were significantly correlated with teachers’ perceptions of the SEL program. For example, teachers’ SEL Comfort and Commitment scores were positively correlated with perceived program effectiveness and program buy-in. Therefore, although findings from this and previous research suggest that SEL Beliefs are not strongly related to teachers’ SEL implementation quality, they may be associated with teachers’ perceptions of the SEL program. Over time, this can influence teachers’ sustained implementation of an SEL program (Han & Weiss, 2005) which suggests that more longitudinal research would further elucidate our understanding of how SEL Beliefs are related to implementation quality over the long term. Another teacher factor examined in relation to SEL implementation quality in Study 1 focused specifically on teachers’ experience with SEL. This construct was operationalized in the present study in terms of teachers’ exposure to training in a specific SEL program, and any other special training in SEL they had undergone. High-quality teacher training and professional development have been related to teachers’ ability to deliver SEL programs with fidelity (Durlak, 2015; Fixsen et al., 2005; Jennings & Frank, 2015). Moreover, previous research (Dusenbury et al., 2005) suggests that teachers with a greater understanding of the underlying concepts of an SEL program and more years teaching a program are more adherent in their implementation of a program. In this study, there was no significant correlation between teachers’ SEL experience and implementation quality. However, there was a strong, positive correlation between teachers’ 82 SEL experience and their SEL Commitment. Thus, as teachers gain more experience implementing an SEL program and their commitment increases, one might see a stronger relation to their sustained implementation of the program. Similar to the discussion of SEL Beliefs and implementation quality, longitudinal research would lend a greater understanding to this relationship. For example, Wanless et al. (2015) examined the relationship between teacher pre-implementation factors and implementation fidelity and found no immediate direct relationship, but they did find that teachers’ engagement in initial training mediated the relation between implementation fidelity observed two years later. These findings are in line with the Process Model of Sustained Implementation (Han & Weiss, 2005) which emphasizes teaching training and ongoing support as integral to implementation sustainability. Job satisfaction was another teacher factor examined in relation to teachers’ implementation quality in Study 1. This factor captures teachers’ perception of the work climate of their school. Durlak and DuPre (2008) have identified this factor as one of many factors potentially influencing implementation quality. In this study, job satisfaction was not significantly and positively associated with implementation quality as hypothesized. However, job satisfaction was negatively associated to implementation dosage and, although not significant, this was a moderate correlation. This finding is in line with previous research that has found that teachers’ self-reported job satisfaction was significantly and negatively associated with implementation dosage of a preventive intervention program for children at risk of developing conduct problems (Klimes-Dougan et al., 2009). In fact, similar to the findings in the present study, Klimes-Dougan et al. (2009) had hypothesized that job satisfaction would correlate positively with implementation quality. When the opposite relationship emerged in their study, the researchers posited that the teachers working in more challenging school 83 environments were more motivated to deliver the program as intended to ensure positive outcomes such as improved student behaviour. Improved student behaviour may help reduce teachers’ stress and create a more positive school climate. This, in turn, could lead to increases in teachers’ job satisfaction. This discussion was in part based on the researchers’ full knowledge of the context in which the program they were evaluating was implemented. In the present study, data regarding the school climate in the schools were not collected, therefore any comparisons cannot be made. However, the findings from Klimes-Dougan et al. (2009) prompt an interesting direction for future research. Further exploration of how school-level organizational factors, either positive or negative, contribute to teachers’ motivation to deliver programs with fidelity will advance our understanding of these factors in relation to program implementation. Durlak and DuPre (2008) identify program features as one of multiple factors that can affect implementation fidelity. Indeed, Domitrovich et al. (2015a) found that only teachers with more positive attitudes towards the SEL program reported more frequent lesson delivery. Given that the present study used data from four evaluations of SEL programs, the difference between the programs was an important variable to consider. Thus, the variable “total number of lessons” was created to capture this difference and was included in analyses. In the present study, total number of SEL program lessons was significantly and negatively correlated with implementation dosage and adherence, and significantly and negatively correlated with the total implementation quality score. This suggests that implementation quality was lower when teachers were delivering programs with more lessons. In fact, given the higher, negative correlation between number of lessons and program adherence specifically, it appears that teachers faced with delivering more lessons over a longer period of time may have found it more challenging to be adherent to delivering the program as intended. 84 The present study was one of the first studies to examine the relation between total number of SEL lessons and implementation fidelity, thus these findings make a significant contribution to our knowledge of factors that can affect implementation quality. Hierarchical regression analyses provided further insight into this relationship. Specifically, findings revealed that the number of lessons in a program explained 30% of the variance in implementation quality. Adding teacher factors to the regression model, including those related to the work context (i.e., job satisfaction and SEL Culture)., SEL Beliefs (i.e., SEL Comfort, Commitment) and SEL experience, explained only 7% more of the variance. Thus, the strongest predictor of implementation quality for the present sample was the number of lessons, and this relationship was both negative and significant in the final model. The number of lessons in the programs included in this study ranged from nine to 16 lessons. In fact, these reflect fewer lessons than are found in most SEL programs. The CASEL Guide (2012) of effective SEL programs provides an account of the total number of sessions per year for each program. These can vary by grade within a program, but among the 26 preschool and elementary programs included in the guide, the overall number of lessons ranges from 7-140 with a mean of 38.25 lessons per year. This average number of lessons is more than double the number included in the program with the greatest number of lessons in this study. It begs the question: if a greater number of lessons may be related to poorer implementation quality as suggested by the findings in this study, how might that affect implementation of programs with over 100 sessions per year? The findings in the present study indicating that the number of lessons in an SEL program is a significant predictor of implementation quality is an important finding which bears further exploration. For example, it will be important to investigate if and how training, consultation, and feedback mediate the relationship between the number of lessons in a program 85 and implementation quality. These findings can also help inform SEL program development as it highlights the total number of lessons as an important consideration in program design. Furthermore, these findings warrant the attention of schools and districts undertaking initiatives to implement SEL programs. They can inform decisions regarding which program to select considering the school’s or district’s capacity to support teachers in implementing SEL programs with fewer or more lessons to deliver each year. 5.2. SEL Program Implementation Quality and Changes in Teacher Factors SEL program evaluations are typically concerned with child-focused outcomes; however, as suggested by the Prosocial Classroom Model (Jennings & Greenberg, 2009), SEL program implementation may also be a pathway to improved teacher outcomes. Few research studies have explored the relation between SEL program implementation and teacher outcomes, hence Study 2 aimed to shed light on the nature and function of teachers’ SEL program implementation in relation to changes in teacher factors, and examine these changes in relation to SEL implementation quality. The teacher factors examined in this study included measures of SEL Beliefs, occupational stress, teaching efficacy, and job satisfaction. Analyses revealed that SEL program implementation was significantly and positively related to increases in teachers’ experience of depersonalization (a symptom of occupational burnout), and significantly and negatively associated with their perceived teaching efficacy. These findings were contrary to what was hypothesized in Study 2 as well as findings in previous research. For example, Ransford et al. (2009) found that teachers who implemented an SEL program reported increased efficacy and reduced burnout. Domitrovich and colleagues (2016) reported similar findings regarding teachers’ reduced experience of burnout and improved efficacy following SEL program implementation. Given the relation between job demands and increases in teacher stress 86 and burnout (Kyriacou, 2001), it is possible that teachers in this sample experienced SEL program implementation as a job demand that was not balanced by adequate support. However, given the relatively small sample size in the present study, it is difficult to make any firm conclusions. Previous research suggests that teacher burnout and efficacy are interrelated. Skaalvik and Skaalvik (2010) found teaching efficacy to be negatively related to emotional exhaustion and depersonalization. Similar results were evident in the present study in relation to Study 2. Specifically, although the correlations were not found to be statistically significant, both emotional exhaustion and depersonalization were found to be negatively related to teaching efficacy. These burnout symptoms were also moderately and negatively related to implementation quality. In a larger sample, these relations may have reached statistical significance, thus these remain important factors to investigate in relation to SEL program implementation. In exploring if changes in teacher factors varied in relation to SEL implementation quality, analyses revealed no significant differences between teachers who evidenced low- versus high-quality SEL program implementation. However, given the small sample size, it was advisable to explore and interpret findings that approached significance because detecting significant differences between groups and relations between variables of interest is difficult with small sample sizes (Leppink et al., 2016). Both a moderate effect size and near statistical significance were found across two analytic methods for one teacher factor: teaching efficacy. However, contrary to the hypothesis that teachers with higher quality implementation would experience increased teaching efficacy, results for Study 2 revealed that teaching efficacy decreased for teachers with high-quality implementation and increased for those with low-quality 87 implementation. Overall, findings from this study suggest that implementation quality might moderate changes in teaching efficacy over time. Intensive, high-quality implementation may increase teachers’ occupational stress and, as suggested above, the interrelation with teaching efficacy may yield changes to this teacher factor as well. However, future research should examine this with a larger sample of teachers. Overall, although modest, findings from Study 2 underscore the need to provide teachers with adequate support to help prevent reductions in efficacy or other teacher factors due to intensive SEL program implementation while promoting sustained high-quality SEL program implementation (Han & Weiss, 2005). Indeed, given the paucity of research exploring relations between changes in teacher factors and SEL program implementation quality, this area demands further research. Ransford et al. (2009) found patterns of teachers at higher risk for lower implementation quality based on pre-implementation factors including burnout and efficacy. Understanding how these teacher factors may affect or be affected by SEL program implementation can help program developers and schools and districts design targeted supports for teachers to bolster their capacity to implement SEL programs well. Furthermore, understanding the relation between well-supported SEL program implementation and changes in teacher factors may expand the perceived benefits of SEL program implementation beyond important student outcomes to teacher outcomes as well. 5.3. Strengths and Limitations The present studies were designed to explore the relationship between teacher factors and SEL program implementation. Several strengths are evident in this research. Notably, this research is one of the first to analyze teacher factor and program implementation data collected from multiple SEL program evaluations. Typically, analyses focus on one SEL program and 88 explore relationships between implementation quality and child outcomes. Examining teacher factors and their relation to implementation quality across four SEL program evaluations enhances our understanding of this under-studied relationship and helps elucidate patterns evident across SEL programs, rather than within one program. Importantly, in the present study, this approach allowed for the discovery that the number of lessons in an SEL program was a significant predictor of implementation quality. Furthermore, the implementation data used in this research were collected through careful monitoring of numerous components of implementation which permitted the computation of a multi-component Total Implementation Quality score. Considering that most program evaluations collect implementation data related to quantity (e.g., dosage and adherence; Durlak & DuPre, 2008; Fixsen et al., 2005), this research demonstrates the potential for multi-dimensional composites of implementation that capture both quantity and quality to provide a more complete and nuanced understanding of how implementation is related to teacher factors. Another important feature of the data from the four SEL program evaluations was the depth of teacher factor data, and that these data were collected before and after program implementation. In Study 1, these data permitted analyses of several pre-implementation teacher factors and their relation to implementation quality. In Study 2, these data permitted analyses of changes in key teacher factors over time—from before to after SEL program implementation—and how the changes were related to implementation quality. Taken together, the type of data collected, and the manner in which they were collected, offer guidance for future research exploring the relationship between teacher factors and SEL program implementation. Such an approach can further inform the development of effective supports for teachers’ sustained SEL program implementation. 89 Although the data and research design permitted important analyses and novel findings, this research is associated with several limitations. A significant limitation was the small sample size evident in both studies. Small sample sizes are recognized as limiting the statistical power for detecting both differences between groups and relations between variables of interest (Leppink et al., 2016). Thus, unable to discern significant differences or relations, the null hypothesis is not rejected whereas with a larger sample size, these may have been detected and the null hypothesis could be rejected. In this research, the limitation of small sample size was addressed in a number of ways. For example, in Study 1, the number of predictor variables used in the multiple regression analyses was limited to comply with the recommendation of a minimum of five cases per variable (Allison, 1999). This resulted in the use of five predictor variables given the sample size of N = 25. Based on statistical guidelines recommending non-parametric tests with small samples (e.g., Altman et al., 1983), Study 2 used bootstrapped one-way between-subjects ANCOVA to assess differences in teachers’ post-implementation scores on each teacher factor variable between low- and high-quality implementation groups. Further, a sensitivity analysis using a conceptually similar test, a repeated-measures ANCOVA, was employed to evaluate the estimates and predictive power of the model. These approaches strengthened the ability to detect significant differences between groups and relations among variables of interest. Another limitation evident in these studies was the use of self-report measures to collect information on teacher factors and implementation quality. Teacher factors such as efficacy, job satisfaction, occupational stress, and beliefs about SEL all pertain to self-perceptions, thus necessitating the use of self-report measures (Ransford et al., 2009). However, that implementation data were also collected in this way can limit their validity especially given 90 findings from previous research indicating a lack of concordance between reported and observed implementation (Begeny et al., 2013; Breitenstein et al., 2010; Koziol & Burns, 1986; Martinez et al., 2014). Yet, other research has found teacher self-reports to be strongly correlated with classroom observations of teaching practice (Desimone et al., 2010). Moreover, research suggests that teachers’ repeated use of self-report measures improves the accuracy of their reporting (Koziol & Burns, 1986), and that teacher reports of treatment integrity were highly accurate when using a standardized protocol completed weekly (Sanetti & Kratochwill, 2011). In accordance with previous research, teachers in the present studies completed a standard implementation calendar every week. This addressed the limitation of self-reported implementation and strengthened the validity of their reported implementation dosage, adherence, and student engagement. Although analyzing teacher factor and program implementation data collected from multiple SEL program evaluations can be considered a strength, it can also be considered a limitation. There is variability not only between the SEL programs in terms of their logic model, scope and sequence, and program components, but also between the contexts in which the programs were delivered. One way this limitation of SEL program difference was addressed the present studies was by controlling for a quantifiable difference between the programs—the number of lessons—in both Study 1 and Study 2 analyses. In this way, the possible effects of this program feature were partialled out such that differences due to other factors and variables could be analyzed. Notably, controlling for the number of lessons in the SEL program led to the discovery of this feature as a significant predictor of implementation quality. Regarding the difference in implementation context (i.e., the school), no data were available to discern the nature of these differences for this research. To remedy this, future research can collect 91 qualitative data through, for example, semi-structured interviews with school staff to elicit a multi-dimensional understanding of the context in which the SEL program was delivered and further elucidate the relation between teacher factors and SEL program implementation. Another related limitation associated with context is the historical context of SEL program implementation. Support for SEL programming has increased markedly over the last decade, so the context for SEL program implementation in schools in 2012, the year of the first study from which data were drawn for this research, may differ from the context of schools participating in studies conducted in later years. For example, teachers participating in later studies may have experienced higher levels of support from school leadership and the community which could have influenced their SEL program implementation quality. 5.4. Implications and Conclusion For well over a decade, researchers have invoked implementation science to help explain student outcomes in evaluations of SEL programs. Null or mixed effects for SEL programs have been reported in the literature (Jones & Doolittle, 2017; Jones et al., 2019); thus, it became important to discern the mechanisms mediating SEL program outcomes in order to develop solutions that could help ensure more consistent, positive outcomes for students. School-based SEL program implementation is a complex undertaking with many factors potentially affecting implementation quality (Durlak & DuPre, 2008; Durlak, 2015). These include factors related to the community (e.g., policies, funding, local mandates), school/system features (e.g., school climate, administrator support), SEL program features (e.g., fit with context, adaptability), professional development (e.g., initial and ongoing training, technical assistance, and coaching), and staff characteristics (e.g., skill base, confidence to deliver the program, perceived need for and benefits of the program). Although some research has focused on the relation between 92 organizational and individual factors and SEL program implementation quality (e.g., Domitrovich et al., 2015a), little research has examined the relation between teacher factors and SEL program implementation in great depth. Han and Weiss (2005) proposed an implementation model that pays special attention to teacher-level factors and their relation to initial, supported, and sustained implementation. Employing the Model of Sustained Implementation by Teacher as a frame, the present research sought to close the aforementioned research gap by closely examining the relationship between teacher factors and SEL program implementation. Despite modest findings, this research has several implications for SEL program design, and teacher training and support. Regarding SEL program design, program developers can consider the finding from Study 1 that the number of lessons in an SEL program is significantly and negatively related to implementation quality in their program development process. For example, this finding suggests a benefit to leaner program design and finding other innovative ways to address program goals outside of the traditional lesson format. This is in line with the work of Jones et al. (2017) to identify evidence-based strategies called SEL kernels that can be easily implemented at no or low cost in a variety of contexts. Another important implication arising from the research findings concerns teacher training and support. Recent scholarship has highlighted the need for preservice and practicing teachers to be trained in SEL, and to have opportunities to develop their own social and emotional competence, in order to support SEL implementation in schools (Mahoney et al., 2020; Schonert-Reichl, 2017). Findings from the present research suggest that implementing SEL programs may prompt changes in teachers’ burnout symptoms and sense of efficacy, and not in the desired direction for high-quality implementers. Thus, taking steps to better prepare 93 teachers for SEL implementation is an important implication of this research. Recent research suggests that programs targeting teachers’ SEL may be helpful. In a meta-analysis of the impact of SEL interventions on teachers, Oliveira and colleagues (2021) found that these interventions can be effective in improving teachers’ burnout symptoms. Therefore, preservice and ongoing teacher professional development programs can support teachers’ capacity for sustained quality SEL implementation by supporting the development of teachers’ social and emotional competence. As Schonert-Reichl (2017) points out, “Teachers are the engine that drives SEL programs and practices in schools and classrooms,” (p. 138). However, teachers are nested within a larger system, and without support embedded at all levels, teachers’ ability to sustain quality SEL implementation is limited (Mahoney et al., 2020). Research suggests systemic approaches to SEL that support the social-emotional competence and SEL capacity of educators can help maximize its benefits (CASEL, 2020; Oberle et al., 2016). Importantly, systemic approaches to SEL go beyond the teacher and classroom context and consider school-, district-, and community-levels factors that can support SEL implementation. Therefore, future research exploring SEL program implementation and sustainability should endeavour to further elucidate not only how teacher factors are related to SEL program implementation, but if and how teachers embedded in larger systems are supported in their SEL implementation, and how this affects implementation quality and outcomes for students. Expanding our understanding in this regard can increase the likelihood of children and youth gaining critical social-emotional competencies to support their thriving in the challenging future that lies ahead. 94 References Allinder, R.M. (1994). The relationship between efficacy and the instructional practice of special education teachers and consultants. Teacher Education and Special Education, 17, 86-95. Allison, P.D. (1999). Multiple regression: A primer. Pine Forge Press. Altman, D.G., Gore, S.M., Gardner, M.J., & Pocock, S.J. (1983). Statistical guidelines for contributors to medical journals. British Medical Journal (Clinical Research Ed.), 286(6376), 1489–1493. https://doi.org/10.1136/bmj.286.6376.1489 Anderson, R., Greene, M., & Loewen, P. (1988). Relationships among teachers’ and students’ thinking skills, sense of efficacy, and student achievement. Alberta Journal of Educational Research, 34(2), 148-165. Atwell, M.N., & Bridgeland, J.M. (2019). Ready to lead: A 2019 update of principals’ perspectives on how social and emotional learning can prepare children and transform schools. A report for the Collaborative for Social and Academic Learning (CASEL), Civic Enterprises, and Peter D. Hart Research Associates. https://casel.org/wp-content/uploads/2019/10/Ready-to-Lead_FINAL.pdf Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. Bandura, A. (1997). Self-efficacy: The exercise of control. Freeman. Battista, M.T. (1994). Teacher beliefs and the reform movement of mathematics. Phi Delta Kappan, 75(6), 463-463, 466-68, 470. Bear, G.G., Yang, C., Mantz, L.S., & Harris, A.B. (2017). School-wide practices associated with school climate in elementary, middle, and high schools. Teaching and Teacher Education, 63, 372-383. https://doi.org/10.1016/j.tate.2017.01.012 95 Beets, M.W., Flay, B.R., Vuchinich, S., Acock, A.C., Li, K.-K., & Allred, C. (2008). School climate and teachers’ beliefs and attitudes associated with implementation of the Positive Action Program: A diffusion of innovations model. Prevention Science, 9, 264-275. https://doi-org.ezproxy.library.ubc.ca/10.1007/s11121-008-0100-2 Begeny, J., Upright, J., Easton, J., Ehrenbock, C., & Tunstall, K. (2013). Validity estimates and functionality of materials and procedures used to monitor the implementation integrity of a reading intervention. Journal of Applied School Psychology, 29(3), 284-304. https://doi-org.ezproxy.library.ubc.ca/10.1080/15377903.2013.810187 Belfield, C., Bowden, B., Klapp, A., Levin, H., Shand, R., & Zander, S. (2015). The economic value of social and emotional learning. New York, NY: Center for Benefit-Cost Studies in Education, Teachers College, Columbia University. https://bit.ly/37MrgtY Berkel, C., Mauricio, A.M., Schoenfelder, E., & Sandler, I.N. (2011). Putting the pieces together: An integrated model of program implementation. Prevention Science, 12, 23-33. https://doi-org.ezproxy.library.ubc.ca/10.1007/s11121-010-0186-1 Brackett, M. A., Reyes, M.R., Rivers, S.E., Elbertson, N.A., & Salovey, P. (2012). Assessing teachers’ beliefs about social and emotional learning. Journal of Psychoeducational Assessment, 30(3), 219-236. https://doi.org/10.1177/0734282911424879 Braun, S.S., Roeser, R.W., Mashburn, A.J., & Skinner, E. (2019). Middle school teachers’ mindfulness, occupational health and well-being, and the quality of teacher-student interactions. Mindfulness, 10(2), 245-255. https://doi.org/10.1007/s12671-018-0968-2 Breitenstein, S.M., Gross, D., Garvey, C.A., Hill, C., Fogg, L., & Resnick, B. (2010). Implementation fidelity in community-based interventions. Research in Nursing & Health, 33, 164-173. https://doi.org/10.1002/nur.20373 96 Bridgeland, J.M, Bruce, M., & Hariharan, A. (2013). The missing piece: A national teacher survey on how social emotional learning can empower children and transform schools. A report for the Collaborative for Social and Academic Learning (CASEL), Civic Enterprises, and Peter D. Hart Research Associates. https://www.casel.org/wp-content/uploads/2016/01/the-missing-piece.pdf Brouwers, A., & Tomic, W. (1999). Teacher burnout, perceived self-efficacy in classroom management, and student disruptive behaviour in secondary education. Curriculum and Teaching, 14(2), 7–26. Brown, E.C., Low, S., Smith, B.H., & Haggerty, K.P. (2011). Outcomes from a school-randomized controlled trial of Steps to Respect: A Bullying Prevention program. School Psychology Review, 40(3), 423-443. Buehl, M.M., & Beck, J.S. (2015). The relationship between teachers’ beliefs and teachers’ practices. In H. Fives, & M.G. Gill (Eds.), International Handbook of Research on Teachers’ Beliefs (pp. 66-84). Routledge. Burley, W. W., Hall, B. W., Villeme, M. G., & Brockmeier, L. L. (1991). A path analysis of the mediating role of efficacy in first-year teachers’ experiences, reactions, and plans. Paper presented at the annual meeting of the American Educational Research Association, Chicago. Buss, M. T., & Hughes, J. N. (2007). Teachers’ attitudes toward emotions predict implementation of and satisfaction with a social-emotional curriculum. Paper presented at the Society for Prevention Research, Washington, DC. Cain, G., & Carnellor, Y. (2008). ‘Roots of Empathy’: A research study on its impact on teachers in Western Australia. Journal of Student Wellbeing, 2(1), 52-73. 97 Carpenter, G.W. (2017). Sensitivity analysis. In M. Allen (Ed.), The SAGE encyclopedia of communication research methods. SAGE Publications, Inc. https://dx-doi-org/10.4135/9781483381411 CASEL. (2020). The CASEL guide to schoolwide social and emotional learning. CASEL. https://schoolguide.casel.org/ CASEL. (2019). What is SEL?. https://casel.org/what-is-sel/ CASEL. (2015). 2015 CASEL Guide: Effective social and emotional learning programs: Middle and high school edition. CASEL. http://secondaryguide.casel.org/casel-secondary-guide.pdf CASEL. (2013). 2013 CASEL Guide: Effective social and emotional learning programs: Preschool and elementary school edition. CASEL. http://casel.org/wp-content/uploads/2016/01/2013-casel-guide-1.pdf Castillo, R., Fernández-Berrocal, P., & Brackett, M.A. (2013). Enhancing teacher effectiveness in Spain: A pilot study of The RULER approach to social and emotional learning. Journal of Education and Training Studies, 1(2), 263-272. https://doi.org/10.11114/jets.vli2.203 Choi, A. (2018). Emotional well-being of children and adolescents: Recent trends and relevant factors. OECD Education Working Papers, No. 169. OECD Publishing. https://doi.org/10.1787/41576fb2-en Cohen, J. (1969). Statistical power analysis for the behavioural sciences. Academic Press Cohen, J. (1992). A power primer. Psychological bulletin, 112(1), 155-159. https://doi.org/10.1037//0033-2909.112.1.155 98 Collie, R.J., Shapka, J.D., & Perry, N.E. (2011). Predicting teacher commitment: The impact of school climate and social-emotional learning. Psychology in School, 48(10), 1034-1048. https://doi.org/10.1002/pits.20611 Collie, R.J., Shapka, J.D., & Perry, N.E. (2012). School climate and social-emotional learning: Predicting teachers stress, job satisfaction, and teaching efficacy. Journal of Educational Psychology, 104(4), 1189-1204. https://doi.org/10.1037/a0029356 Collie, R.J., Shapka, J.D., Perry, N.E., & Martin, A.J. (2015). Teachers’ beliefs about social-emotional learning: Identifying teacher profiles and their relations with job stress and satisfaction. Learning and Instruction, 39, 148-157. https://doi.org/10.1016/j.learninstruc.2015.06.002 Corcoran, R.P., Cheung, A.C.K., Kim, E., & Xie, C. (2018). Effective universal school-based social and emotional learning programs for improving academic achievement: A systematic review and meta-analysis of 50 years of research. Educational Research Review, 25, 56-72. https://doi.org/10.1016/j.edurev.2017.12.001 Cronk, B.C. (2018). How to use SPSS: A step-by-step guide to analysis and interpretation. Routledge. Dane, A.V., & Schneider, B.H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18(1), 23-45. Denham, S.A., & Brown, C. (2010). “Plays nice with others”: Social–Emotional learning and academic success. Early Education and Development, 21(5), 652-680. https://doi.org/10.1080/10409289.2010.497450 99 Desimone, L., Smith, T., & Frisvold, D. (2010). Survey measures of classroom instruction: Comparing student and teacher reports. Educational Policy, 24, 267–329. https://doi.org/10.1177/0895904808330173 Dix, K.L., Slee, P.T., Lawson, M.J., & Keeves, J.P. (2012). Implementation quality of whole-school mental health promotion and students’ academic performance. Child and Adolescent Mental Health, 17(1), 45–51. https://doi.org/10.1111/j.1475-3588.2011.00608.x Domitrovich, C.E., Bradshaw, C.P., Poduska, K.H., Buckley, J.A., Olin, S., …, & Ialongo, N.S. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1(3), 6-28. https://doi.org/10.1080/1754730X.2008.9715730 Domitrovich, C.E., Yibing, L., Mathis, E.T., & Greenberg, M.T. (2015a). Individual and organizational factors associated with teacher self-reported implementation of the PATHS curriculum. Journal of School Psychology, 76, 168-185. https://doi.org/10.1016/j.jsp.2019.07.015 Domitrovich, C.E., Pas, E.T., Bradshaw, C.P., Becker, K.D., Keperling, J.P., Embry, D.D., & Ialongo, N. (2015b). Individual and school organizational factors that influence implementation of the PAX Good Behavior Game intervention. Prevention Science, 16, 1064-1074. https://doi.org/10.1007/s11121-015-0557-8 Dowling, K., & Barry, M.M. (2020a). Evaluating the implementation quality of a social and emotional learning program: A mixed methods approach. International Journal of Environmental Research and Public Health, 17(9),3249. https://doi.org/10.3390/ijerph17093249 100 Dowling, K., & Barry, M.M. (2020b). The effects of implementation quality of a school-based social and emotional well-being program on students’ outcomes. European Journal of Investigation in Health, Psychology and Education, 10, 595-614. https://doi.org/10.3390/ejipe10020044 Downey, R.G., & King, C.V. (1998). Missing data in Likert ratings: A comparison of replacement methods. The Journal of General Psychology, 125(2), 175-191. http://doi.org/10.1080/00221309809595542 Dugard, P., & Todman, J. (2006). Analysis of pre-test-post-test control group designs in educational research. Educational Psychology: An International Journal of Experimental Educational Psychology, 15(2), 181-198. https://doi.org/10.1080/0144341950150207 Duncan, R., Washburn, I.J., Lewis, K.M., Bavarian, N., DuBois, D.L., Acock, A.C., Vuchinich, S., & Flay, B.R. (2017). Can universal SEL programs benefit universally? Effects of the Positive Action Program on multiple trajectories of social-emotional and misconduct behaviors. Prevention Science, 18, 214-224. https://doi.org/10.1007/s11121-016-0745-1 Durlak, J.A. (1998). Why program implementation is important. Journal of Prevention & Intervention in the Community, 17(2), 5-18. https://doi.org/10.1200/J005v17n02_02 Durlak, J.A. (2015). What everyone should know about implementation. In J.A. Durlak, C.E. Domitrovich, R.P. Weissberg, & T.P. Gullotta (Eds.), Handbook of social and emotional learning: Research and practice (pp. 395-405). Guilford Press. Durlak, J.A., & DuPre, E.P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350. https://doi.org/10.1007/s10464-008-9165-0 101 Durlak, J.A., Dymnicki, A.B., Taylor, R.D., Weissberg, R.P., & Schellinger, K.B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405-432. https://bit.ly/2CZDLnS Durlak, J.A., Domitrovich, C.E., Weissberg, R.P, & Gullotta, T.P. (Eds.). (2015). Handbook of social and emotional learning: Research and practice. Guilford Press. Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W.B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237–256. https://doi.org/10.1093/her/18.2.237 Dusenbury, L., Brannigan, R., Hansen, W.B., Walsh, J., & Falco, M. (2005). Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research, 20(3), 308-313. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7(1), 1-26. http://doi.org/10.1214/aos/1176344552 Elias, M. J. (2014). The future of character education and social-emotional learning: The need for whole school and community-linked approaches. Journal of Research in Character Education, 10(1), 37–42. Evans, R., Murphy, S., & Scourfield, J. (2015). Implementation of a school-based social and emotional learning intervention: Understanding diffusion processes within complex systems. Prevention Science, 16, 754-764. https://doi.org/10.1007/s11121-015-0552-0 Fetters, M.K., Czerniak, C.M., Fish, L., & Shawberry, J. (2002). Confronting, challenging, and changing teachers’ beliefs: Implications from a local systemic change professional 102 development program. Journal of Science Teacher Education, 13(2), 101-130. https://doi.org/10.1023/A:1015113613731 Fimian, M. J. (1984). The development of an instrument to measure occupational stress in teachers: The teacher stress inventory. Journal of Occupational Psychology, 57(4), 277-293. https://doi.org/10.1111/j.2044-8325.1984.tb00169.x Fives, H., & Buehl, M.M. (2012). Spring cleaning for the “messy” construct of teachers’ beliefs: What are they? Which have been examined? What can they tell us? In K.R. Harris, S. Graham, & T. Urdan (Eds.), APA Educational Psychology Handbook: Vol. 2. Individual Differences and Cultural and Contextual Factors (pp. 471-499). American Psychological Association. Fixsen, D.L., Naoom, S.F., Blasé, K.A., Friedman, R.M., & Wallace, F. (2005). Implementation research: A synthesis of the literature (FMHI Publication No. 231). University of South Florida, Louis de la Parte Florida Mental Health Institute, the National Implementation Research Network. https://unc.live/34ATKES Galea, S., Merchant, R.M., & Lurie, N. (2020). The mental health consequences of COVID-19 and physical distancing: The need for prevention and early intervention. JAMA Internal Medicine, 180(6), 817-818. https://doi.org/10.1001/jamainternmed.2020.1562 Garibaldi, M., Ruddy, S., Osher, D., & Kendziora, K. (2015). Assessment of climate and conditions for learning. In J.A. Durlak, C.E. Domitrovich, R.P. Weissberg, & T.P. Gullotta (Eds.), Handbook of social and emotional learning: Research and practice (pp. 348-359). Guilford Press. Geving, A. M. (2007). Identifying the types of student and teacher behaviours associated with teacher stress. Teaching and Teacher Education, 23, 624–640. 103 Gibson, S., & Dembo, M. (1984). Teacher efficacy: A construct validation. Journal of Educational Psychology, 76(4), 569-582. https://doi.org/10.1037/0022-0663.76.4.569 Greenberg, M.T., Katz, D.A., & Klein, L.C. (2015). The potential effects of SEL on biomarkers and health outcomes: A promissory note. In J.A. Durlak, C.E. Domitrovich, R.P. Weissberg, & T.P. Gullotta (Eds.), Handbook of social and emotional learning: Research and practice (pp. 81-96). Guilford Press. Greenberg, M.T., Brown, J.L., & Abenavoli, R.M. (2016). Teacher stress and health: Effects on teachers, students, and schools. Edna Bennett Pierce Prevention Research Center: Pennsylvania State University. https://rwjf.ws/3hFKqGO Greenberg, M.T., Domitrovich, C.E., Weissberg, R.P., & Durlak, J.A. (2017). Social and emotional learning as a public health approach to education. The Future of Children, 27(1), 13-32. https://bit.ly/2L80diX Guskey, T.R. (1998). Teacher efficacy, self-concept, and attitudes toward the implementation of instructional innovation. Teaching and Teacher Education, 4(1), 63-39. https://doi.org/10.1016/0742-051X(88)90025-X Hamre, B. K., Pianta, R. C., Mashburn, A., & Downer, J. T. (2012). Promoting young children's social competence through the preschool PATHS curriculum and MyTeachingPartner professional development resources. Early Education and Development, 23(6), 809–832. https://doi.org/10.1080/10409289.2011.607360 Han, S. S., & Weiss, B. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33(6), 665-679. https://doi.org/10.1007/s10802-005-7646-2 104 Haney, J.J., Czerniak, C., & Lumpe, A.T. (1996). Teachers’ beliefs and intentions regarding the implementation of science education reform strands. Journal of Research in Science Teaching, 3(9), 971-993. https://doi-org.ezproxy.library.ubc.ca/10.1002/(SICI)1098-2736(199611)33:9<971::AID-TEA2>3.0.CO;2-S Hawkins, J. D., Kosterman, R., Catalano, R. F., Hill, K. G., & Abbott, R. D. (2005). Positive adult functioning through social development intervention in childhood: Long-term effects from the Seattle Social Development Project. Archives of Pediatrics & Adolescent Medicine, 159, 25–31. Heckman, J.J., Stixrud, J., & Urzua, S. (2006). The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor Economics, 24(3), 411-482. https://doi.org/10.1086/504455 Herman, K.C., Hickmon-Rosa, J., & Reinke, W.M. (2018). Empirically derived profiles of teacher stress, burnout, self-efficacy, and coping and associated student outcomes. Journal of Positive Behavior Interventions, 20(2), 90–100. https://doi.org/10.1177/1098300717732066 Hill, K.G., Bailey, J.A., Steeger, C.M., Hawkins, J.D., Catalano, R.F., Kosterman, R., Epstein, M., & Abbott, R.D. (2020). Outcomes of childhood preventive intervention across 2 generations: A nonrandomzied controlled trial. JAMA Pediatrics, E1-E8. https://doi.org/10.1001/jamapediatrics.2020.1310 Humphrey, N., Hennessey, A., Lendrum, A., Wigelsworth, M., Turner A, Panayiotou, M., et al. (2018). The PATHS curriculum for promoting social and emotional well-being among children aged 7–9 years: a cluster RCT. Public Health Research, 6(10). 105 Jennings, P.A. (2015). Early childhood teachers’ well-being, mindfulness, and self-compassion in relationship to classroom quality and attitudes towards challenging behavior. Mindfulness, 6(4), 732-743. https://doi.org/10.1007/s12671-014-0312-4 Jennings, P. A., & Frank, J. L. (2015). Inservice preparation for educators. In J. A. Durlak, C. E. Domitrovich, R. P. Weissberg, & T. P. Gullotta (Eds.), Handbook of social and emotional learning: Research and practice (p. 422–437). Guilford Press. Jennings, P., Frank, J., & Montgomery, M. (2020). Social and emotional learning for educators. In C.N. Singh, & A.K. Duraiappah (Eds.), Rethinking learning: A review of social and emotional learning frameworks for education systems (pp. 125-154). New Delhi. UNESCO MGIEP. https://mgiep.unesco.org/rethinking-learning Jennings, P.A., & Greenberg, M.T. (2009). The prosocial classroom: Teacher social and emotional competence in relation to student and classroom outcomes. Review of Educational Research, 79(1), 491-525. https://doi.org/10.3102/0034654308325693 Jones, D.E., Greenberg, M., & Crowley, M. (2015). Early social-emotional functioning and public health: The relationship between kindergarten social competence and future wellness. American Journal of Public Health, 105(11), 2283-2290. https://doi.org/10.2105/AJPH.2015.302630 Jones, S.M., Bailey, R., Brush, K., & Kahn, J. (2018). Preparing for effective SEL implementation. Harvard Graduate School of Education. Jones, S.M., & Bouffard, S.M. (2012). Social and emotional learning in schools: From programs to strategies. Social Policy Report, 26(4), 1-33. https://doi.org/10.1002/j.2379-3988.2012.tb00073.x 106 Jones, S. M., Brush, K., Bailey, R., Brion-Meisels, G., McIntyre, J., Kahn, J., Nelson, B., & et al. (2017). Navigating SEL from the inside out: Looking inside & across 25 leading SEL programs: A practical resource for schools and OST providers: (Elementary school focus). Harvard University, Graduate School of Education. http://www.wallacefoundation.org/knowledge-center/Documents/Navigating-Social-and-Emotional-Learning-from-the-Inside-Out.pdf Jones, S.M., & Doolittle, E.J. (2017). Social and emotional learning: Introducing the issue. The Future of Children, 27(1), 3-11. https://bit.ly/37kK8Qq Jones, S.M., & Kahn, J. (2017). The evidence base for how we learn: Supporting students’ social, emotional, and academic development. https://bit.ly/34uwk49 Jones, S.M., McGarrah, M.W., & Kahn, J. (2019). Social and emotional learning: A principled science of human development in context. Educational Psychologist, 54(3), 129-143. https://doi.org/10.1080/00461520.2019.1625776 Joyce, B.R., & Showers, B. (2002). Designing training and peer coaching: Our needs for learning. In J. Bruce & B. Showers (Eds.), Student Achievement Through Staff Development, 3rd ed. Association for Supervision and Curriculum Development. Kagan, D.M. (1992). Implications of research on teacher belief. Educational Psychologist, 27(1), 65-90. https://doi.org/10.1207/s15326985ep2701_6 Kallestad, J.H., & Olweus, D. (2003). Predicting teachers’ and schools’ implementation of the Olweus Bullying Prevention Program: A multilevel study. Prevention & Treatment, 6(1), Article 21. https://doi.org/10.1037/1522-3736.6.1.621a 107 Kim, M.C., Kozan, K., Kim, W., & Koehler, A.A. (2013). Technology integration: From implementation to dynamic scaffolding. In M.G. Moore (Ed.), Handbook of Distance Education, third edition (pp. 299-317). Taylor & Francis. Klimes-Dougan, B., August, G.J., Lee, C-Y.S., Realmuto, G.M., Bloomquist, M.L., Horowitz, J.L., & Eisenberg, T.L. (2009). Practitioner and site characteristics that relate to fidelity of implementation: The Early Risers Prevention Program in a going-to-scale intervention trial. Professional Psychology: Research and Practice, 40(5), 467-475. https://doi.org/10.1037/a0014623 Koziol, S.M., & Burns, P. (1986). Teachers’ accuracy in self-reporting about instructional practices using a focused self-report inventory. The Journal of Educational Research, 4, 205-209. https://doi.org/10.1080/00220671.1986.10885678 Kress, J. S., & Elias, M. J. (2006). School-based social and emotional learning programs. In K. A. Renninger & I. E. Sigel (Eds.), Handbook of child psychology, Vol. 4, 6th ed. (pp. 592–618). Wiley. Kumpfer, K.L., Alvarado, R., Tait, C., & Turner, C. (2002). Effectiveness of school-based family and children’s skills training for substance abuse prevention among 6-8 year-old rural children. Psychology of Addictive Behaviors, 16(4S), S65-S71. Kwak, S.K., & Kim, J.H. (2017), Statistical data preparation: Management of missing values and outliers. Korean Journal of Anesthesiology, 70(4), 407-411. https://doi.org/10.4097/kjae.2017.70.4.407 Kyriacou, C. (2001). Teacher stress: Direction for future research. Educational Review, 53(1), 27-35. https://doi.org/10.1080/00131910120033628 108 Leppink, J., Winston, K., & O’Sullivan, P. (2016). Statistical significance does not imply a real effect. Perspectives on Medical Education, 5, 122-124. https://doi.org/10.1007/s40037-016-0256-6 Low, S., Cook, C. R., Smolkowski, K., & Buntain-Ricklefs, J. (2015). Promoting social-emotional competence: An evaluation of the elementary version of Second Step®. Journal of School Psychology, 53(6), 463–477. https://doi.org/10.1016/j.jsp.2015.09.002 Low, S., Smolkowski, K., & Cook, C.R. (2016). What constitutes high-quality implementation of SEL programs? A latent class analysis of Second Step® implementation. Prevention Science, 17, 981–991. https://doi.org/10.1007/s11121-016-0670-3 Low, S., Van Ryzin, M. J., Brown, E. C., Smith, B. H., & Haggerty, K. P. (2014). Engagement matters: Lessons from assessing classroom implementation of Steps to Respect: A Bullying Prevention Program over a one-year period. Prevention Science, 15, 165–176. Lumpe, A.T., Haney, J.J., & Czerniak, C.M. (2000). Assessing teachers’ beliefs about their science teaching context. Journal of Research in Science Teaching, 37(3), 275-292. https://doi.org/10.1002/(SICI)1098-2736(200003)37:3<275::AID-TEA4>3.0.CO;2-2 Mahoney, J.L., Durlak, J.A., & Weissberg, R.P. (2018). An update on social and emotional learning outcomes. Phi Delta Kappan, 100(4), 18-23. Mahoney, J. L., Weissberg, R. P., Greenberg, M. T., Dusenbury, L., Jagers, R. J., Niemi, K., Schlinger, M., Schlund, J., Shriver, T. P., VanAusdal, K., & Yoder, N. (2020). Systemic social and emotional learning: Promoting educational success for all preschool to high school students. American Psychologist. Advance online publication. http://dx.doi.org/10.1037/amp0000701 109 Martinez, R.G., Lewis, C.C., & Weiner, B.J. (2014). Instrumentation issues in implementation science. Implementation Science, 9(118), 1-9. https://doi.org/10.1186/s13012-014-0118-8 Maslach, C. (1999). Progress in understanding teacher burnout. In R. Vandenberghe, & A.M. Huberman (Eds.), Understanding and preventing teacher burnout: A sourcebook of international research and practice (pp. 211-222). Cambridge University Press. https://doi.org/10.1017/CBO9780511527784.014 Maslach, C., Jackson, S.E., & Leiter, M.P. (1997). Maslach burnout inventory, 3rd edition. Consulting Psychologists Press. Maslach, C., Schaufeli, W.B., & Leiter, M.P. (2001). Job burnout. Annual Review of Psychology, 52(1), 397-422. McCormick, M.P., Capella, E., O’Connor, E.E., & McClowry, S.G. (2015). Context matters for social-emotional learning: Examining variation in program impact by dimensions of school climate. American Journal of Community Psychology, 56, 101-119. https://doi.org/10.1007/s10464-015-9733-z Midgley, C., Feldlaufer, H., Eccles, J.S. (1989). Change in teacher efficacy and student self- and task-related beliefs in mathematics during the transition to junior high school. Journal of Educational Psychology, 81(2), 247-258. https://doi.org/10.1037/0022-0663.81.2.247 Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., Gheen, M., Kaplan, A., Kumar, R., Middleton, M.J., Nelson, J., Roeser, R., & Urdan, T. (2000). Manual for the Patterns of Adaptive Learning Scales (PALS). University of Michigan. Milner, A.R., Sondergeld, T.A., Demir, A., Johnson, C.C., & Czerniak, C.M. (2012). Elementary teachers’ beliefs about teaching science and classroom practice: An examination of 110 pre/post NCLB testing in science. Journal of Science Teacher Education, 23(2), 111-132. https://doi.org/10.1007/s10972-011-9230-7 Moffitt, T.E., Arseneault, L., Belsky, D., Dickson, N., Hancox, R.J., Harrington, H., …Caspi, A. (2011). A gradient of childhood self-control predicts health, wealth, and public safety. Proceedings of the National Academy of Sciences, 108, 2693-2698. Moore, W., & Esselman, M. (1992, April). Teacher efficacy, power, school climate and achievement: A desegregating district’s experience. Paper presented at the annual meeting of the American Educational Research Association, San Francisco. Myers, D.C., Durlak, J.A., & Wandersman, A. (2012). The quality implementation framework: A synthesis of critical steps in the implementation process. American Journal of Community Psychology, 50, 462-480. https://doi.org/10.1007/s10464-012-9522-x Neff, K. D. (2003). The development and validation of a scale to measure self-compassion. Self and Identity, 2, 223-250. https://doi.org/10.1080/15298860309027 Oberle, E., Schonert-Reichl, K.A., Hertzman, C., & Zumbo, B.D. (2014). Social-emotional competencies make the grade: Predicting academic success in early adolescence. Journal of Applied Developmental Psychology, 35, 138-147. https://doi.org/10.1016/j.appdev.2014.02.004 Oberle, E., Domitrovich, C.E., Meyers, D.C., & Weissberg, R.P. (2016). Establishing systemic social and emotional learning approaches in schools: A framework for schoolwide implementation. Cambridge Journal of Education, 46(3), 277-297. https://doi.org/10.1080/0305764X.2015.1125450 111 Oberle, E., & Schonert-Reichl, K.A. (2016). Stress contagion in the classroom? The link between classroom teacher burnout and morning cortisol in elementary school students. Social Sciences & Medicine, 159, 30-37. https://doi.org/10.1016/j.socscimed.2016.04.031 Oberle, E., Gist, A., Cooray, M.S., & Pinto, J.B.R. (2020). Do students notice stress in teachers? Associations between classroom teacher burnout and students’ perceptions of teacher social-emotional competence. Psychology in Schools, 57(11), 1741-1756. https://doi.org/10.1002/pits.22432 Oliveira, S., Roberto, M.S., Veiga-Simão, A.M., & Marques-Pinto, A. (2021). A meta-analysis of the impact of social and emotional learning interventions on teachers’ burnout symptoms. Educational Psychology Review. Advance online publication. https://doi.org/10.1007/s10648-021-09612-x Osher, D., & Berg, J. (2017). School climate and social and emotional learning: The integration of two approaches. Edna Bennet Pierce Prevention Research Center, Pennsylvania State University. https://www.prevention.psu.edu/uploads/files/rwjf443059.pdf Pajares, M.F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 62(3), 307-332. https://doi.org/10.3102/00346543062003307 Ransford, C.R., Greenberg, M.T., Domitrovich, C.E., Small, M., & Jacobson, L. (2009). The role of teachers’ psychological experiences and perceptions of curriculum supports on the implementation of a social and emotional learning curriculum. School Psychology Review, 38(4), 510-532. Reyes, R.M., Brackett, M.A., Rivers, S.E., Elbertson, N.A, & Salovey, P. (2012). The interaction effects of program training, dosage, and implementation quality on targeted student 112 outcomes for the RULER approach to social and emotional learning. School Psychology Review, 41(1), 82-99. https://bit.ly/37hgHyZ Ringwalt, C. L., Pankratz, M. M., Gottfredson, N., Jackson-Newsom, J., Dusenbury, L., Giles, S. M., Curry, D., & Hansen, W. (2009). The effects of students’ curriculum engagement, attitudes towards their teachers, and perception of their teachers’ skills on school-based prevention curriculum outcomes. Journal of Drug Education, 39, 223–237. Roeser, R. W., & Midgley, C. (1997). Teachers’ views of issues involving students’ mental health. The Elementary School Journal, 98, 115-133. https://doi.org/10.1086/461887 Roeser, R. W., Schonert-Reichl, K. A., Jha, A., Cullen, M., Wallace, L., Wilensky, R., Oberle, E., Thomson, K., Taylor, C., & Harrison, J. (2013). Mindfulness training and reductions in teacher stress and burnout: Results from two randomised, waitlist-control field trials. Journal of Educational Psychology, 105, 787-804. https://doi.org/10.1037/a0032093 Rohrbach, L.A., Graham, J.W., & Hansen, W.B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22(2), 237-260. https://doi.org/10.1006/pmed.1993.1020 Ross, J.A. (1992). Teacher efficacy and the effect of coaching on student achievement. Canadian Journal of Education, 17(1), 51-65. Rowe, H. L., & Trickett, E. J. (2018). Student diversity representation and reporting in universal school-based social and emotional learning programs: Implications for generalizability. Educational Psychology Review, 30, 559–583. https://doi.org/10.1007/s10648-017-9425-3 Sanetti, L.M.H., & Kratochwill, T.R. (2011). An evaluation of the Treatment Integrity Planning Protocol and two schedules of treatment integrity self-report: Impact on implementation 113 and report accuracy. Journal of Educational and Psychological Consultation, 21(4), 284-308. https://doi.org/10.1080/10474412.2011.620927 Schonert-Reichl, K.A. (2017). Social and emotional learning and teachers. The Future of Children, 27(1), 137-155. https://www.jstor.org/stable/44219025 Schonert-Reichl K.A., & Weissberg R.P. (2014) Social and emotional learning: Children. In T.P. Gullotta & M. Bloom, (Eds.), Encyclopedia of Primary Prevention and Health Promotion. Springer. https://doi.org/10.1007/978-1-4614-5999-6_133 Schonert-Reichl, K.A., Oberle, E., Lawlor, M.S., Abbott, D., Thomson, K., Oberlander, T.F., & Diamond, A. (2015). Enhancing cognitive and social-emotional development through a simple-to-administer mindfulness-based school program for elementary school children: A randomized controlled trial. Developmental Psychology, 51(1), 52-66. https://doi.org/10.1037/a0038454 Schulting, A. B., Malone, P. S., & Dodge, K. A. (2005). The effect of school-based kindergarten transition policies and practices on child academic outcomes. Developmental Psychology, 41(6), 860–871. https://doi.org/10.1037/0012-1649.41.6.860 Shriver, T.P., & Weissberg, R.P. (2020). A response to constructive criticism of social and emotional learning. Phi Delta Kappan, 101(7), 52-57. https://doi.org/10.1177/0031721720917543 Siegel, S. (1957) Nonparametric statistics. The American Statistician, 11(3), 13–19. https://doi.org/10.2307/2685679 Siegel, S., & Castellan, N.J. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.). McGraw-Hill. 114 Skaalvik, E.M., & Skaalvik, S. (2010). Teacher self-efficacy and teacher burnout: A study of relations. Teaching and Teacher Education, 26(4), 1059-1069. https://doi.org/10.1016/j.tate.2009.11.001 Sklad, M., Diekstra, R., De Ritter, M., & Ben, J. (2012). Effectiveness of school-based social, emotional, and behavioral programs: Do they enhance students’ development in the area of skill, behavior, and adjustment? Psychology in the Schools, 49(9), 892-909. https://doi.org/10.1002/pits.21641 Social and Character Development Research Consortium. (2010). Efficacy of Schoolwide Programs to Promote Social and Character Development and Reduce Problem Behavior in Elementary School Children (NCER 2011–2001). National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Solomon, D., Battistich, V., Kim, D., & Watson, M. S. (1997). Teacher practices associated with students’ sense of the classroom as a community. Social Psychology of Education, 1, 235–267. Stipek, D.J., Givvin, K.B., Salmon, J.M., & MacGyvers, V.L. (2001). Teachers’ beliefs and practices related to mathematics instruction. Teaching and Teacher Education, 17(2), 213-226. https://doi.org/10.1016/S0742-051X(00)00052-4 Taylor, C., Harrison, J., Haimovitz, K., Oberle, E., Thomson, K., Schonert-Reichl, K., & Roeser, R. (2016). Examining ways that a mindfulness-based intervention reduces stress in public school teachers: A mixed-methods study. Mindfulness, 7, 115-129. https://doi.org/10.1007/s12671-015-0425-4 Taylor, R.D., Oberle, E., Durlak, J.A., & Weissberg, R.P. (2017). Promoting positive youth development through school-based social and emotional learning interventions: A meta- 115 analysis of follow-up effects. Child Development, 88(4), 1156-1171. https://doi.org/10.1111/cdev.12864 Trentacosta, C.J., & Fine, S.E. (2010). Emotion knowledge, social competence, and behavior problems in childhood and adolescence: A meta-analytic review. Social Development, 19, 1-29. https://doi.org/10.1111/j.1467-9507.2009.00543.x Tschannen-Moran, M., & Woolfolk Hoy, A. (2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17(7), 783-805. https://doi.org/10.1016/S0742-051X(01)00036-1 Tyson, O., Roberts, C.M., & Kane, R. (2009). Can implementation of a resilience program for primary school children enhance the mental health of teachers? Australian Journal of Guidance & Counselling, 19, 116–130 Vroom, E.B., Massey, O.T., Yampolskaya, S., & Levin, B.L. (2020). The impact of implementation fidelity on student outcomes in the Life Skills Training program. School Mental Health, 12, 113-123. https://doi.org/10.1007/s12310-019-09333-1 Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, L., Blachman, M., Dunville, R., & Saul, J. (2008). Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology, 41, 171-181. Wanless, S.B., & Domitrovich, C.E. (2015). Readiness to implement school-based social-emotional learning interventions: Using research on factors related to implementation to maximize quality. Prevention Science, 16, 1037-1043. https://doi.org/10.1007/s11121-015-0612-5 116 Wanless, S.B., Rimm-Kauffman, S.E., Abry, T., Larsen, R.A., & Patton, C.L. (2015). Engagement in training as a mechanism to understanding fidelity of implementation of the Responsive Classroom approach. Prevention Science, 16, 1107-1116. https://doi.org/10.1007/s11121-014-0536-5 Weare, K., & Nind, M. (2011). Mental health promotion and problem prevention in schools: What does the evidence say? Health Promotion International, 26(S1), 29-69. https://doi.org/10.1093/heapro/dar075 Wehby, J. H., Maggin, D. M., Moore Partin, T. C., & Robertson, R. (2012). The impact of working alliance, social validity, and teacher burnout on implementation fidelity of the Good Behavior Game. School Mental Health, 1–12. Wenz-Gross, M., Yoo, Y., Upshur, C. C., & Gambino, A. J. (2018, October). Pathways to kindergarten readiness: The roles of Second Step Early Learning curriculum and social emotional, executive functioning, preschool academic and task behavior skills. Frontiers in Psychology, 9, 1-19. https://doi.org/10.3389/fpsyg.2018.01886 Wiglesworth, M., Lendrum, A., Oldfield, J., Scott, A., ten Bokkel, I., Tate, K., Emery, C. (2016). The impact of trial stage, developer involvement and international transferability on universal social and emotional learning programme outcomes: A meta-analysis. Cambridge Journal of Education, 46, 347–376. Woolfolk, A. E., Rosoff, B., & Hoy, W. K. (1990) Teachers’ sense of efficacy and their beliefs about managing students. Teaching & Teacher Education, 6, 137-148. https://doi.org/10.1016/0742-051X(90)90031-Y Zins, J.E., Bloodworth, M.R., Weissberg, R.P., & Walberg, H.J. (2007). The scientific base linking social and emotional learning to school success. Journal of Educational and 117 Psychological Consultation, 17(2-3), 191-210. https://doi.org/10.1080/10474410701413145 Zinsser, K.M., Shewark, E.A., Denham, S.A., & Curby, T.W. (2014). A mixed-method examination of preschool teacher beliefs about social-emotional learning and relations to observed emotional support. Infant and Child Development, 23, 471-493. https://doi.org/10.1002/icd.1843 118 Appendices Appendix A Program Evaluation Descriptions A.1 SMART and MindUP Study This study’s aim was to examine the singular and combined effects on student and teacher outcomes of two mindfulness-based programs, one for educators and one for students. The program for teachers used in the study was SMART in Education, a program designed for Kindergarten to Grade 12 educators and professional support staff. With elements drawn from the Mindfulness-Based Stress Reduction (MBSR) program, Western psychology, and contemplative practices, the goal of SMART is to increase educators’ ability to integrate mindfulness into daily life and challenge self-limiting beliefs that can cause suffering in order to improve their overall well-being. SMART is a 20-hour course delivered in person to groups of between 12 and 25 participants. Divided up into eight two-hour sessions and a four-hour retreat, SMART uses a mixture of didactic teaching, experiential exercises, visualizations, flexible processes, and home practice to ensure learning and transfer, and to accommodate different learning styles. SMART facilitators are highly trained with over 100 hours of training required in order to be certified. The course content includes topics in the following three areas: mindfulness-based stress reduction, mindfulness-based emotion skills, and mindfulness-based compassion and forgiveness (Taylor et al., 2016). The program for students used in this study was the MindUP program. It is a mindfulness-based SEL program designed to foster children’s social and emotional competence, psychological well-being, and self-regulation and decrease acting-out behaviors and aggression. The program accomplishes this through 15 lessons implemented by classroom teachers with the objective to enhance children’s self-awareness, social awareness, focused attention, self-regulation, problem solving, pro-social behaviors 119 (helping, sharing, and cooperating), and positive human qualities, such as happiness, optimism, and altruism. The study took place in the fall of 2011 in two school districts in Western Canada. Participants included children ages 9-12 (N = 546), and their teachers (N = 16). Teachers (and their students) were randomly assigned to four conditions: 1) SMART only condition (N = 2): Teachers in this condition received the SMART training; 2) SMART + MindUP condition (N = 3): Teachers in this condition received the same SMART training as the condition 1. They also implemented the MindUP program following a one-day training; 3) MindUP only condition (N=8): Teachers in this condition only taught the MindUP program receiving the same training and curricular support as condition 2; and, 4) Control condition (N = 5): Teachers in this condition did not receive SMART training or implement the MindUP program. Although student outcome data were collected, this research used teacher data only. Study 1 employed pre-test data and implementation diaries from the MindUP only condition (N = 8). Study 2 employed pre- and post-test data and implementation diaries from the MindUP only condition (N = 8). 120 A.2 Kindness in the Classroom Study The purpose of this randomized controlled trial was to evaluate the effectiveness of the Kindness in the Classroom (KitC) Curriculum, a universal SEL program that promotes kindness, resilience, and school success for elementary school children. The program accomplishes this with activities that promote knowledge and skills associated with social-emotional competence delivered across 16 lessons. The study took place in 2014 and 2015 in two school districts in Western Canada. Participants included children in grade four and five (N = 700) and their teachers (N = 30) who were randomly assigned to two conditions: 1) KitC intervention (N = 14): Teachers in this condition attended a half-day training session to learn to use KitC, then they delivered the lessons over a six-month period; and, 2) Control condition (N = 15): Teachers in this condition did not receive KitC training or implement the lessons. Although student outcome data were collected, this research used teacher data only. Study 1 employed pre-test data and implementation diaries from the KitC condition (N = 14). Study 2 employed pre- and post-test data and implementation diaries from the KitC condition (N = 15). 121 A.3 Taxi Dog RCT This randomized controlled trial sought to evaluate the effectiveness and feasibility of the Taxi Dog Educational curriculum, a universal SEL program designed to foster the social-emotional competence, resilience, and school success of children in early elementary school. The curriculum is based on the children’s book “The Adventures of Taxi Dog” featuring the character Maxi the Taxi Dog. Students learn self-regulation and SEL skills through nine play-based lessons and accompanying activities that include video stories and interactive tools on the program’s website. The study took place in a large urban school district in Western Canada. Participants included children in kindergarten to grade three (N = 375) and their teachers (N = 15) who were randomly assigned to two conditions: 1) Taxi Dog condition (N = 9): Teachers in this condition were provided with the program manual, accompanying materials (puppets, videos, books, and posters), and access to the Taxi Dog website which provides additional resources; and, 2) Control condition (N = 6): Teachers in this condition did not implement the Taxi Dog program. Teachers in the Taxi Dog condition completed online training prior to implementation that provided them with an overview of the Taxi Dog program, the links between SEL and the Taxi Dog curriculum, an overview of each of the Taxi Dog Lessons, a step-by-step guide for delivery of each of the lesson plans, and lesson plans, videos, and materials. In addition, teachers received implementation support from a certified Taxi Dog program provider during program implementation. Student data were collected for this study, but this research only used teacher data. Study 1 employed pre-test data and implementation diaries from the Taxi Dog condition (N = 9). Study 2 did not employ data from this study. 122 A.4 SEL and Biology Feasibility Study The aim of this randomized controlled trial was two-fold: to evaluate the effectiveness of the MindUP program for middle school students in terms of its effect on students’ healthy adjustment, social and emotional skills, and success in school; and to learn more about the relation between MindUP and biology through examination of students’ stress physiology (via cortisol) and genetic and epigenetic factors. A collaboration between the Faculty of Education and the Faculty of Medicine at The University of British Columbia, this study took place in one middle school in Western Canada. Participants included grade 6 and 7 teachers (N = 4) and their students (N = 120) who were randomly assigned to two conditions: 1) MindUP condition (N = 2): Teachers in this condition participated in a one-day training in the MindUP program, then implemented the lessons over 15 weeks; and 2) Control condition (N = 2): Teachers in this condition did not implement the MindUP Program. Student data were collected for this study, but this research only used teacher data. Study 1 employed pre-test data and implementation diaries from the MindUP condition (N = 2). Study 2 employed pre- and post-test data and implementation diaries from the MindUP condition (N = 2). 123 Appendix B Consent Forms B.1 SMART and MindUP RCT Teacher Consent Form University of British Columbia Department of Pediatrics British Columbia Children's and Women’s Health Center UBC Faculty of Education 2125 Main Mall Vancouver, BC, Canada V6T 1Z4 Tel: 604.822.3131 Fax: 604.822.2684 Tel: 604-822-0000 Fax: 604-822-0000 educ.ubc.ca Teacher Consent Form "Effectiveness of Mindfulness Education Programs on Children's Social-Emotional Competence, Psychological Well-Being Cognitive Control, and Stress Reactivity” Principal Investigator: Dr. Kimberly Schonert-Reichl, Associate Professor, Department of Education, UBC Co-Investigators: Dr. Tim Oberlander, Developmental Paediatrician, Department of Paediatrics, UBC Dr. Adele Diamond, Professor, Developmental Cognitive Neuroscience, UBC Ms. Eva Oberle, Ph.D. candidate, Department of Education, UB Contact Person: Jenna Whitehead, MA Student Email: jenna.whitehead@alumni.ubc.ca Telephone: (604) 603-3296 This study is being organized by educators at the Coquitlam School Board and Dr. Kimberly A. Schonert-Reichl (Faculty of Education, University of British Columbia), Dr. Tim Oberlander (Department of Pediatrics, Children’s and Women’s Health Centre), Dr. Adele Diamond (Developmental Cognitive Neuroscience, UBC), and Ms. Eva Oberle (Faculty of Education, University of British Columbia). It is hoped that the results of this study will help parents and educators better understand children’s emotional and social development and therefore be better equipped to improve education for all. Listed below are several aspects of this project that you need to know. Purpose: The purpose of this study is to evaluate the effectiveness of the “MindUp” program – an educational program for children, designed to promote children’s psychological social responsibility, well-being, and academic success, and the SMART (Stress Management and Relaxation Techniques) program – an intervention program for teachers, designed to improve teachers’ ability to deal with stress and enhance their well-being and satisfaction as a teacher. 124 The MindUp program consists of teaching a series of simple techniques designed to enhance self awareness, focused attention, problem solving abilities, stress reduction, conflict resolution, and prosocial behaviours in children (such as, sharing, helping, and cooperating). MindUp is being taught in several schools throughout the Vancouver School District as part of the District’s goal to promote students’ social and emotional learning and social responsibility. Some of the children who participate in the research study will receive the MindUp program in their classroom while other children in the study will not receive the program (comparison group). Teachers who do not receive training in the MindUp program initially, and whose classroom is thus part of the comparison group will receive all MindUp materials after the study has been completed and may implement the program then if they wish. The SMART-program is a program for teachers, and it consists of a series of afternoon and weekend workshops. You may or may not (comparison group) participate in SMART throughout the duration of this research project. Those teachers who are part of the comparison group and do not take part in SMART throughout the research study will be given a CD with guided mindfulness practices by John Kabat-Zinn after the study has been completed. After the study has been completed, teacher will also receive a gift certificate as a small honorarium for participating in the study, and we will organize a pizza party in your classroom, providing pizza for all children (including those who do not receive consent for participation). Note that if you are assigned to a condition involving SMART and/or MindUp, we will carry any cost for you for participating in those programs and the material involved. Please not that this study is conducted as a randomized control trial, which means that teachers will be randomly assigned to either receiving MindUp only, SMART and MindUp, SMART only, or no intervention for the duration of this research project. Assigning teachers randomly to one of those conditions, and not letting them choose which condition they would like to be part of is important to obtain valid data from this study that allows us to answer our research questions. Procedure: If you agree to participate, we will work closely with you to schedule study sessions during your class’s regular school day. We would first schedule a time with you to come in and hand out parental permission slips to students in your class. We would then visit your classroom for nine separate sessions: three sessions will take place in October 2011 on three consecutive days in a week, three sessions will take place in January 2012, and three sessions will take place in the end of the school year. At all three times (October, January, end of the school year) the three classroom visits will involve the same data collection done by graduate research assistants as described in the following section: 1. On the first visit (ca. 50-60 minutes) we will ask participating students to fill out some questionnaires that ask about their background, feelings about themselves, their peers, and school (these are described in more detail below). Participating students will complete one set of questionnaires in the next couple of weeks and another set of questionnaires at near the end of the school year. Research assistants will be there to explain the directions and make sure students understand the instructions; each question will be read out loud by a researcher assistant and a second research assistant will be in the class to help with any specific question a student may have during the questionnaire. The first questionnaire asks about background, such as age, gender, family composition, and language spoken at home. Another set of questionnaires asks about students’ feelings about themselves, their classroom, and their positive social behaviours. The third set of 125 questionnaires asks students to provide ratings of their classmates’ positive classroom behaviours, and the last questionnaire asks for information on their feelings about school. Both prior to and upon completion of the questionnaire portion of the study, it is explained to participants that their answers are only their own and should be kept private from others. Further, the purpose of the peer nomination task is explained: the peer nomination task gives the researchers the students' opinion of the class composition to help us learn more about classroom dynamics. Children who do not participate in this research will be given an activity to do that is related to their regular classroom instruction. Please note: Students who are not participating in the research study will have their names removed from the peer nomination task in the questionnaire. 2. On day 2, we want to learn about the daily pattern of substances found in children’s saliva. To learn about this, we will ask participating children to give us a saliva (spit) samples 3 times during one day (this takes ca. 5 minutes each time): when students first come to school, before lunch, and right before dismissal Note: All saliva samples will be destroyed after we have done our testing. 3. On day 3, we want to learn about the development of children’s self-control, rule learning, and memory and see how these “cognitive control” behaviours: 1) might change as a result of participation in the ME program, and 2) are associated with children’s psychological well-being and academic success. To learn about this, we will be giving children games to play and problems to solve on the computer (ca. 15 minutes per child). Specifically, students will be asked to respond to pictures using various rules that we will explain to them. During the course of the game, the rules might change. Before each game we will explain the rules and go over them, giving students an opportunity to practice. In games where the rules change, we will explain that and explain what they will change to. We will do our very best to make sure that students understand how to play a game before we start. We never rush or criticize anyone, and try to keep each student engaged so that he or she performs well. Most children enjoy the individual attention. The computer session will be done individually with each participating student and takes about 15 minutes to complete. In addition, we will ask teachers to complete two to three different measures as part of this study (depending on the condition you will be assigned to in this study): a) If you are implementing the MindUp program, we ask you to complete a weekly implementation diary in which you note what activities you completed in a given week, and to what extend you implemented the activities in the curriculum. Completing the diary will take no longer than 2-5 minutes per week. b) In addition, ALL teachers will be asked to complete a brief questionnaire at the pretest (fall 2011), mid-point (January 2012), and posttest (June 2012). This survey includes questions about teachers’ demographic background, mindful attention, and beliefs about social and emotional learning. Completing this survey will take approximately 20 minutes at each time point. c) Furthermore, ALL teachers will be asked to complete a brief survey for each child, rating children’s behaviors in the classroom and student-teacher relationship. This survey will take approximately 10 minutes per child. What will teachers be asked to do by this study? • Collect permission slips from the children 126 • Provide class lists to Ms. Jenna Whitehead • Co-ordinate with Ms. Jenna Whitehead times that are convenient to distribute the permission slips • Co-ordinate with Ms. Jenna Whitehead times for study session appointments • Complete a brief survey assessing various dimensions of each child’s social behaviours in the classroom. You will be asked to complete this checklist twice – once during the next month and again in May. Each checklist will take approximately 5-10 minutes to complete per child. • Indicate if/when would be appropriate to have a pizza lunch • Complete the MindUp implementation diary (weekly ca. 2 minutes) if you are implementing MindUp • Complete a survey about experiences during SMART if you are participating in the SMART program • Written consent to participate in this study will be requested from participating teachers. Risks: For the questionnaire portion of this study, it is important for you to know that it is not a test and there are no right or wrong answers – we are not in any sense “testing” the children. We are only interested in finding out children’s opinions and feelings. It is hoped that the results of this study will help teachers and parents better understand the way that students think and improve education for all. For the portion of this research in which we collect students’ saliva (spit), you should know that helping with this project will not hurt your students or make them sick. The dental rolls used to collect saliva will taste like paper. There are no known risks or side effects of the cortisol collection to children’s development. There are no known risks for completing the teacher surveys. Teachers’ participation in this project is voluntary. At any given time, teachers can decide to withdraw from participating in the study. Confidentiality: Any information resulting from this research study will be kept strictly confidential. All documents will be identified only by code number and kept in a secured information system and locked filing cabinet. The identity of the participants in this study (both teachers and students) will be entirely confidential. No information that discloses your or your students’ identities will be released or published without specific consent to the disclosure. Neither you nor your students will be identified by name in any reports of the completed study. Copies of the relevant data, which identify the participants only by code number, may be published in scientific journals, but no participant will be identified by name. However, research records identifying participants may be inspected in the presence of the Investigator or his or her designate by representatives of the UBC Research Ethics Board for the purpose of monitoring the research. Who can I talk to if I have any questions? If you have any questions at any time during this project, you may contact Dr. Kimberly Schonert-Reichl: Kimberly.schonert-reichl@ubc.ca, (604) 822-2215 or Ms. Jenna Whitehead, (604) 604-3296. Furthermore, you may contact the Research Subject Information Line in the UBC Office of Research Services at 604-822-8598 or, for long distance, e-mail to RSIL@ors.ubc.ca or call toll free under 1-877-822-8598. We would appreciate it if you could indicate on the slip provided on the attached page whether or not you would like to participate. Would you kindly sign and date the attached slip where indicated? Thank you very much for considering this request. 127 Sincerely, Kimberly Schonert-Reichl Principal Investigator Associate Professor Department of Educational and Counselling Psychology, and Special Education, UBC Email: Kimberly.schonert-reichl@ubc.ca Phone: 604 822 3420 Co-Investigators Dr. Tim Oberlander, Professor, Department of Paediatrics, UBC Dr. Adele Diamond, Professor, Developmental Cognitive Neuroscience, UBC Ms. Eva Oberle, Ph.D. student, Department of Educational and Counselling and Special Education, UBC TEACHER CONSENT FORM Study Title: "Effectiveness of Mindfulness Education Programs on Children's Social-Emotional Competence, Psychological Well-Being, Cognitive Control, and Stress Reactivity” Principal Investigator: Dr. Kimberly Schonert-Reichl, Associate Professor, Department of Educational and Counselling Psychology and Special Education University of British Columbia, Vancouver, B.C. KEEP THIS PORTION FOR YOUR RECORDS I understand that my participation in the above study is entirely voluntary, and that I or students in my class may refuse to participate, or I or my students are free to withdraw from the study at any time without any consequences. I have received a copy of this consent form for my own records. I consent to my participation in this study and in signing this document. I have read and understand the attached letter regarding the study entitled "Effectiveness of Mindfulness Education Programs on Children's Social-Emotional Competence, Psychological Well-Being, Cognitive Control, and Stress Reactivity” I have also kept copies of both the letter describing the study and this permission slip. _______ Yes, I agree to participating in this project. _______ No, I do not agree to participate. Teacher’s Signature Printed Name Date School Name DETACH AND RETURN TO PROJECT COORDINATOR I understand that my participation in the above study is entirely voluntary, and that I or students in my class may refuse to participate, or I or my students are free to withdraw from the study at any 128 time without any consequences. I have received a copy of this consent form for my own records. I consent to my participation in this study and in signing this document. I have read and understand the attached letter regarding the study "Effectiveness of Mindfulness Education Programs on Children's Social-Emotional Competence, Psychological Well-Being, Cognitive Control, and Stress Reactivity” I have also kept copies of both the letter describing the study and this permission slip. _______ Yes, I agree to participating in this project. _______ No, I do not agree to participate. Teacher’s Signature Printed Name Date School Name 129 B.2 Kindness in the Classroom RCT Teacher Consent Form November, 2014 Dear ____, We are writing to invite you to participate in a research study that we are conducting evaluating the effectiveness of a social and emotional learning program designed to promote kindness and well-being in elementary school students. Our study, titled “Evaluating the Effectiveness of the Random Acts of Kindness Curriculum in Elementary School Children: A Randomized Controlled Trial,” will take place in elementary schools in the Vancouver School District from approximately November, 2014 to June, 2015. Who is conducting this study? This study is being organized by educators in the Vancouver School District and Dr. Kimberly A. Schonert-Reichl in the Faculty of Education at the University of British Columbia. This project is also linked to the Vancouver School District’s focus on promoting students’ social and emotional learning and social responsibility in school. We hope that the results of this study will help parents and educators better understand how to promote students’ emotional and social development in relation to school success and, therefore, be better equipped to improve education for all. What is the purpose of this study? The goal of this study is to evaluate the effectiveness of a social and emotional learning (SEL) promotion program for children that focuses on promoting kindness and well-being: the Random Acts of Kindness Curriculum. The Random Acts of Kindness (RAK) Curriculum focuses on promoting kindness, resiliency, and well-being in children during the elementary school years. The program provides opportunities for children to enhance their prosocial behaviours (for example, sharing, helping, cooperating) and well-being by engaging in activities identified to promote knowledge and skills associated with kindness and compassion. More specifically, the RAK lessons provide opportunities for children to enhance their social and emotional learning by engaging in activities identified to promote caring for self and others, including emotional literacy and emotion understanding, empathy, perspective-taking, and kindness. Research has demonstrated that such skills and knowledge lead to increases in children’s positive social behaviours and school success. How will the study work? In order to understand whether the program is effective or not, we are conducting a scientific randomized controlled trial. Teachers who choose to participate will be randomly chosen (with a coin flip) to either teach the program during the study, or be a comparison group in which they continue with their regular classroom routine. A randomized controlled trial allows us to understand whether any changes are a result of the program itself, and not other variables, for example teaching style or class composition. Faculty of Education Vancouver Campus Educational and Counselling Psychology, and Special Education 2125 East Mall 5th Floor Scarfe Office Blk Vancouver, B.C. Canada, V6T 1Z4 Tel: 604-822-0242 Fax: 604-822-3302 130 We understand that teachers who choose to participate in our research study do so because they are interested in implementing the Random Acts of Kindness curriculum. With this in mind, we have made arrangements so that every teacher who participates in the research has the opportunity to get the RAK training and materials regardless of whether they are randomly chosen for the Random Acts of Kindness (RAK) implementation group or the comparison classroom group. The teachers in the RAK implementation group will receive a ½ day of training in the RAK curriculum in early January, 2015 and will start the lessons with their class immediately following the training. These teachers will also receive a ½ day booster session midway through the program implementation (in about March). The teachers in the comparison classrooms will not start any lessons in January; they will continue with their regular classroom routine. The teachers in the comparison classrooms will have the opportunity to receive the RAK training after the study has ended (the RAK training will be made available to teachers in September, 2015). What will you be asked to do if you choose to participate in the study? • Provide your written consent to participate in this study. • Complete a survey about your background and teaching experience (this will take approximately10-15 minutes to complete). • Complete a survey in which you rate participating students’ behaviours. This survey will ask you to rate their classroom and social behaviours and should take approximately 5-10 minutes per student. You will be asked to complete these surveys three times during the school year (November/December, March, May/June). Note that you will receive a ½ day EOC for each of these 3 time periods. • Host research assistants in your classroom during the three data collection periods (November/December, March, and June) for approximately 50 minutes each time. These research assistants will administer surveys to the children as a group during regular school hours. • Complete a weekly “implementation calendar” throughout the duration of the study. This is a brief survey in which you will be asked to answer questions and record lessons related to any social and emotional learning activities that you implement during the duration of the study. If you are randomly chosen to implement the Random Acts of Kindness Curriculum, in addition to the above-mentioned tasks, you will be asked to: • Implement the Random Acts of Kindness Lessons from January until May. Note that each lesson is approximately 20 – 45 minutes in length. • Attend a half-day training on the Random Acts of Kindness Curriculum in early January. • Attend a half-day booster session in the spring (note that EOC costs will be covered for each of these sessions). What are the benefits of participating in this study? Teachers and students will help contribute to the growing knowledge about the development of social-emotional well-being in children and adolescents. Your students will also have the opportunity to learn about scientific research. Additionally, all teachers will receive the Random Acts of Kindness Curriculum materials and program training free of charge. As previously mentioned, teachers randomly chosen to be in the Comparison Group will receive these at the end of the study. Will you be compensated for participating in the study? 131 As a token of appreciation, we will be providing you with a $50 gift card for participating. You will also be provided with a half-day release with an EOC during each of the three data collection periods to compensate for the time it takes you to fill out the surveys. Additionally, you will receive a half-day release for both the training session and booster session. All classrooms will receive a pizza party at the end of the study as a token of our gratitude to you and your students for participating. Compensation is not dependent on completion of the project, but may be pro-rated should you choose to withdraw before completion of the study. Are there any risks if you participate in this study? There are no known risks to teachers for participating in this study. Your participation in this project is voluntary. At any given time, you can decide to withdraw from participating in the study, even after signing this consent form. Refusing to participate or withdrawing from the study will not jeopardize your job or professional standing in any way. How will your privacy be maintained? Any personal information resulting from this research study will be kept strictly confidential. No school, teacher, or students will be identified by name in any reports of the completed study. All documents will be identified by code number only, and kept in a secured information system and locked filing cabinet in Kimberly Schonert-Reichl’s research lab at UBC. Only the researchers and research assistants will know the identity of the participants in this study (both teachers and students). No information that discloses your or your students’ identities will be released or published without specific consent. Research records identifying participants may be inspected in the presence of the Investigator or her designate by representatives of the UBC Research Ethics Board for the purpose of monitoring the research. How will results be disseminated? Findings from the study, in which the participants are identified by code number only, may be published in reports, scientific journals, and/or included in presentations. A summary of the overall study results will be made available to participants upon request once analysis is completed. Who can you contact if you have questions about the study? If you have any questions at any time during this project, you may contact Dr. Kimberly Schonert-Reichl: kimberly.schonert-reichl@ubc.ca, (604) 822-2215 or Ms. Jenna Whitehead: jenna.whitehead@alumni.ubc.ca, (604) 604-3296. Who can you contact if you have any concerns about the study? If you have any concerns about your rights as a research participant and/or your experiences while participating in this study, you may contact the Research Subject Information Line in the UBC Office of Research Services at 604-822-8598 or if long distance e-mail RSIL@ors.ubc.ca or call toll free 1-877-822-8598. Sincerely, Kimberly A. Schonert-Reichl, Ph.D. Principal Investigator Professor, Department of Educational and Counselling Psychology, and Special Education, UBC 132 TEACHER CONSENT FORM Study Title: Evaluating the Effectiveness of the Random Acts of Kindness Curriculum in Elementary School Children: A Randomized Controlled Trial Researcher: Kimberly A. Schonert-Reichl, Ph.D. Professor, Department of Educational and Counselling Psychology and Special Education, 2125 Main Mall, University of British Columbia, Vancouver, B.C. V6T 1Z4 Phone: 604.822.2215; e-mail: kimberly.schonert-reichl@ubc.ca Contact: Jenna Whitehead PhD Student, Department of Educational and Counselling Psychology and Special Education, 2125 Main Mall, University of British Columbia, Vancouver, B.C., V6T 1Z4 Phone: 604.603.3296; e-mail: jennaw86@gmail.com Taking part in this study is entirely up to you. You have the right to refuse to participate in this study. If you decide to take part, you may choose to withdraw from the study at any time without giving a reason and without any negative impact. Your signature below indicates that you have received a copy of this consent form for your own records. Your signature indicates that you consent to participate in this study. ____________________________________________________ Participant Signature Date ____________________________________________________ Participant Printed Name 133 TEACHER CONSENT FORM Please return this form to the researchers. Study Title: Evaluating the Effectiveness of the Random Acts of Kindness Curriculum in Elementary School Children: A Randomized Controlled Trial Researcher: Kimberly A. Schonert-Reichl, Ph.D. Professor, Department of Educational and Counselling Psychology and Special Education, 2125 Main Mall, University of British Columbia, Vancouver, B.C. V6T 1Z4 Phone: 604.822.2215; e-mail: kimberly.schonert-reichl@ubc.ca Contact: Jenna Whitehead PhD Student, Department of Educational and Counselling Psychology and Special Education, 2125 Main Mall, University of British Columbia, Vancouver, B.C., V6T 1Z4 Phone: 604.603.3296; e-mail: jennaw86@gmail.com Taking part in this study is entirely up to you. You have the right to refuse to participate in this study. If you decide to take part, you may choose to withdraw from the study at any time without giving a reason and without any negative impact. Your signature below indicates that you have received a copy of this consent form for your own records. Your signature indicates that you consent to participate in this study. ____________________________________________________ Participant Signature Date ____________________________________________________ Participant Printed Name 134 B.3 Taxi Dog RCT Teacher Consent Form 135 136 137 138 139 140 B.4 SEL and Biology Feasibility Study Teacher Consent Form 141 142 143 144 145 146 147 148 149 150 151 152 153 154 Appendix C Measures C.1 Teacher Demographic Questionnaires Study 1 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT 2. Kindness in the Classroom RCT 155 3. Taxi Dog RCT 4. SEL and Biology Feasibility Study Study 2 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT See above. 2. Kindness in the Classroom RCT See above. 3. SEL and Biology Feasibility Study See above. 156 C.2 Teacher SEL Experience Surveys Study 1 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT 157 158 2. Kindness in the Classroom RCT 159 3. Taxi Dog RCT 160 161 4. SEL and Biology Feasibility Study 162 163 Study 2 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT See above. 2. Kindness in the Classroom RCT See above. 3. SEL and Biology Feasibility Study See above. 164 C.3 Job Satisfaction Items Study 1 Two items from modified Teacher Stress Inventory (TSI) (Fimian, 1984) 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT 2. Kindness in the Classroom RCT See above. 3. Taxi Dog RCT See above. 4. SEL and Biology Feasibility Study See above. Study 2 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT See above. 2. Kindness in the Classroom RCT See above. 3. SEL and Biology Feasibility Study See above. 165 C.4 Teacher SEL Beliefs Scale Brackett et al. (2012) 166 C.5 Maslach Burnout Inventory – Educator’s Survey (Maslach et al., 2001) Study 2 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT 167 2. Kindness in the Classroom RCT See above. 3. SEL and Biology Feasibility Study See above. 168 C.6 Personal Teaching Efficacy Subscale (Midgley et al., 2000) Study 2 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT 169 2. Kindness in the Classroom RCT 3. SEL and Biology Feasibility Study 170 C.7 Implementation Diaries Study 1 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT 171 2. Kindness in the Classroom RCT 172 173 3. Taxi Dog RCT 174 4. SEL and Biology Feasibility Study 175 Study 2 1. SMART (Stress Management and Relaxation Techniques) in Education and MindUP RCT See above. 2. Kindness in the Classroom RCT See above. 3. SEL and Biology Feasibility Study. See above. 176 Appendix D Summaries of Extension Activity Implementation D.1 SMART and MindUP Study: Lesson Extension Implementation Summary Lesson Number Mean Extension Activities Delivered (%) Range (%) 1 63 0-100 2 44 0-100 3 44 0-100 4 50 0-100 5 25 0-50 6 29 0-50 7 31 0-100 8 19 0-100 9 6 0-50 10 29 0-50 11 33 0-100 12 8 0-50 13 20 0-50 14 33 0-50 15 33 0-50 Note. N = 8 177 D.2 Kindness in the Classroom Study: Lesson Extension Implementation Summary Lesson Number Mean Extension Activities Delivered (%) Range (%) 1 5.3 0-25 2 3.6 0-25 3 3.9 0-22 4 7.6 0-33 5 4.9 0-25 6 3.7 0-13 7 3.3 0-22 8 2.8 0-11 9 7.2 0-50 10 6 0-13 11 4.7 0-33 12 3.9 0-11 13 4.6 0-25 14 2.9 0-25 15 5.1 0-22 16 3.4 0-13 Note. N = 14 178 D.3 Taxi Dog RCT: Lesson Extension Implementation Summary Lesson Number Mean Extension Activities Delivered (%) Range (%) 1 25 0-50 2 39 0-100 3 12.5 0-50 4 7.7 0-100 5 25 0-50 6 25 0-50 7 37.5 0-100 8 33 0-100 9 0 N/A Note. N = 9 179 D.4 SEL and Biology Feasibility Study: Lesson Extension Implementation Summary Lesson Number Mean Extension Activities Delivered (%) Range (%) 1 50 0-100 2 50 0-100 3 0 N/A 4 0 N/A 5 0 N/A 6 33.5 0-67 7 50 0-100 8 0 N/A 9 0 N/A Note. N = 2
Thesis/Dissertation
eng
Human Development, Learning, and Culture
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Schonert-Reichl, Kimberly
The relationship between teacher factors and social and emotional learning program implementation
Text