Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Investigating the impact of social-emotional interventions : an evaluation of the "Welcome To My Life"… Hearle, Nikki Jade 2008

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2008_fall_hearle_nikki_jade.pdf [ 6.26MB ]
Metadata
JSON: 24-1.0054537.json
JSON-LD: 24-1.0054537-ld.json
RDF/XML (Pretty): 24-1.0054537-rdf.xml
RDF/JSON: 24-1.0054537-rdf.json
Turtle: 24-1.0054537-turtle.txt
N-Triples: 24-1.0054537-rdf-ntriples.txt
Original Record: 24-1.0054537-source.json
Full Text
24-1.0054537-fulltext.txt
Citation
24-1.0054537.ris

Full Text

INVESTIGATING THE IMPACT OF SOCIAL-EMOTIONAL INTERVENTIONS: AN EVALUATION OF THE "WELCOME TO MY LIFE" PROGRAM  by  Nikki Jade Hearle Bachelor of Arts, University of British Columbia, Canada, 2003  A THESIS SUBMITTED IN P A R T I A L F U L F I L L M E N T OF THE REQUIRMENTS FOR THE D E G R E E OF  M A S T E R OF ARTS  in  The Faculty of Graduate Studies (School Psychology)  THE UNIVERSITY OF BRITISH C O L U M B I A (Vancouver) July 2008  © Nikki Jade Hearle, 2008  Abstract The transition from elementary school to high school is challenging for many students. High school introduces a more complex social framework, larger student populations, and often higher risk activities. This transition occurs during a developmental period when self-esteem is the lowest for girls, which can make them susceptible to a variety of risk factors (e.g., tobacco, alcohol, and drug use, unsafe sexual practices).  The Young Women Christian Association  ( Y W C A ) , in collaboration with the Vancouver School Board (VSB), developed a 6-8 week, after-school program called Welcome To M y Life (WTML) to prepare Grade 7 girls for this transition. The goal of this thesis was to work in partnership with program developers to conduct an evaluation of the W T M L program, informed by research on program evaluation within the area of social and emotional learning. To this end, 42 Grade 7 girls at two program sites, 12 who participated in the W T M L program and 30 who did not, completed self-report surveys tapping self-concept/self-esteem (Self Description Questionnaire-II) as well as perceived developmental assets (Developmental Assets Profile) both before and after the program. Results of analyses comparing the self-reports of students who did participate in the W T M L program with those who did not indicated no significant program effects. Changes in reported self-esteem/self-concept and perceived positive assets were in the expected direction, but were not significant. The general failure to demonstrate treatment effects may be attributed to a number of limitations of the current evaluation project, including the small sample size and the limited time frame considered. Nevertheless, evaluations of participant satisfaction, based on surveys obtained from 8 program facilitators and 22 participants indicated that program facilitators felt appropriately trained and that the program was positively received by students.  As well, evaluations of  implementation integrity, based on surveys completed by 8 program facilitators indicated that the  program was well-implemented. Thus, the W T M L program holds promise as a means of preparing Grade 7 girls for the transition to secondary school, and should be further evaluated with a larger and more representative sample.  Suggestions for further development of the  program and effective collaboration efforts are provided.  Table of Contents Abstract  ii  Table of Contents  iv  List of Tables  v  Acknowledgements  vi  C H A P T E R ONE  Introduction  1  CHAPTER TWO  Review of the Literature  3  CHAPTER THREE  Methodology  39  C H A P T E R FOUR  Results  50  C H A P T E R FIVE  Discussion  76  References  96  List of Tables Table 4.1  Statistics for SDQ-II Scores  51  Table 4.2  Statistics for DAS Scores  52  Table 4.3  Results of Statistical Analyses for SDQ-II  57  Table 4.4  Results of Statistical Analyses for DAS  60  Table 4.5  Time Spent Preparing For Modules  63  Table 4.6  Reported Rates of Activity Completion  66  Table 4.7  Reported Rates of Routine Activity Completion  67  Table 4.8  Reported Preparedness to Facilitate Modules  71  Table 4.9  Support for Module Activities  72  Table 4.10  Perceived Enjoyment of Activities  73  Acknowledgements I would like to express my sincere appreciation to my supervisor. Dr. Shelley Hymel, who I have known in many capacities over the past five years: as a boss, supervisor, and most importantly mentor and friend.  Shelley's guidance, support, and belief in me has been  instrumental in my working with high risk children, attending grad school, and finally beginning a career as a school psychologist. In regards to the current thesis project, she has provided me with encouragement and motivation during every phase, trial, and success. I am continuously awed by the breadth and depth of her knowledge of social-emotional learning, child and adolescent development, and program evaluation, as well as her ability to convey this knowledge to a wide audience of listeners. One only needs to have a single conversation with Shelley to appreciate the efficiency of her brain as she makes connections between ideas, theories, studies (from small to large spanning multiple decades), and experiences....and realize that his/her neurons will never fire so quickly! I would also like to thank Dr. Nand Kishor, who through class work and consultation on this thesis has transformed the study of statistics for me from a fear and anxiety-inducing monster to an exciting field which provides multiple tools for one's tool belt. His belief in my capabilities has made me feel empowered to persevere during many points during this project. I am thankful to Dr. K i m Schonert-Reichl whose optimism and belief that efforts in our field can make a meaningful difference are both inspiring and contagious. I would also like to extend my gratitude to Dr. Lynn Miller who always brought a sense of humour and needed lightness to the field of psychopathology and evaluation, which always served to decrease my stress and put things in proper perspective.  The Young Women's Christian Association (YWCA) and Vancouver School Board (VSB) made this project possible by providing access to their program, schools, and students. In particular. Lis Peterson (YWCA) embarked on the collaborative journey of evaluating the Welcome To M y Life program with me and I think that we both learned and grew from the experience. M y family always believed that I could do this.  I want to thank them for their  unwavering and unconditional support and confidence in me and for being proud of every feat, whether small or large in nature. And finally, I would like to thank my partner Trevor Brady who became my husband during my time as a grad student. Through the many late nights, grumpy days, and stress of grad school, Trevor was tested on how much he meant the vows of "for better or for worse" and he handled it all with grace and patience.  CHAPTER 1 Introduction Research has identified the transition to high school as one that is difficult for many students (e.g., Simmons & Blyth, 1987). Changes in school structure, academic course load, and puberty can be challenging for students if they do not have the time management, social skills, and resources necessary to adapt to the altered demands. The transitional period also comes at a time when students, particularly girls, face their lowest levels of self-esteem (e.g., Lameiras & Rodriguez, 2003; Robins, Trzesniewski, Tracy, Gosling, & Potter, 2002) and self-concept (e.g.. Marsh, 1989). These low feelings of self-worth, in addition to the stress of transitioning to a new, more complex school, can put students at risk for internalizing and externalizing symptoms (e.g., Egan & Perry, 1998; Hillegers, Burger, Wals, Reichart, Verhulst, Nolen, & Ormel,, 2004; Robinson, Garber, & Hilsan, 1995; Simmons et al., 1987). As peer relationships become a priority (e.g., Pratt & George, 2005), the social world simultaneously becomes more difficult to manage. Increases in relational aggression occur (e.g.. Hazier, 1996; Pellegrini & Long, 2002) and social networks temporarily decrease (e.g., Cantin & Boivin, 2004). Students who do not manage to adapt socially and do not make friends or connections to the school become at risk for emotional distress (e.g., Wentzel, Barr, & Caldwell, 2004) and dropping out of school (e.g., Coie, Dodge, & Kupersmidt, 1990; Parker & Asher, 1987). Given these challenges and stressors, is it imperative that schools provide student support through various prevention and intervention efforts (see Hymel, Schonert-Reichl & Miller, 2006). As part of such efforts, the Vancouver School Board, in partnership with the local Young Women's Christian Association ( Y W C A ) , entered into a unique school-community partnership in order to develop the "Welcome To M y Life" program to take preventative action to prepare  girls for the transition to high school. Although such school-based, preventative programs have great potential for success, programs of this nature must be exposed to sound research scrutiny. Moreover, given that programs can be costly in terms of both time and money, there is a social responsibility to use strategies and programs that are empirically validated. The purpose of the present study was to evaluate the Y W C A Welcome To M y Life program. In the chapter which follows, I review and summarize the current literature on socialemotional program evaluation and the developmental trends and social-emotional functioning for students during the transition from elementary school to high school.  CHAPTER 2 Literature Review The need for social-emotional programming in schools has been increasingly documented in the research literature (Greenberg, Weissberg, O'Brien, Zins, Fredericks, Resnik, & Elias, 2003; Hymel, Schonert-Reichl & Miller, 2006; Simmons & Blyth, 1987). However, equally important is the need for evaluation in order to determine the impact of such programs on the student populations they are meant to serve. Lideed, research evaluating the impact of schoolbased, universal programming aimed at reducing bullying and victimization has been met with mixed success (e.g.. Smith, Schneider, Smith & Ananiadou, 2004; Vreeman & Carroll, 2007). Given these mixed findings, it becomes the responsibility of schools to ensure that interventions and programs that are implemented are empirically supported and that new programs include evaluation in their development.  In recent years, this notion has been captured in a growing  effort to support "evidence-based" prevention and intervention efforts within education (Slavin, 2008). Although there is some common understanding  of the term,  "evidence-based"  programming has become a catch phrase of sorts among researchers, policy makers, and school personnel. Over the past two decades, prevention research has progressed from theory-driven prevention initiatives aimed at specific negative outcomes, to an evaluation of these programs through increasing scientific rigour, to stakeholders demanding that implemented programs have a foundation of research support behind them (Prevention Research Center for the Promotion of Human Development, 2005). The adoption of the No Child Left Behind Act in the US (U.S. Department of Education, 2002) and the heightened calls for accountability in practice have no doubt encouraged this trend further.  The term "evidence-based programs" loosely refers to programs that have been shown to derive positive outcomes through sound research methods. The terms efficacy and effectiveness are often used to describe evidence-based programs in the research literature (Prevention Research Center for the Promotion of Human Development, 2005). A program is said to be efficacious when under ideal, pure circumstances, as is often the case with "demonstration" or "model" programs, a program yields significant positive outcomes.  Evaluations of this sort  typically employ random assignment and a control group to ensure confidence in causal claims relating to the intervention. The program evaluation must also utilize psychometrically sound measures, yield replicable outcomes of statistical and practical significance, and there must be evidence of persisting benefits after a long-term follow-up evaluation (Society for Prevention Research, 2004). Effectiveness refers to benefits obtained from an evaluation of a program that is implemented in a "real world setting" or how the program would "typically" be implemented (Prevention Research Center for the Promotion of Human Development, 2005). To meet the criteria of effectiveness, in addition to meeting the standards of efficacy, the program must be implemented in a naturalistic setting, with typical facilitators (e.g., teachers, not researchers), a real world sample, and an investigation into potentially differential effects for different demographic groups (Society for Prevention Research, 2004). Certainly both of these types of evidence are important, as are the different types of experiments that can yield them, specifically various experimental (i.e., randomized control trials), quasi experimental, and non experimental (e.g., post treatment evaluation, quahtative evaluation) designs.  Borrowing from medical research, the traditional "golden standard" for  program evaluation still rests with the randomized controlled trial (RCT) design. In the typical  R C T design, subjects are randomly assigned to either treatment (experimental) or non-treatment (control) groups, allowing for an empirical test of the program's causal effect within a particular sample or population. In particular, random assignment aims to control for selection bias and extraneous variables which may impact participants' responses to an intervention.  Due to  conditions (e.g., individual differences, number and type of other interventions participants exposed to) being heavily controlled, the R C T design is conducive to internal validity and the ability to make causal statements regarding the direct impact of an intervention or phenomenon on a given sample or population. In other words, there is some certainty that what was done (e.g., intervention, program) caused what was observed to be the outcome (e.g., increase in selfesteem; decrease in aggressive behaviour), hiternal validity, the experimental method, and the determination of "cause and effect", has been often prioritized to the point where this design is considered the only true means of determining "evidence-based practice".  This has resulted in  other viable methods of research being overlooked and viewed as less scientifically rigorous despite their unique merits and contributions to understanding the complex effects of an intervention (McCall & Green, 2004). Specific strengths and limitations of various research designs are discussed in further detail in later paragraphs. In quasi experimental designs, group assignment is not random and potential pre-test differences between individuals are addressed statistically (Prevention Research Center For The Promotion Of Human Development, 2005). For example, subjects may decide whether or not to participate in a program or they may select which program to participate in among a number of options.  Differences between the groups are assessed and investigated on pre-intervention  measures to determine whether initial differences are present that could potentially contribute to the impact the intervention has on the participants.  Designs of this nature may also match  subjects based on a common criteria (e.g., classroom) from the participant and comparison group. There are inherent strengths and limitations to both R C T and quasi experimental designs which are discussed below.  The strong internal validity that RCTs yield is certainly an  undisputed strength for evaluation research.  However, a narrow and sole focus on internal  validity may come at the expense of other important features of intervention evaluation. McCall and Green (2004) discuss the risk of relying too heavily on randomized controlled experiments and the folly of assuming that RCTs alone constitute the measure of "evidence-based" programs. A main concern involves the hallmark feature of the experimental design, random assignment, and its potential inadequacy for evaluating how a program will be implemented in the real world. Although RCTs allow statements of causality to be made with more confidence, they argue that in the real world, resources are not randomly allocated. Resources are assigned to people who need them most and participants choose to seek treatment and the type of treatment they wish to receive. Furthermore, participant choice or input into treatment options likely has an important influence on the outcomes of a treatment, which is consistent with the literature on treatment acceptability and outcome expectancy (AUinder & Gates, 1997; Elliot, Witt, & Kratochwill, 1991; Sugai & Tindal, 1993; Telzrow & Beebe, 2002; Waas & Anderson, 1991). Participants are more likely to adhere to and gain positive benefits from treatments that they view as relevant, acceptable, fair, reasonable, and having a high likelihood of success. These criteria would more likely be met when a participant chooses an intervention, rather than receives an assignment to it. McCall and Green (2004) also discuss the methods and criteria frequently used to select participants in RCTs to control for extraneous variables which may confound treatment outcomes or decrease between group differences. This includes the restriction of the eligibility  criteria for participation in a study which may be dissimilar to what the program eligibility would be when the program is distributed. For example, if a researcher is investigating the impact of a classwide social-emotional program on students, they may not want to include classrooms that are simultaneously administering other social-emotional programs because it would be difficult to isolate the effect of any individual program. Although understanding the direct impact of an intervention is important, the effort to decrease erroneous factors may create an artificially homogeneous sample which is not reflective of the population that the intervention would be disseminated to in the "real world" or how it would be administered (i.e., amidst other interventions). This purposeful uniqueness of the sample decreases the generalizability (external validity) of the outcomes. Although the effects of the intervention would be well understood for the sample in the individual experiment, the impact it would have when more widely distributed would be less clear. As such, McCall and Green recommend research designs that allow for more within-treatment group analyses, as opposed to between group, treatment vs. control comparisons, to understand for whom the program works and with what "dosage". Still, many researchers argue that quasi-experimental designs can not sufficiently yield the accuracy and quality of data as experimental designs, and should be viewed as hierarchically below it (Cook, 2004; Cottingham, 2004; Mook, 1983). The internal validity that "randomized control trial" or experimental designs offer and the external validity or generalizability that non experimental and quasi-experimental designs permit underscore the need for a multi-pronged approach to understanding the effects of a program, with no single research design or approach yielding a full evaluation of the impact of a program under different conditions or contexts.  Multiple research designs are needed to expose the  complex relationship between a program and its outcomes (Cook, 2004).  Brooks-Gunn (2004)  suggests that experimental and non-experimental methods be integrated to answer more questions regarding the impact of an intervention or prevention program.  For example,  randomization could be used to answer questions of causality and to control for levels of implementation. Conversely, non-experimental methods could be used to determine the range of practical significance across implementations and to answer questions in populations not conducive to random assignment (e.g., teen pregnancy). Biglan, Mrazek, Camine, and Flay (2003) have proposed a set of criteria for grading the quality of evaluation information collected for a particular program. Seven "grades" are outlined with "1" referring to the strongest evaluative support and "7" referring to the scientifically weakest. For a grade of "7" a program has qualitative support, including endorsement by experts in the relevant field, and/or anecdotal support. Grade "6" refers to pre- and post-intervention data from only program participants during a single administration of an intervention. Grade "5" incorporates evidence from program and comparison participants who were not randomized to groups. To obtain a grade of "4", a program must have yielded significant results from an R C T or an interrupted times-series design replicated across three cases. The criterion for grade "3" is evidence of support obtained by a single research group from multiple RCTs or interrupted timeseries experiments. Grade "2" evidence meets the grade "3" criteria, with the addition that the evidence has been obtained by two or more independent research teams. Grade "1" incorporates the evidence of efficacy of grade "2" with evidence of effecfiveness. The intervenfion must have been administered by the intended facilitators to the intended population and evaluated in its intended context with investigation into implementation. This proposed grading system conceptualizes intervention evaluation as a process, one that is ongoing and multifaceted. In that regard, evidence cannot and should not be based on a  single study or type of resecirch design. Studies that answer different questions through different methods contribute to the overall understanding of a program's effects. In the initial stages of program development, it is not often feasible to direct substantial resources towards large scale and scientifically rigorous evaluation. The program, no matter how well conceived, would not likely be ready for that level of scrutiny. However, as a program shows promise of effectiveness through less intensive measures and becomes stronger as a result of repeated administrations and investigation into its implementation, it is more appropriate to increase the level of evaluation. In an ideal world, the evaluation of a program would follow this model. A program would be evaluated in ideal and controlled conditions so that the effects would be clear and well understood. Increasingly, it would be disseminated to broader population and evaluated in those contexts to gain a greater understanding of the complexity of the effects. Often when conducting evaluations, researchers must compromise between these ideals and what is feasible and available in the interest of evaluation. As such, the evaluation of a program is not always able to follow a prescribed set of sequential steps; however, each study, even if not ideal, contributes to a greater understanding of the intervention's effectiveness. Cost Effectiveness From a prevention and cost effectiveness standpoint, adoption of empirically-validated programs and continual evaluation of promising programs is paramount.  When funds are  limited, it is important that money is spent judiciously and on programs that have a high likelihood of success.  Program selection should be based on evidence of effectiveness and  efficacy, as well as consistency with the presenting problem. Often times, there is a discrepancy between program usage and what has been shown to be effective in the research literature. Drug and alcohol prevention programs serve as an illustrative example.  The astounding use of  tobacco, alcohol, and drugs among adolescents led to a push for school-based initiatives to address the problem. The Drug Abuse Resistance Education (DARE) is currently the most popular school-based drug use prevention program (Dusenbury, Falco, & Lake, 1997; Kanof, 2003). According to the D A R E website (http://www.dare.com/home/about dare.asp), D A R E is administered in 75% of US school districts and over 36 million children and adolescents worldwide receive the D A R E curriculum annually. The high cost to public and private funds and high usage (Thombs, 2002) of the program certainly suggests the need, from an economic and ethical standpoint, for it to have a solid research foundation verifying its effectiveness. Despite its popularity among police officers, who typically facilitate it, school personnel, parents and others who offer anecdotal evidence of its effectiveness (Berg, 1997; Curtis, 1999; Donnermeyer & Wurschmidt, 1997), objective research has not verified D A R E ' s successful outcomes. Compared with students who did not participate in D A R E , no significant differences have been found in terms of reduction or prevention of tobacco, alcohol, and substance use in the short-term (Clayton, Cattarello, & Johnstone, 1996; Ennett, Tobler, Ringwalt, & Flewelling, 1994; Kanof, 2003) or long-term (Clayton et al; Dukes, Stein, & Ullman, 1997; Kanof; Lynam et al., 1999; Thombs, 2002; West & O'Neal, 2004). However, in a six-year follow-up. Dukes, Stein, and Ullman found significantly lower levels of serious drug use (e.g., inhalants, cocaine) for adolescents who had participated in D A R E , compared to adolescents who had not. Conversely, one study found that D A R E was related to an increase in experimentation with hallucinogens (Wysong, Aniskiewicz, & Wright, 1994). It is not just the unjustified use of D A R E that is the problem with substance prevention programs in schools. Of the 47 tobacco, alcohol, and substance use prevention curricula being administered in elementary and secondary school classrooms, only 21% have undergone  sufficient research scrutiny to warrant their use (Dusenbury, Falco, & Lake, 1997). Among the ten well-researched programs, few were found to yield long lasting results. In addition, the most promising prevention programs are largely underutilized in favour of less effective ones (Dusenbury & Falco, 1995). While students may not be worse off from participating in D A R E or another ineffective substance prevention program, given the amount of money and resources that are spent on them, there is a social responsibility to ensure that they are working. Given the lack of empirical evidence supporting them, that money could be redirected towards the evaluation and dissemination of more promising programs. It is certainly important to choose programs that have a high likelihood of success, based on previous evidence of efficacy and effectiveness.  However, this does not ensure that a  program will be successful in every place and every time and with every sample in which it is implemented. A n examination of the effectiveness of the Olweus anti-buUying program, and other school-wide anti-buUying programs with similar core components, implemented across different countries yielded inconsistent results (e.g.. Smith, Schneider, Smith, & Ananiadou, 2004).  The researchers hypothesized that perhaps the fit between the core components and  societal culture and school system quality contributed to the heterogeneous program effects. Specifically, the greatest benefits were seen in the program's country of origin, Norway, and similar cultures, whereas weaker results were seen in countries more dissimilar to Norway (Smith et al.). This finding suggests that even though a program has a large research base supporting its effectiveness, for accountabiUty purposes, it nevertheless should be evaluated at each implementation to verify continued effectiveness in any new context. In conducting these program evaluations, two aspects of the program are particularly important to investigate, due to the strong impact they have on program outcomes:  implementation integrity (how accurately an intervention is administered) and treatment acceptability (the degree to which intervention stakeholders view an intervention as fair, reasonable, and likely to succeed). Each is discussed below. Implementation Integrity The area of implementation integrity (also treatment fidelity and treatment integrity), or the degree to which a program was implemented consistently with the program design, has received a great deal of attention recently and will no doubt strengthen the field of program evaluation. Often times, despite good intentions, programs are not carried out as they were intended. For example, time constraints may lead to a component or activity not being fully completed, if at all, or because of inadequate training, the facilitator may not have the skills necessary to administer the program properly. Facilitators may not realize the importance of "following the script" and view going with the "natural flow" of the program as being more important than following the program explicitly. In other cases, a program facilitator can be put in a position in which they are required to run a program that they do not agree with or see value in. This individual may subsequently not facilitate the program to optimal standards or may do so in an unenthusiastic manner. A facilitator may also believe that the program is not working and stop implementing it. In any case, it is important when evaluating a program to know what was administered. Without such information, researchers cannot be sure if modifications by facilitators influenced change more than the program itself or if no changes were realized because a program was not administered accurately. In fact, some researchers argue that program implementation should be measured prior to outcomes (McCall et al., 2004), to prevent drawing conclusions of effectiveness regarding a program that may have not really even been implemented (Shinn,  2003). Despite the important information this can offer to program evaluation, until recently, the majority of prevention research did not include any measure of implementation (Greenberg, Domitrovich, Graczyk, & Zins, 2005; Kam, Greenberg, & Wall, 2003). Implementation data can help researchers understand the reasons behind heterogeneous and insignificant findings of program evaluations. Datta (2003) discusses this phenomenon of programs being "murdered by evaluation", where programs are deemed ineffective due to a lack of investigation into implementation integrity and control group experience.  Datta cites the  example of the Comer program evaluation (Milsap, Chase, Obeidallah, Perz-Smith, Brigham, & Johnston, 2000), an initiative to improve school culture, where insignificant differences were found between the Comer schools and non-Comer schools until implementation integrity was examined. Comer schools with high implementation integrity showed significant benefits over low implementation integrity schools. Similar results were found when examining treatment fidelity across implementation sites of  an anti-bullying program  (Salmivalli,  Kaukiainen, Voeten, & Sinisammal, 2004).  Implementation integrity varied widely across classrooms and subsequently, so did attitude and behavioural outcomes.  In classrooms where there was high implementation integrity of the  program, a 69% reduction in victimization was found and in low implementation integrity classrooms, a more modest 29.5% decrease was observed.  In terms of attitudes towards  bullying, only the groups who received high program implementation significantly improved. A subsequent study of the same program (Salmivalli, Kaukiainen, & Voeten, 2005) found congruent results.  Again, low implementation of the program resulted in less reduction of  victimization (-15% to -29%) compared to high implementation (-46% to -57%).  Consistent with the previously presented data, Battistich's (2000) evaluation of the Child Development Project, a school-wide program to enhance social-emotional skills and classroom climate, was largely affected by degree of implementation integrity.  Initial comparisons of  program and control or no-treatment schools revealed insignificant differences on measures of self-esteem, prosocial behaviour, aggressive behaviour, educational motivation, and other indices of social-emotional functioning. When implementation integrity data were considered, however, results revealed that students in schools that had committed to change on a school-wide basis saw significant improvements on social-emotional functioning measures when compared to comparison schools. Students in the high implementation schools saw protective benefits that lasted into middle school. Over the past decade, an increasing number of studies have demonstrated the importance of evaluating the fidelity or integrity of implementation when evaluating the impact of a given intervention, especially in terms of school-based interventions.  Indeed, an evaluation of the  Promoting Alternative Thinking Skills (PATHS) program demonstrated a significant effect for level of implementation (as well as administrator/principal support) and positive outcomes (Kam, Greenberg, & Walls, 2003). Similarly, in evaluating the impact of an anti-bullying intervention that focused on changing group attitudes and peer group/bystander behaviour as well as the behaviour of children who bully, Salmivalli and colleagues demonstrated that overall reductions in bullying were much greater in classes with high implementation integrity than in classes in which implementation integrity was low (Salmivalli, Kaukiainen, Voeten, Sinisammal, 2004). Finally, multi-site evaluation of Resolving Conflict Creatively, a school-based socialemotional program that emphasizes peer mediation and conflict resolution training, yielded similar findings of a positive "dosage-response" relationship. Researchers found that that the  program led to larger social-emotional and academic benefits when the children were exposed to substantial curriculum, as opposed to few or no lessons (Aber, Jones, Brown, Chaudry, & Samples, 1998; Brown, Roderick, Lantieri, & Aber, 2004; Selfridge, 2004). Given the surprisingly low number of program sites that implement programs with high fidelity (e.g., 45% in Salmivalli et al., 2004; and 42% in Battistich, 2000), and the finding that implementation integrity strongly influences outcomes, it is of the utmost importance that a measure of implementation be incorporated into any program evaluation. Without this, one cannot be sure what is being investigated, and perhaps as Datta (2003) said, many would-be effective programs are "murdered by evaluation." As well as providing information on implementation integrity following the facilitation of a program, including a measure that encourages facilitators to reflect on and formatively monitor their implementation integrity may actually lead to increased adherence to the intervention (Telzrow & Beebe, 2002). In their review of school-wide bullying programs with similar core components Smith, Schneider, Smith, and Ananiadou (2004) found that including some measure of program monitoring was related to significantiy higher positive outcomes for reduction of victimization. In addition to verifying the program integrity that occurred during an intervention trial, Dane and Schneider (1998) have identified conditions that promote program integrity. Components that promote, or increase the likelihood of program adherence, include the availability and use of adequate program manuals, sufficient facilitator training, and ongoing support and supervision for program facilitators.  Other factors that have been identified as  increasing implementation integrity are addressing behaviours with social validity and treatment acceptability (Telzrow & Beebe, 2002).  To summarize, investigation into implementation integrity increases the researchers' ability to interpret the effects of an intervention because it has been demonstrated to have a strong relationship with program outcomes (Dane & Schneider, 1998; Greenberg, Domitrovich, Graczyk, & Zins, 2005). This particularly allows for the interpretation of outcomes for how the program would typically be administered when adopted, rather than when facilitated by researchers under ideal conditions.  Information on implementation can also guide program  development and improvement, advance and support the theoretical framework of the program, and provide documentation of program adherence to comply with ethical and legal obligations (Greenberg et al.). Treatment Acceptability Since the degree to which a program is implemented can so drastically affect its outcomes, it is important to look at the variables that affect implementation integrity.  A  predominant factor that influences implementation integrity is treatment acceptability or the degree to which a treatment is deemed acceptable, appropriate, beneficial, fair, and feasible by the intervention facilitators and recipients (Sugai & Tindal, 1993). Outcome expectancy refers to the positive outcomes and negative side effects that program stakeholders believe will result from the intervention (Waas & Anderson, 1991).  Perceived likelihood of success of the  intervention has often been found to be the variable most predictive of treatment acceptability (Spreat & Walsh, 1994; Telzrow et al., 2002). Whether or not a parent, teacher, or whoever else may be administering the intervention agrees with it, largely affects whether he/she will carry the program out with high implementation integrity and continue with it as long as recommended (Allinder & Gates, 1997; Elliot, Witt, & Kratochwill, 1991).  Research has shown that parents tend to initiate treatment with their children when they perceive the recommended treatment to be acceptable.  Conversely, low ratings of treatment  acceptability were related to not pursuing treatment for their child's A D H D 3-4 months following diagnosis (Krain, Kendall, & Power, 2005).  Treatment acceptability no doubt  contributes to the roughly 50% of A D H D children whose families do not initiate or adhere to recommended treatments. Many non-program specific variables have been found to be related to treatment acceptability and adherence. Facilitators tend to view treatments more favourably when they employ positive reinforcement rather than response-cost (i.e., removal of desired stimulus) and punishment approaches (Jones, Eyberg, Adams, & Boggs, 1998; Waas et al., 1991); when treatments goals are phrased to increase positive behaviours, rather than decrease negative ones (Jones et al.; Telzrow & Beebe, 2002); when there is a high perceived severity of the problem (Martens, Witt, Elliott, & Darveaux, 1985); and when there is the perception of a good intervention-context fit (Telzrow & Beebe). Although these variables should be considered in the development, dissemination, and adoption of a program, program-specific variables that lead to high implementation should be considered as secondary program evaluation components (Forgatch, 2003). Treatment acceptability also plays a large role in the school setting with regard to schoolbased programming.  Teacher "buy in" has been found to be a significant predictor of  implementation integrity and positive outcomes (Selfridge, 2004), so much so that some institutions have employed methods to determine where to allocate resources based on likelihood of success. Support and leadership from the school principal as well as 70% teacher support is required by the N Y C Department of Education of Resolving Conflicts Creatively Program for  participation in the program (Brown et al., 2004). A n evaluation of the Promoting Alternative Thinking Skills (PATHS) program also demonstrated a significant link between principal support (and quality of implementation) and positive outcomes (Kam et al., 2003). Olweus (2004) found that the program facilitator variables that most strongly predicted good implementation included viewing the program goals as important, as their responsibility, and within their capabilities to address.  As well, the degree to which teachers reviewed the  program material and perceived there to be a bullying problem in their classroom improved implementation (Kallestad & Olweus, 2003).  Variations in outcomes were more strongly  correlated with these classroom and facilitator differences, than school level differences. In summary, the degree to which stakeholders believe an intervention will work and view it as manageable impacts their beliefs regarding its acceptability. As a result of being perceived as highly acceptable, or fair and reasonable, a facilitator will be more likely to implement a program with high integrity and continue implementing it for the duration of the designated time period.  Since these factors have been linked to outcomes, it is important that program  evaluations include measures of implementation integrity and treatment acceptability. Social-Emotional Factors A focus in schools around educating children as a whole has developed considerable momentum in the past decade.  A recent summary of issues and challenges that today's  adolescents face (Greenberg, Weissberg, O'Brien, Zins, Fredericks, Resnik, & Elias, 2003) highlighted larger schools, a high prevalence of untreated mental health difficulties, and high risk sexual and violent behaviour. These factors, in addition to the generally negative impact of the media on adolescents in terms of aggression, risky sexual behaviour, body image, and substance use (Roberts, Henriksen, & Foehr, 2004), have left many concerned about adolescents' well-  being and development. For many, such concerns have contributed to increasing emphasis on addressing social, emotional and behavioural challenges in schools (see Hymel, Schonert-Reichl & Miller, 2006). Social-emotional learning (SEL) and social responsibility have become important areas of the school curriculum. In British Columbia, social responsibility is listed among reading, writing, math, and information technology as a key area for education, with specific standards and expectations (BC Ministry of Education, http://www.bced.gov.bc.ca/pcrf stands/). Although emphasis on social responsibility is voluntary in British Columbia, in the United States, the state of Illinois has mandated social-emotional learning in all of its classrooms (Illinois Learning Standards, http://www.isbe.state.il.us/ils/social emotional/standards.html ). The  Collaborative for  Academic, Social,  and  Emotional Learning  (CASEL;  www.casel.org), a leading institution in the promotion and evaluation of social-emotional learning (SEL) programs, cites five components for social and emotional learning (SEL): selfawareness, or the ability to identify one's own emotions and strengths; social awareness, such as perspective taking; self-management, like being able to manage one's own emotions; relationship skills, like assertive communication, compromise, and developing personal boundaries; and responsible decision making, like taking personal responsibility and problemsolving. Also important for child and adolescent development is instilling in them a sense of autonomy, belonging, and competency (Christenson & Havsy, 2004). Fortunately, many programs have been developed to prevent negative developmental trajectories and promote pro-social behaviour, positive and effective coping strategies, and problem-solving skills.  Successful prevention programs, among other things, have targeted  substance use (Dusenbury et al., 1995), bullying (Smith, Pepler, & Rigby, 2004), emotional and  behaviour problems (Greenberg, Kusche, Riggs, 2004) and peaceful conflict resolution skills (Aber et al., 1998; Brown et al., 2004). In addition to creating more well-rounded, responsible citizens, a school focus on socialemotional learning can also lead to improvements in academic achievement (see Hymel et al, 2006 for a review).  For example, school-based, social-emotional prevention/intervention  programs such as Promoting Altemative Thinking Strategies or PATHS (Greenberg et al., 2004), the Resolving Conflicts Creatively program (Brown et al., 2004) and the Peaceful Schools Project (Fonagy, Twemlow, Vemberg, Sacco & Little, 2005) have each documented enhanced academic outcomes, as well as improvements in social-emotional functioning. As well, a recent analysis of over 300 studies on SEL by Weissberg and Durlak (2005) has shown that children who participate in social-emotional intervention and prevention programs demonstrate stronger achievement, higher grades, better attendance and school liking as well as more constructive and less problematic behaviour than students who did not participate in such programs. Academic improvement is particularly strong when social-emotional learning is integrated into academic areas (Elias, 2004). Taken together, then, a growing body of research emphasizes the importance of schoolbased efforts to foster social and emotional growth, with evidence that such efforts yield positive effects on academic performance as well (see Hymel et al., 2006 for a review). With regard to the present evaluation project, we now turn to a consideration of particular aspects of social and emotional development that are emphasized in the Welcome To My Life (WTML) program. Self-Concept and Self-Esteem Self-concept and self-esteem refer to two distinct aspects of one's self perceptions. Selfconcept is the intellectual perception or cognitive assessment of one's abilities or capabilities  across various domains, whereas self-esteem is an affective, and often global appraisal of the self in positive versus negative terms. High self-esteem and self-concept have typically been linked to desirable psychological, academic, social, and behavioural outcomes, whereas deficits in these areas have been linked to negative outcomes and high risk behaviour (for a review, see Haney & Durlack, 1998). Currently, self-concept is conceptualized as a hierarchy of self perceptions (Shavelson, Hubner, & Stanton, 1976, as cited in Marsh, 1990a) and self-concept theorists believe that one's self-concept is derived from a dynamic interaction between one's experiences, various environments, social comparisons, and the individual's appraisals of these (Marsh, 1990a). It is also believed that one's behaviour is guided by one's self-concept; the individual is more likely to partake in behaviours congruent with one's self-concept and reject behaviours that are inconsistent (Heider, 1958; Towberman & McDonald, 1993). Relationships between self-concept and particular developmental trajectories have been well documented.  A large base of research, in particular, addresses the relationship between  self-concept profiles and the development of psychopathology and negative health outcomes. Strong relationships have been found between self-concept deficits and internalizing behaviours (e.g., anxiety, depression) and less clear relationships between self-concept and externalizing behaviours (e.g., aggression and delinquent behaviour) (Egan & Perry, 1998; Metalsky, Joiner, Hardin, & Abramson, 1993; Robinson, Garber, & Hilsan, 1995). In their longitudinal study, Egan and Perry found that a low general self-concept, and specifically a low social self-concept, an important area during adolescence, were related to peer victimization and internalizing difficulties, such as anxiety and depression. Robinson, Garber, and Hilsan found low self-worth to predict depression, but not externalizing behaviours.  These students were particularly  vulnerable to depression and their low self-worth led them to be more likely to respond to stress, during the transition from elementary school to junior high school, with depressive symptoms. Conversely, high or positive self-worth was found to buffer the effect of negative attribution style, a well-researched predictor of depression, when exposed to stressors during the school transition. Similar results were found in the development of depressive symptoms in college students in response to academic failure and low self-esteem (Metalsky et al.). Poor self-concept has also been related to negative health behaviours among adolescents. Thorton, Douglas, and Houghton (1999) found that deficits in the physical self-concept domain in particular were related to the onset and maintenance of tobacco use, especially among adolescent females.  Similar results have been found with regard to poorer self-concept and  increased risk for alcohol use (Towberman et al., 1993) and gang involvement (Herrmann, McWhirter, & Sipsas-Herrmann, 1997). A number of studies have found low self-esteem to be related to current, as well as future, high risk sexual behaviour (e.g., unprotected sex, being paid for sex) in female adolescents (Ethier, Kershaw, Lewis, Milan, Niccolai, & Ickovics, 2006; Miller, Forehand, & Kotchick, 2000) and adults (Sterk, Klein, & Elifson, 2004). In addition, the use of tobacco (Sperber, Peleg, Friger, & Shvartzman, 2001), alcohol, and marijuana (von Sydow, Lieb Pfister, Hofler, & Wittchen, 2002) in adolescence appears to be correlated with low self-esteem. Adolescents with low self-esteem are also more likely to use less effective coping and problem solving strategies than individuals with high self-esteem (MuUis & Chapman, 2000). Prevention and intervention programs aimed at improving self-esteem and self-concept have yielded mixed results.  Prevention programs aimed at increasing female students' self-  esteem and body image have had some success in improving both areas as well as preventing  (Zabinski, Wilfley, Calfas, Winzelberg, & Taylor, 2004) and decreasing disordered eating behaviours (O'Dea & Zabinski, 2000; Wilfley, Calfas, Winzelberg, & Taylor, 2004), although the effects may not be long lasting (McVey, Davis, Tweed, Shaw, 2004). Haney and Durlack (1998) reviewed programs aimed at increasing self-esteem and self-concept and evaluated whether they improved the outcomes associated with low levels of both. They found that selfesteem and self-concept programs were extremely successful at improving behavioural and academic functioning of children with high externalizing problems.  Outcomes depended on  initial symptom presentation, with students who had externalizing behaviours benefiting most, followed by students with internalizing behaviours, and then students with a combination of both.  Although this review looked at both self-concept and self-esteem interventions, it is  surprising that children with externalizing symptoms benefited most, considering that selfconcept deficits are more consistently found to be related to internalizing problems, not externalizing problems (Egan et al., 1998; Metalsky, et al., 1993; Robinson et al., 1995). Most evaluations of these self-concept and self-esteem programs lacked follow up data, so long-term effects are not known. To summarize, research has clearly documented links between self-concept/self-esteem and a variety of behavioural and mental  health outcomes  among adolescents,  and  intervention/prevention efforts aimed at enhancing self-esteem have met with some success, although results are mixed. Concerns about self-esteem and self-concept may be especially warranted, however, during the transition from elementary to secondary schools, as discussed in the next section.  The Transitional Period The transition from elementary school to high school represents a particularly difficult period for many youth, especially when these transitions coincide with the developmental shift from childhood into adolescence (e.g., pubertal change) (Simmons & Blyth, 1987; Lord, Eccles, McCarthy, 1994). heterogeneously,  Many other life changes and available resources affect individuals  thereby leading to varied outcomes prior to, during, and following the  transitional period (McDougall & Hymel, 1998; Simmons et al., 1987). As such, a variety of risk and protective factors affect the quality of an individual's transition from elementary to secondary school and one's emergence from it, as reviewed below. Group Level Organizational Context Changes. While schools certainly vary in culture and context, there are nearly universal aspects of junior high and secondary schools relative to elementary schools (Simmons, Carlton-Ford, Blyth, 1987).  First, junior high and secondary  schools are typically larger than elementary schools, hi the Simmons et al. sample, students went from elementary schools with an average of 59 grade mates to a junior high school with an average of 403 grade mates. Depending on the student, this change could potentially lead to negative outcomes like loneliness or alienation, or positive outcomes like access to new and varied students with whom to become friends. A second change occurs in terms of departmentalization.  Unlike elementary school,  where a student would typically have a single teacher and the same classmates for the majority of his/her classes, in high school, students have classes with many different students and teachers. It is not entirely clear how these factors relate to various social and academic outcomes (Simmons et al.). However, McDougall and Hymel (1998) found that most students had a relatively easy transition when moving into a small middle school where the same students  typically had classes together, a finding that supports the idea of size and departmentalization as potential risk factors. Attitudes and Perceptions. Survey research indicates that students have mixed feelings prior to transitioning to middle school and high school (Akos & Galassi, 2004). Students looked forward to the increased freedom at their new school, including being able to choose their own classes and take elective courses, making new friends, and participating in school events. In terms of concerns, students reported worrying about bullying, getting lost in the school, and handling increased academic demands. Students at the lower end of the grade distribution (e.g.. Grade 7 in a Grade 7-9 school) are more likely to feel anonymous and to dislike school than students at the upper end of the grade distribution (Simmons & Blythe, 1987). Students at the bottom of the grade distribution feel significantly more anonymous than same grade peers at a school where they are at the top of the grade distribution (e.g.. Grade 7 in a K-8 school).  This effect is largely related to  transitioning to a new school, having a larger number of same age and older peers at one's school, and what the authors refer to as the "top dog" phenomenon. Students in their first year of junior high school are also more likely than same age peers who did not enter junior high school, to feel more discontinuity in their schooling. Other studies have found that the transition is not particularly stressful or troublesome for adolescents and that they report feeling sufficiently happy and successful during the initial entry period into middle school (McDougall & Hymel, 1998). The sample used in the McDougall and Hymel study, however, attended a relatively small Grade 7 and 8 middle school where the students moved as a group from class to class which may have decreased the sense of discontinuity in the transition to middle school.  Self-Esteem/Self-Concept During the School Transition.  Developmentally, many  phenomena are occurring during the period in which students transition to middle school or high school with regard to self-esteem and self-concept. It is a point in their development when selfesteem (Lameiras & Rodriguez, 2003; Robins, Trzesniewski, Tracy, Gosling, & Potter, 2002) and self-concept (Marsh, 1989) are often at their lowest, which puts students at risk for a variety of high risk behaviours and negative consequences, as previously discussed.  Students who  reported difficulties during the initial months of middle school are also the students who reported poor self-concept and social adjustment (Lord et al., 1994; McDougall & Hymel, 1998). Thus, a strong self-concept appears to be related to a less difficult transition. Although some studies have not documented significant gender differences relating to self-esteem during the transition period (Deihl, Vicary, & Deike, 1997; Lameiras & Rodriguez, 2003), many other studies have found modestly, but consistently lower global self-esteem (Khanlou, 2004; Kling, Hyde, Showers, & Buswell, 1999; Quatman & Watson, 2001; Simmons & Blyth, 1987; Simmons, Blyth, Van Cleave, & Bush, 1979) and self-concept (Watkins, Dong, & Xia, 1997) among female adolescents. Prior to adolescence (age 9-12), girls and boys report fairly similiar levels of self-esteem. While many students of both sexes experience a decline in global self-esteem during adolescence, girls tend to decrease more drastically (Robins et al., 2002) and greater variability is seen in female (relative to male) adolescent self-image (Simmons et al., 1987). A summary of the research on age and levels of self-concept found that many, although not all, studies document a decline in self-concept during preadolescence, and none found an increase during this period (Marsh, 1990b). In addition, girls tend to experience a dechne in self-esteem prior to and upon entering junior high school that same age girls who are not entering a new school or boys in either school  environment do not experience (Cantin & Boivin, 2004; Lord et al., 1994; Simmons & Blyth, 1971; Simmons et al., 1979). Given that age and biological factors are held constant, this decrease appears almost solely related to factors surrounding the transition to a new school, where they move from "top dog" to "bottom dog" position, and where there are more same age and older peers per grade level, creating a more impersonal organizational context.  For a  considerable portion of girls, the recovery of self-esteem following the transition to junior high takes years (Simmons, Carlton-Ford, & Blyth, 1987). This risk is especially high for girls experiencing the strongest decline during the transition period to junior high school. In one study of self-esteem during school transitions, Delhi, Vicary, and Deike (1997) distinguished three groups with distinct trajectories: stable low self-esteem (16%), slight increases in selfesteem (37%), and stable high self-esteem (47%) over the four years following the transition to junior high school. This confirms prior evidence (Simmons et al.) that many students do not recover their self-esteem significantly, and typically not quickly. The relationship between transitioning to junior high school and self-concept is more complex.  Decreases in perceived academic competence (Lord et al., 1994; Sinmions et al.,  1987) along with simultaneous increases in perceived social competence have been observed (Cantin et al., 2004). Perceived competence in academic areas can serve as a protective factor against decreased self-esteem for students transitioning to junior high school (Cantin et al.). Low scores on self-concept and self-esteem measures put students at risk for unhealthy peer relationships (Marsh, Parada, Yeung, & Healey, 2001). Given the importance of self-concept and self-esteem as a potential risk / protective factor (low vs. high self perceptions) in adolescent development, as well as research documenting general declines in self-concept/self-esteem during adolescence and during the  transition to secondary school, especially for girls, a major focus of the W T M L program is on fostering and maintaining a positive sense of self.  Accordingly, an evaluation of self-  concept/self-esteem among participants in the W T M L program was deemed a critical inclusion in the present study. In evaluating self-concept/self-esteem, however, it is important to recognize the multidimensional nature of the construct. Although individuals may have an overall sense of positive versus negative self esteem, the multi-faceted nature of self-perceptions has been increasingly recognized. Accordingly, it becomes important to assess self-concept in a number of domains of the adolescent's life. Stressors and Psychological Weil-Being. The more life change events that an adolescent experiences during the period of time surrounding the transition to junior high school (e.g., school change, pubertal change, early dating, geographical mobility, major family disruption), the more likely there will be detrimental effects to their self-esteem, grade point average, and extracurricular participation (Sinmions, Burgeson, & Blyth, 1987). Negative life events, school hassles, and other stressors coinciding with the time of transition put adolescents at increased risk for both depressive and externalizing problems (Hillegers, Burger, Wals, Reichart, Verhulst, Nolen, & Ormel, 2004; Robinson, Garber, & Hilsan, 1995). The proximity and quantity of these changes (as opposed to less and/or being spread over a longer period of time) appear to decrease the adolescent's coping abilities, and may contribute to negative consequences which persist years later (Grade 9 and 10), especially for girls (Simmons et al.). Academic areas appear particularly resistant to recovery (Simmons et al.). The level of support that students feel from people in different areas of their life, as well as their level of perceived empowerment was evaluated as an outcome for this study.  Peer Relationships.  Peer evaluations and social adjustment become a particularly  important influence on students' self-esteem during the transition period (McDougall et al., 1998; Simmons et a l , 1987). The shift to secondary school is associated with a heightened desire to belong and be a member of a peer group, often times as a way of establishing one's own identity (Pratt & George, 2005).  Surprisingly, although there is a greater desire for  companionship and students report a more integral role for peer support, some research suggests a temporary decline in social network size around the time of entering junior high school (Cantin et al., 2004). Having poor relationships with peers puts children at risk for an unsuccessful transition (Hawkins, Smith, & Catalano, 2004).  Peer rejection, in particular, appears to be  associated with a number of negative outcomes (McDougall, Hymel, Vaillancourt & Mercer, 2001). Although starting and maintaining friendships is easy for some, many students need additional instruction to develop skills in this area. There can be devastating outcomes for those who lack the necessary social skills to have healthy friendships.  Students with few or no  reciprocated friends have been found to have poorer academic achievement (Bemdt & Keefe, 1995; DeRosier, Kupersmidt, & Patterson 1994; Wentzel, Barr, & Caldwell, 2004) and display less prosocial behaviour than those with reciprocated friends in early adolescence (Wentzel et al.). Group membership has been found to be the most significant predictor of concurrent and long-term education functioning (Wentzel & Caldwell, 1997). Wentzel, Barr, and Caldwell also found significant emotional distress among those students with fewer friends. These children with poorer social relationships also face a greater likelihood of dropping out of school (Coie, Dodge, & Kupersmidt, 1990; Parker & Asher, 1987).  Studies investigating the prevalence of bullying across grade level have documented a discouraging trend. As students reach early and middle adolescence, they are increasingly less likely to report  assisting peer  victims (O'Connell,  Pepler, Craig,  1999; O'Connell,  Sedighdeilami, Pepler, Craig, Connolly, Atlas, Smith, & Charach, 1997), report decreasing concern for victims in bullying situations (Rigby & Slee, 1991), and see aggression less negatively than at other points of childhood and adolescence (Pellegrini, Bartini, & Brooks, 1999).  Concurrently, bullying has also been shown to increase in frequency during late  childhood to early or middle adolescence, with a peak generally occurring around age 9 to 15 (Hazier, 1996). In particular, the transition to middle school is related to an increase in bullying (Pellegrini & Long, 2002), which may be a way of establishing the social hierarchy in a new school social setting. Bullying dynamics put bullies, victims, and bystanders at increased risk for social-emotional and psychological maladjustment (Rigby, 2003; Rigby, 1998; Carney, 2000; Eslea et al., 2003). The protective importance of positive peer relationships during the transitional period between elementary and high school, as well as the potential risks for students who lack these social connections, is well documented. highlighted in the W T M L program.  The development of friendship skills is an area  Given the program goals and the importance of peer  relations, perceived relationship competence with female and male peers was investigated as one potential outcome of the W T M L program in the present study. Familial and School Issues. School personnel and family members can have a large positive impact on adolescent students. Having a supportive family and good relationships with educators has found to strongly contribute to the success of adolescents in impoverished conditions (Floyd, 1997) and reduce the likelihood of high risk behaviour (Resnick et al., 1997).  While parents may feel less important in their adolescent's life, the parent-child relationship still has a large influence on their functioning outside the home. Rigby (1993) found that prosocial behaviour in adolescence was correlated with familial well-being and positive parent-adolescent relationships. Conversely, peer difficulties were related to poor relationships with parents and poor familial health. Poor family functioning was also found to have a moderate relationship with victimization among adolescent girls. Familial support also serves as a protective factor against much of the stress and difficulties associated with transitioning to high school. Adolescents who perceived their parents to be supportive of them felt a stronger sense of school membership following the transition (Isakson & Jarvis, 1999), which is, in turn, related to school motivation and academic outcomes (Goodenow, 1993).  hicreased self-esteem and more positive social attitudes following the  transition to junior high school were related to parents respecting their adolescents' autonomy and allowing them to participate in decision-making (Isakson et al.). From an extensive review of the research, Christenson and Peterson (1998, as cited in Christenson & Havsy, 2004) identified six factors across school, family, and community contexts that relate to school success and likely aid students during potentially difficult transitional periods. These include clearly stated and specific expectations and goals, structure, learning opportunities, support, quality of relationships, and being a good role model. Given the protective influence of school and familial support for female students during the transition from elementary school to high school, and the W T M L emphasis on strengthening these relationships, perceived school and parental support was investigated as potential outcome variables in this study.  Academic Achievement. There is some evidence to suggest a decUne in academic achievement following the transition to junior high school (Simmons, et al., 1987) and to high school (Isakson et al., 1999), especially for students who experience high stress during the transition. Other research; however, suggests that the relationship between the transition and academics is more complex and dependent on individual factors.  Duchesne, Larose, Guay,  Vitaro, and Tremblay (2005) observed that the academic functioning of students progresses along three development trajectories: students with consistently high grades from Grade 5 to secondary school (63% of students), students whose grades are stable and low (23%), and students who grades drop significantly (14%). Those students who experienced a decline in academic functioning over the transitional years tended to be aggressive, peer rejected, and have a poorer home life. However, it is unclear how the transition affected academic functioning as the students' decline began prior to the transition to high school. Student attendance also typically decreases following school transitions (Barone, Aguirre-Deandreis, & Trickett, 1991; Isakson et al., 1999). To summarize, students are at risk for a decline in academic functioning during the transition. Perceived academic competence was investigated in this study. Summary of the Factors Involved in the Elementary School to High School Transition. As indicated in the preceding review, there are a number of factors, both risk and protective that may impact a student's ability to cope with the transition to secondary school. Accordingly, in evaluating any prevention/intervention program aimed at preparing students to better address this transitional period, it becomes important to evaluate their perceptions of support and competence across a number of different factors. Furthermore, in the present evaluation study, effort was  made to examine changes in student perceptions of their competencies (self-concept) as well as their perceived available assets and support in a broad range of areas. Welcome To My Life The Y W C A created the Welcome To M y Life program (WTML; See Appendix A for more detailed program information) in response to Vancouver School Board (VSB) concerns that many girls were having difficulty transitioning to high school. Based on developmental research and information obtained from focus groups with Grade 7 girls, the Y W C A proposed that many of the girls felt ill prepared for high school and the socially complexity it entails. The issues that arose became  themes of the program, including self-esteem,  relationship skills, time  management, personal boundaries, body image concerns, parental relationships, problemsolving, and self-awareness. Program development was also influenced by the McCreary Centre Society (Tonkin, Murphy, Lee, Saewyc, & The McCreary Centre Society, 2002) and Girls in Canada 2005 reports (Calhoun Research and Development & C. Lang Consulting, 2005; both as cited in the Y W C A Welcome To M y Life program manual) which further emphasized the importance of social skills, problem solving, and trusting relationships across contexts for girls during the adolescent period. The Search Institute's (http://www.search-institute.org/) research on developmental assets also strongly influenced the development of the W T M L program. Developmental assets are positive experiences and attributes that are related to healthy psychological and social outcomes, 40 of which have been identified by the Search Institute (further description in Measures section). A l l activities in the W T M L program were designed to target specific developmental assets. The W T M L program was offered at four Vancouver schools during the spring of 2007. W T M L consisted of 6-8 modules (2.5-3 hours each) offered through weekly, after-school  sessions at ttie selected schools.  A l l Grade 7 girls at the selected schools were invited to  participate in W T M L ; the program was available for minimal or no cost for those who could not afford it. Given the vast literature pointing to declines in self-esteem/self-concept as a significant risk factor during the adolescent school transition, as well as the protective influence of positive peer and parental relationships, the W T M L program focussed on the development of these areas. W T M L program developers aimed to increase Grade 7 girls' developmental assets, relationship skills, and self-esteem through collaborative and activity-based learning which are consistent with the B C Ministry of Education's Integrated Resource Package (IRP; as cited in the Y W C A Welcome To M y Life Manual). If the program is deemed to be promising by the evaluation, it may become more broadly available for Grade 7 girls in Vancouver area schools. Program Promotion The program was promoted to Grade 7 students by the program developers in a number of ways. A pizza party and scavenger hunt was held at each school during a lunch hour to talk about the program. School principals promoted this event to the Grade 7 girls to ensure a high attendance. Program developers and school principals verified that these efforts were successful in yielding a strong turnout. At the end of this event, the girls were provided with information and permission forms to take home. These forms were also mailed to the houses of all Grade 7 girls by the schools. The previously described program promotion methods were ones completed by the program developers and Y W C A staff systematically for all program sites. Informal methods of recruitment were also conducted by various members of the school staff.  Given the variable  support for the program among classroom teachers and principals, it could be expected that some  classroom teachers would be more diligent in reminding students about the program and encouraging them to register in it.  Individual students may have also been differentially  encouraged or discouraged to participate in the program based on the likelihood they would benefit from the program or detract from other participants' enjoyment of the program. This was not systematically evaluated, but information of this nature was informally provided to the program evaluator by school staff members (see Limitation section in Chapter 5). The Present Study As described in this chapter, social-emotional programs have been increasingly developed to address documented needs and enhance positive development. The evaluation of such programs has become increasingly sophisticated and valued. The transition from elementary school to high school presents many challenges to female students. Given this need, the Y W C A developed the W T M L program to help Grade 7 girls develop the skills, self-concept/self-esteem, and perceived assets to transition successfully. To summarize, a growing body of literature underscores the importance of addressing social-emotional development within our schools (Hymel et al., 2006), as well as the need to support students, especially girls, as they navigate the transition from elementary to secondary schools (Simmons & Blyth, 1987).  The W T M L program, developed by the Y W C A in  partnership with the Vancouver School Board, represents one attempt to address these issues in a time-limited, after-school program designed for Grade 7 girls. A primary focus of the program was on enhancing student self-esteem/self-concept and helping students understand the support and assets that are available to them during this transition In keeping with a growing emphasis on the importance of "evidence-based practice" and the need to verify the effectiveness, in addition to the efficacy of school-based programs  (Prevention Research Center for the Promotion of Human Development, 2005), the purpose of the present research was to evaluate the impact of the W T M L program on its intended participants. Grade 7 girls who would soon be making the transition to secondary school. Of primary interest was an outcome evaluation, considering whether participants changed in areas emphasized by the program, namely self-esteem/self-concept and perceived assets. Using a prepost comparison, quasi-experimental design, W T M L participants were compared with grademates who chose not to participate in the program in terms of self-reports of self-esteem/selfconcept as well as perceived developmental assets collected both before and after the program. It was expected that participants would demonstrate greater gains in self-concept/self-esteem from pre- to post-program implementation and would also be more able to recognize the developmental assets available to them to support them in their transition. Ideally, this evaluation study would have included random assignment of subjects to experimental and control groups ( W T M L or no W T M L ) or a waitlist control group ( W T M L now vs W T M L later). However, given school commitments to the program, random assignment was not an option for this evaluation. "Wait list" control group opportunities were explored, but were also not possible, given that the Y W C A had not yet established school program sites for the subsequent program implementation, and therefore "future" participants would not be able to complete the "pre-treatment" questionnaires prior to the time the program would be administered in the experimental schools. In light of these limitations, the most viable comparison group for participants in the W T M L program was the group of girls at the program school sites who chose not to participate in the program. The advantages and disadvantages of this type of design are considered more fully in the Discussion section of this report.  Of additional interest was the degree to which the program was faithfully administered {implementation integrity) and supported {treatment acceptability) by the program facilitators, and  whether  satisfaction).  participants  themselves  found  the  program  useful/effective  {participant  Implementation integrity has been shown to have a significant impact on the  outcomes of an intervention (Aber et al., 1998; Brown et al., 2004; Salmivalli et al., 2004). Programs which are administered as they are intended are more likely to yield significant effects than programs implemented with less fidelity. The degree to which program facilitators agree with the intentions of the program and view it as feasible, fair, and likely to succeed (treatment acceptability)  impacts  the  degree to  which  they  implement  the  program  faithfully.  Implementation integrity is additionally promoted by high quality facilitator training manuals and facilitator training (Dane & Schneider, 1998). Collecting implementation data throughout the duration of the administration of the program has also been shown to increase implementation integrity (Telzrow et al., 2004). To this end, following the facilitation of each module, program facilitators completed a questionnaire regarding the level of implementation integrity and treatment acceptability for the session. Facilitators indicated whether or not they completed the activities that were in the lesson plan, how long they spent preparing for the module, and how much they agreed with the goals and activities of the session. In addition, regarding participant satisfaction. Grade 7 female program participants from all four program sites, including the two that participated in the evaluation study, completed three open-ended questions following the completion of the entire program that addressed their satisfaction with WTML. The data collected from students and facilitators were used to address three major research questions: 1) Does participation in the Welcome To M y Life program lead to increases  in self-esteem and self-concept when compared to the comparison group? 2) Do participants in the Welcome To M y Life program increase their perceptions of their positive attributes or assets, relative to students who did not participate in the program? 3) Were the participants satisfied with the Welcome To M y Life program? 4) What was the level of implementation integrity and treatment acceptability during the facilitation of the program?  CHAPTER 3 Research Methodology This study investigated the impact of the Welcome To M y Life program on Grade 7 girls self-reports of self-esteem, self-concept and perceived positive assets. Implementation integrity, treatment acceptability, and participant satisfaction were also evaluated.  L i this chapter, the  study participants, measures, and methods used to conduct this evaluation are described. Evaluation Stakeholders There were many different stakeholders and roles involved in the facilitation and evaluation of the W T M L program. First, members of the Y W C A staff created and developed the program and oversaw the training of program presenters and facilitation of the program. These individuals are referred to as the "program developers" for the remainder of the text. Second, within the elementary school context, school principals or vice principals served as the primary school contact for the program.  Third, "program facilitators" or "facilitators" for W T M L  included the U B C Trek students and community volunteers (Wisdom Champions) who facilitated and presented the program to the Grade 7 girls at individual elementary schools. U B C Trek students are undergraduate or graduate students who, through participation with the U B C Trek program, are involved in community initiatives and programming in underprivileged Vancouver areas for a minimum of four months for 1-4 hours weekly. Wisdom Champions are volunteer members of the community, typically older than the U B C Trek students, who provided guidance, support, and facilitation during the program'  Finally, perhaps one of the most  important stakeholders are the Grade 7 girls who participated in the W T M L program, hereafter  There were also high school mentors involved in the program; however, their role was less defined and consistently used, so they were not investigated during this evaluation.  referred to as the "program participants" or "participants". In evaluating the W T M L program in the present study, effort was made to address the perceptions of each of these stakeholders. Participants Student participants for the outcome evaluation portion of the study were drawn from two elementary schools in a large urban school district in western British Columbia that agreed to implement the Y W C A Welcome To M y Life (WTML) program in the spring of 2007. Students were recruited for the Welcome to M y Life Program by the Y W C A , in partnership with the VSB. A l l Grade 7 girls were invited to participate in the program. Girls who chose not to participate in the program were invited to participate in the evaluation study as comparison subjects. Parental permission forms were sent home to all parents (see Appendix B). Students who returned the permission forms (regardless of whether parents did or did not give permission) were entered into a draw for a $25 gift certificate to a local music store (i.e.. Future Shop). The final outcome evaluation sample included 42 Grade 7 female students from two elementary schools (n=27 and 15). Of that sample, 12 girls participated in the W T M L program (n=6 at each of the participating schools) and 30 girls comprised the "no program" comparison group (n=21 and 9, respectively, at each participating school). Pre-Intervention  Post-Intervention  Participant Group  n= 12  n= 12  Comparison Group  n = 30  n = 30  With regard to an evaluation of participant satisfaction, following the final module of W T M L , program participants were asked to answer three questions, two open-ended, regarding their satisfaction with the W T M L program. The students who completed these questions were  those from the present sample who participated in the W T M L program as well as participants from two program sites that did not participate in the outcome assessment project. Across the four school sites, a total of 22 participants were present during the final module to answer the questions (n=4 from each participating school in the present study and n=5 and 9, respectively, from the two additional schools). Finally, with regard to implementation integrity and treatment acceptability, information about the program was also collected from eight facilitators across all four program sites. Program facilitators at each of the program sites were consistent throughout the duration of the program. Data Collection and Procedures For those involved in the primary outcome assessment, two self-report, paper-and-pencil questionnaires were completed by the all the Grade 7 female students who received parental consent for participation in the evaluation project and who themselves agreed to participate, including program and comparison girls at the two schools that participated in the outcome assessment. The students completed the measures during two 45-60 minute class periods, at one school and during lunch hours at the other school, with all sessions overseen by the author. Students completed the questionnaires on two occasions, once before the program was implemented and once after it was implemented. Students with parental consent were informed that participation in the study was on a voluntary basis and that they would not be penalized for withdrawing from the study. Students were read information regarding the project and were asked to provide written assent (See Appendix B). A l l information gathered from participants of this study was considered confidential. A numerical code was used for identification and no personal information was on the questionnaires.  Program participants from all four program sites, both ones that participated in the outcome assessment and ones that did not, answered three questions regarding program satisfaction using a paper-and-pencil format on the final day of the program.  These post-  program satisfaction surveys were administered by program facilitators.  Finally, the author provided program facilitators at all four program sites with implementation questionnaires via electronic mail, following the administration of each module. Upon completion, they were returned, via email or paper copy, to the Y W C A program developers who collected them for the author. Reminders were sent via email to facilitators who had not yet returned the questionnaire for the previous week's session. Measures Two self-report  surveys were completed for the outcome assessment by both  participating and comparison students to evaluate the impact of the Welcome To M y Life (WTML) program at two program sites, prior to and following the administration of the program. Copies of the surveys are provided in Appendix C. In addition, for the participant satisfaction evaluation three questions regarding their impressions of the program, two of which were openended, were completed by the program participants at all four program sites during the final program module.  Copies of the questionnaire, as administered to students are provided in  Appendix D. Ratings of implementation integrity and treatment acceptance were completed by program facilitators at all four program sites following each program module. Copies of the rating forms on which they provided their ratings are presented in Appendix E. The measures used in this evaluation study are discussed in further detail in the following paragraphs. Self Description Questionnaire-II (SDQ-II). Self-concept is the intellectual perception of oneself across various domains, whereas self-esteem is an affective, global appraisal of the self  in positive versus negative terms. In the present study, these two areas were measured by the Self Description Questionnaire II (SDQ-II; Marsh, 1990), a 102 item, self-report, pencil and paper measure that can be completed in 15-20 minutes.  Three academic areas (Reading,  Mathematics, and General School) and seven non-academic self-concept areas (Physical Abilities, Physical Appearance, Same Sex Relations, Opposite Sex Relations, Parent Relations, Emotional Stability, and Honesty-Trustworthiness) are measured by the SDQ-II. The General Self sub-scale serves as a measure of overall self-esteem. Of interest in this study were student reports of (a) overall self-esteem, (b) overall self-concept, as measured as a sum of scores across self-concept domains, and (c) self-concept in each domain. To complete the SDQ-II, adolescents responded to statements (e.g., 17. "I look forward to English classes. ", 63. "My parents understand me") by checking one of six options: "False", "Mostly False", "More False Than True", "More True Than False", "Mostly True", or "True". Items in which a "True" or "Mostly True" response would indicate poor self-concept/self-esteem (e.g., 86. "/ have trouble with most school subjects. ", 36. "Nothing I do ever seems to turn out right. ") were reverse coded for scoring to not artificially inflate scores. Student responses to items in each self concept domain and to items tapping overall self-esteem were summed to create a composite score in each case, and student responses to all items were summed to create an overall self-concept composite. Higher scores on all 11 sub-scales as well as the self-concept composite reflected higher or more positive reports of self-concept or self-esteem. Extensive psychometric support for the SDQ-II has been derived from reliability studies, factor analyses, and mulitrait-multimethod designs (see Marsh, 1990). Internal consistency has typically been strong, with alpha coefficients obtained for the self-concept and self-esteem subscales ranging from .83 to .91 (median = .87) and the composite self-concept score having a  mean coefficient alpha of .94. Construct validity of the SDQ has been supported by factor analytic studies, which have verified the domain structure of the measure. Self-concept scores have been shown to be relatively stable; when retested after a seven-week interval period, subscale reliability ranged from .72 to .88 across sub-scales. Developmental Assets Profile. Developmental assets are experiences and personal traits that the Search Institute has identified as being related to psychological and social well-being and development. Higher levels of developmental assets are correlated with making safer choices and being protected from negative outcomes and high risk behaviour (Search Institute, 2004). In the present study, changes in perceived assets were assessed using the Developmental Assets Profile (DAP, Search Institute, 2004). The D A P was created by the Search Institute as a measure of individuals' developmental assets to complement the Search Institute Profiles of Student Life: Attitudes and Behavior ("A&B"), which estimates the community presence of 40 assets as it relates to a group of youth. Unlike the " A & B " , the D A P is appropriate for program evaluation as it is sensitive to changes (in asset category scores) across time. This self-report, pencil-andpaper measure takes approximately 10-20 minutes for students to complete. The D A P is a 58-item questionnaire in which youth rate statements (e.g., "I enjoy learning. ", "I stand up for what I believe in. ") on a 4-point scale (0, 1, 2, 3) in terms of whether the item is "Not At A l l or Rarely", "Somewhat or Sometime", "Very or Often", or "Extremely or Almost Always" like them. Student responses for items in each scale were averaged, with derived scores ranging from zero to three. The average was then multiplied by 10 and rounded off to the nearest whole number (.5 and above round up to the nearest integer), for scale scores ranging from 0 to 30. Internal and External Asset Composite scores were obtained by computing the average of the scale scores that form the composite. The four external asset categories  (Support, Boundaries and Expectations, Empowerment, and Constructive Use of Time) were averaged to form the External Asset Score. The same procedure was used to derive the Internal Asset Score from the four internal asset categories that comprise it (Commitment To Learning, Positive Values, Social Competencies, and Positive Identity). The Internal and External Asset scores were summed to form a Total Asset Score (0-60). Alternatively, scores can be interpreted across contexts or domains that the student functions in (Personal, Social, Family, School, and Community).  Context scores were computed in the same manner as the other scale scores  described above. Computed scores for the Internal Asset, External Asset, and individual scales (asset and context) can be used to place students in one of the four categories of perceived developmental assets: (1) Excellent (26-30); (2) Good (21-25); (3) Fair (15-20) or; (4) Low (014). Derived Total Asset Scores can be classified also be classified as falling in the Excellent (51-60), Good (41-50), Fair (30-40), or Low (0-29) range.  The present study investigated  reported perceived assets as measured by the (1) Total Asset Score, (2) Internal Asset Score, (3) External Asset Score, (4) Individual asset domain scores, and (5) Context domain scores. Information regarding the reliability and validity of the D A P is limited given the newness of the measure. However, initial investigations of its psychometric properties based on field testing by the test developers show promise (Search Institute, 2004). Mean internal consistencies were relatively high: .81 across the eight asset category scales, .88 across the five context scales, .93 for the Internal Assets composite, .95 for the External Assets composite, and .97 for the Total Assets composite scale. Averaged test-retest reliability over a two-week period was moderate for the eight asset scores (Mean r=.79). Internal Asset scale (r=.86). External Asset scale (r=.84), and Total Asset scale (r=.87). In terms of validity, the D A P demonstrated .62-.82 correlations with parallel measures on the Search Institute's Attitudes and Behaviours (A&B) survey.  Participant Satisfaction Questionnaire. Participating Grade 7 girls, from all four program schools, answered three questions regarding the program's perceived success in preparing the girls to transition to high school, what they learned as a result of participating in the program, and if they would recommend it to a friend. See Appendix D for a copy of the questionnaire. Responses to open-ended questions were reviewed and then categorized based on the naturally arising categories and range of response topics. Inter-rater reliability was established for the categorization of the responses for all of the responses. Following the evaluator categorizing the responses, a second rater^ was provided with the responses on place cards and asked to place them in the most applicable category. Rates of interrater reliability for individual questions are discussed in the Results chapter. Responses (Y/N) to the question, "Would you recommend the Welcome To My Life program to a friend?" were tallied to determine the percentage of students who would and would not recommend the program to a friend. Responses to "Do you feel more prepared to enter high school as a result of participating in this program? Why or why not?" were classified initially as indicating "yes" or "no" to feeling better prepared for the transition due to W T M L . "Yes" responses were categorized as referring to gaining an increased awareness of the issues the may encounter when they enter high school, developing new skills for coping with the transition, or increasing their level of self-confidence. "No" responses were categorized as the participant still having unanswered questions and concerns about the transition or already knowing the information presented in the program. Responses to the question, "How has participating in the Welcome To My Life program helped you (e.g, feel better about yourself solve problems with peers better, improve your  ' The second rater was a fellow graduate student whose training has included graduate level coursework in research methodology, statistical analyses, and data collection.  relationships with your parents, improve your body image, gain a better understanding of the demands of high school)?" were categorized as indicating (a) improved relationships (e.g., with parents or friends), (b) increased understanding or preparedness for high school, (c) increased self-esteem/self-awareness, or (d) vague/no response. When a respondent's answer addressed more than one category of improvement, it was included in all of the relevant categories Lnplementation hitegrity and Treatment Acceptability. Shinn (2003) calls it a "Type III" error when conclusions are drawn regarding the effectiveness of programs without investigation into the level of implementation, especially considering the prevalence of partial and inadequate implementations. Measures of implementation integrity or the degree to which a program is implemented the way it is supposed to be, and treatment acceptability, or the degree to which stakeholders view the program to be fair, reasonable, appropriate, and likely to succeed, were filled out by program facilitators. Telzrow and Beebe (2002) recommend the use of formative treatment integrity checks and intervention monitoring as a means of increasing treatment integrity. In the present study, facilitators completed a measure of implementation integrity following each module, or session, where they indicated which activities in the module's plan were completed (Yes/No).  Percentages were derived for each module and across the entire  duration of the program for how many module specific activities were facilitated, how many routine activities were facilitated (e.g., review of previous module, module evaluation) and how frequently the overarching themes of the program were incorporated into the module (e.g., importance of adults). When there was inconsistent reporting of the completion of an activity (Y/N) between respondents at the same program site, an average was taken across the respondents to yield a percentage of the activities administered during each module. Facilitators  were also asked how much time they spent preparing for the module independently and with their co-facilitator (0-15 minutes, 15-30 minutes, 30-60 minutes, 60-90 minutes, over 90 minutes).  The median amount of reported preparation time, independently and with a co-  facilitator, across each module was calculated to allow for investigation into trends (e.g., independent vs. with co-facilitator preparation time, preparation time variations across modules). Treatment acceptability and outcome expectancies have been found to be related to the implementation level of programs (Telzrow et al., 2002). Questions of this nature addressed the facilitators' view of the importance of the activities and whether or not they felt they had sufficient training and skills to facilitate them. Items that addressed support for the philosophical basis of the program and activities (e.g., "/ think that social-emotional learning and skills training are critical to the transition of girls from elementary school to high school", "I believe in the value of all the activities in this module "), as well as ones that addressed having the necessary skills to facilitate the module (e.g., "Upon completion of the training I received, I felt prepared to facilitate the session. ", "I believe I had the necessary skills to facilitate the activities in the module. ") were answered on a five point scale with " 1 " being strongly disagree and "5" being "strongly agree".  Facilitators were also asked to indicate, with the same five-point  response format, the perceived enjoyment of the activities (e.g., "The activities in the module were engaging and enjoyed by the girls. "). Responses to these questions were averaged across program sites for each module.  The implementation integrity and treatment acceptability  questionnaires are provided in Appendix E. Dane and Schneider (1998) encourage investigation into the quality of the training manuals and the training of facilitators as both components have been found to promote implementation integrity. In the present study, these components were assessed through two  methods. First, facilitators were asked in the implementation integrity/treatment acceptability questionnaires to indicate how prepared they felt to administer each module based on the training they received (as described above). Second, the training manual and level of facilitator training was informally evaluated. Results of this evaluation are described in the Results section. Prior to the commencement of the program, the program evaluator provided the program facilitators with an explanation for the implementation integrity and treatment acceptability questionnaires, as well as, description of each construct and their importance to program facilitation and evaluation. Implementation integrity and treatment acceptability questionnaires were sent to each facilitator through e-mail by the program developers following each program module. Although reminders were sent when completed questionnaires were not returned, and although the program developers encouraged facilitators to complete the forms, completed questionnaires were not obtained from all the program sites for every module. At least one facilitator completed a questionnaire for every module at two schools. At a third school, at least one facilitator filled out a questionnaire for six of the seven modules. At the fourth school, at least one facilitator completed a questionnaire for four of the seven modules and the first page was completed for a fifth module. A l l received questionnaires were included in this study.  CHAPTER 4 Results This chapter includes preliminary analyses of the obtained data and discusses the primary analyses and results of the evaluation of the W T M L program. Primary analyses examined the impact of the W T M L program on Grade 7 girls in the areas of self-concept/self-esteem and perceived positive assets. In addition, qualitative and informal evaluations were used to explore issues of program implementation, particularly the level of implementation integrity, treatment acceptability, program promotion, facilitator training, and participant satisfaction with the program. Preliminary Analysis The data obtained from the pre- and post-intervention administrations of the SDQ-II and D A P were first examined for normalcy (means and standard deviations across scales), skewness, and kurtosis.  The internal consistency of individual scales and composite scores were also  investigated. Descriptive statistics for SDQ-II and D A P data is presented in Table 4.1 and Table 4.2, respectively. As shown in these tables, the data was deemed suitable for use in the proposed analyses (ANCOVAs) based on its psychometric quality which was sufficient for research purposes in terms of normal distribution and internal consistency of scales.  Statistics for SDQ-II Scores SDQ-ll Composites/Scales  Time 1 Mean (SD)  Time 2 Mean (SD)  Coefficient Alpha Combined Group (A'=42) a, ai .88 .93  Comparison Group («=30) 442.30 (42.91)  Participant Group (/7=12) 456.08 (38.84)  Combined Group (N=42) 446.23 (41.79)  Comparison Group(n=30) 461.73 (58.02)  Participant Group(n=12) 476.83 (40.21)  Combined Group (N=:42) 466.05 (53.05)  General Self (Self-Esteem)  46.07 (6.07)  47.75 (5.75)  46.55 (5.96)  49.23 (8.15)  52.08 (5.18)  50.05 (7.47)  .67  .83  Non Academic Areas Physical Appearance Honesty-Trustworthiness Physical Abilities Emotional Stability Parent Relations Same-Sex Relations Opposite-Sex Relations  29.63 (7.12) 48.00 (7.67) 36.90 (6.97) 40.57 (6.23) 39.30 (5.86) 48.17(5.72) 28.23 (9.19)  30.25 (7.24) 38.00 (7.78) 41.08 (8.53) 38.17(6.69) 32.25 (10.50) 53.08 (6.82) 36.75 (7.52)  29.81 (7.07) 45.14(7.87) 38.10(7.59) 39.88 (6.38) 37.29(8.01) 49.57 (6.38) 30.67 (9.49)  32.40 (8.02) 47.97 (8.52) 36.47 (6.98) 41.67 (9.03) 40.47 (7.00) 50.30 (6.20) 31.33 (10.35)  34.00 (6.90) 35.67 (10.46) 41.83 (5.91) 43.58 (6.14) 35.75 (8.90) 51.25 (5.05) 39.83 (5.37)  32.86 (7.67) 44.45 (10.60) 38.00 (7.06) 42.21 (8.29) 39.12 (7.78) 50.57 (5.85) 33.76 (9.93)  .80 .78 .83 .45 .80 .65 .88  .83 .86 .81 .70 .85 .64 .91  Academic Domain Areas Math 40.20(14.92) 43.75 (11.12) 41.21 (14.90) Verbal 40.47 (8.95) 44.42 (8.39) 41.60 (8.88) General School 44.77 (10.93) 50.58 (8.02) 46.43 (10.43) ai = alpha coefficient of time 1 scores; ai - alpha coefficient of time 2 scores.  42.40(12.85) 43.27 (9.20) 46.23 (1.56)  45.17 (11.61) 46.83 (9.03) 50.83 (8.08)  43.20(12.43) 44.29 (9.20) 47.55 (10.04)  .93 .77 .90  .92 .82 .90  Total Self-Concept  Statistics for D A P scores DAP Composites/Scales  Time 1 Mean (SD)  Time 2 Mean (SD)  Total Asset Score  Comparison Group(n=30) 43.62 (7.36)  Participant Group (n=12) 41.70 (8.08)  Combined Group (N=42) 43.07 (7.52)  Comparison Group (n=30) 43.12(7.79)  Participant Group(n=12) 42.57 (8.23)  Combined Group (A^=42) 42.96 (7.82)  Internal Asset Score Commitment To Learning Positive Values Social Competencies Positive Identity  21.29 (3.68) 21.57 (4.45) 22.12(4.21) 21.54 (4.93) 19.94 (4.41)  20.63 (3.66) 22.38 (5.00) 20.30 (3.71) 20.10(3.17) 19.72 (4.65)  21.10(3.64) 21.80 (4.56) 21.60 (4.11) 21.13 (4.51) 19.88 (4.42)  20.99 (4.32) 21.10(5.03) 21.79 (4.63) 21.29(5.15) 19.78 (6.35)  21.83 (4.21) 22.50 (4.56) 21.36 (4.60) 21.25 (5.14) 22.22 (4.34)  External Asset Score Support Empowerment Boundaries & Expectations Constructive Use of Time  22.33 (4.06) 23.10(5.28) 21.89(4.28) 24.41 (3.88) 19.92 (5.85)  21.07 (5.13) 19.88 (6.71) 22.64 (4.84) 21.76 (5.89) 20.00 (4.52)  21.97 (4.37) 22.18 (5.83) 22.10(4.40) 23.65 (4.63) 19.94 (5.45)  22.13 (3.87) 22.95 (4.07) 22.78 (4.51) 23.89 (4.02) 18.92 (6.52)  Context Areas Personal 21.21 (3.94) 20.71 (4.72) 19.49 (4.31) Social 22.21 (4.23) 21.87 (4.17) 21.03 (4.05) Family 24.60 (4.58) 23.38 (5.61) 20.33 (6.91) School 23.13 (4.46) 23.33 (4.43) 23.83 (4.53) Community 19.75 (4.98) 19.86 (4.81) 20.14(4.55) ai - alpha coefficient of time I scores; « 2 = alpha coefficient of time 2 scores.  21.28 (4.52) 21.59 (4.42) 24.57 (4.38) 22.93 (4.16) 19.33 (4.77)  Coefficient Alpha Combined Group (A^=42) «2  .93  .95  21.23 (4.25) 21,50(4.89) 21.67 (4.57) 21.28 (5.09) 20.48 (5.90)  .88 .68 .74 .67 .64  .93 .77 .85 .80 .86  20.74 (4.39) 19.40 (5.36) 22.92 (4.39) 20.83 (5.96) 19.79 (6.35)  21.73 (4.02) 21.94 (5.34) 22.82 (4.43) 23.02 (4.79) 19.17(6.41)  .88 .77 .55 .74 .24  .88 .76 .66 .80 .54  21.79 (4.60) 21.99 (4.59) 19.92 (5.33) 22.75 (4.75) 20.00 (4.62)  21.43 (4.49) 21.70 (4.42) 23.24 (5.07) 22.88 (4.31) 19.52 (4.68)  .76 .79 .87 .76 .78  .83 .84 .85 .79 .78  Primary Analyses A primary question addressed in the present study was whether or not participation in the Welcome To M y Life (WTML) program led to increases in self-esteem and self-concept as measured by the SDQ-IL  To address this question, a series of analyses of covariance  (ANCOVAs) were conducted, first with overall self-esteem as the dependent variable and subsequently with domain-specific self-concept scores as the dependent variables and finally the overall composite for self-concept as the dependent variable.  The analyses of covariance  allowed for an examination of the effect of the program on Time 2 (post-program) selfesteem/self-concept scores after controlling for initial. Time 1 (pre-program) scores, comparing students who did and did not participate in the program. Specifically, for these analyses. Time 2 self-esteem/self-concept scores served as the dependent variable(s), and group membership (participant or comparison group) served as the independent variable, with Time 1 selfesteem/self-concept scores as the covariate. A separate A N C O V A was conducted for each of the self-esteem/self-concept measures, including an overall measure of self-esteem, plus an overall self-concept composite score as well as 11 different domain-specific self-concept sub-scale scores, tapping self-perceptions across both academic (Reading, Mathematics, and General School) and non academic domains (Physical Abilities, Physical Appearance, Same-Sex Relations, Opposite-Sex Relations, Parent Relations, Emotional Stability, and HonestyTrustworthiness). A second research question addressed whether or not participation in W T M L resulted in increases in perceived positive assets when initial scores of perceived assets were controlled for. To answer this, a series A N C O V A s were conducted with group membership (participant or comparison group) as the independent variable, time two perceived positive asset scores as the  dependent variable, and time one perceived asset scores as the covariate. This analysis was run for Total Assets, Internal Assets, External Assets, and Context Assets, as well as the scales that compose these composite scores. The Internal Asset score is comprised of four scale scores: Commitment to Learning, Positive Values, Social Competencies, and Positive Identity.  The  scale scores that form the External Asset score include Support, Empowerment, Boundaries and Expectations, and Constructive Use of Time. The five contexts considered included Personal, Social, Family, School, and Community. Analyses of Covariance (ANCOVA), as opposed Analyses of Variance (ANOVA) were chosen because the former are better able to handle uneven cell sizes between experimental and control groups, and reduce the impact of not having randomized sampling and group assignment. A N C O V A s , which are an integration of A N O V A and regression methods, allow for the determination of a correlation between two variables when another theoretically related variable is controlled for.  For these analyses, a p value of .05 was chosen to determine statistical  significance and Cohen's (1982) criteria was chosen to determine effect size. Cohen (1988) recommends that effects of R" = 0.01 be considered small, R"^ = .09 be considered medium, and R = .25 be considered large. Results of these analyses for each of the dependent variables are presented in the subsections that follow. Given the small sample used this study and the relative imbalance of program participants to comparison students, which decreases the likelihood of detecting statistically significant findings, additional investigation looked into percentage change across measures as a result of program participation.  Impact Analysis (Mohr, 1995) recommends the use of  regression to evaluate the impact of an intervention, or the percentage of change that is attributable to the treatment, particularly when assignment conditions are not random. Results  are discussed for scale and composite score when the change attributable to the intervention was found to be 1% or greater. Self-Esteem.  The first A N C O V A conducted examined whether students who  participated in the W T M L program improved in their reported self-esteem or perceived selfworth (General Self score), relative to non-participants, after controlling for initial levels of overall self esteem. Specifically, an A N C O V A was conducted with Time 2 SDQ-II self-esteem scores serving as the dependent variable, program participation as the independent variable and Time 1 SDQ-II self-esteem scores as the covariate. Time 1 self-esteem scores were entered in the first block of the regression model and program participation was entered into the second block in order to determine whether the differences in overall self esteem were evident across experimental and control groups once initial scores were controlled for. Results were not statistically significant  {Rechange  =.009,  Fchange (1,39) = .577,  p = .452), although a 0.9% increase  in variability in self-esteem was attributable to participation in the program.. Total Self-Concept. Another A N C O V A was conducted to address the question of whether participation in the W T M L program led to improved ratings of overall self-concept relative to comparison students, over and above initial self-concept scores.  The dependent  variable was Time 2 SDQ-II total self-concept scores, program participation was the independent variable, and Time 1 SDQ-II total self-concept scores served as the covariate. The effect of participating in the W T M L program was not statistically significant (/?^change =-001, = .045,  p = .833). This change reflected a  0.1%  Fchange (1,39)  increase in total variance of self-concept in  program participants as a result of W T M L . Self-Concept Domains.  A series of A N C O V A s were conducted, with Time 2 self-  concept domain scores as the dependent variable, program participation as the independent  variable, and Time 1 scores as the covariate. Results of the A N C O V A s indicated that changes in domain scores due to program participation were not statistically significant. Results of these analyses are presented in Table 4.3. Although the observed changes across domains were not significant, increases in variance of 1% or greater as a result of program participation were found in three domains. Specifically, a 4.7% increase was observed on the Emotional Stability scale, a 2.6% increase was reported on the Physical Abilities scale, and a 1.1% increase was seen on the Opposite-Sex Relations scale. Self-Esteem and Self-Concept Summary. Across SDQ-II scores of self-esteem, overall self-concept, and domain specific self-concept scores, statistically non-significant program effects were observed when initial scores were controlled for.  The W T M L program did,  however, lead to notable increases in variance in three specific domains of self-concept: Emotional Stabihty (4.7%), Physical Abilities (2.6%), and Same-Sex Relations (1.1%). Table 4.3 for a summary of statistical analyses results for SDQ-II scores.  See  Results of Statistical Analyses for SDQ-II Domain  df  AF  P  Total Self-Concept  .001  1  .045  .833  Percentage Change -1-0.1%  General Self (Self-Esteem)  .009  1  .577  .452  -1-0.9%  .006 .019 .026 .047 .000 .011 .011  1 1 1 1 1 1 1  .297 2.177 2.565 3.309 .036 .601 .939  .589 .148 .117 .077 .851 .443 .339  -1-0.6% -1.9% -1-2.6% -14.7% 0.0% -1.1% -1-1.1%  Academic Domain Areas Math .000 Verbal .000 General School .000 NoteA^=42. *p < .05. **p<.00l.  1 1 1  .002 .017 .006  .967 .896 .940  0.0% 0.0% 0.0%  Non Academic Areas Physical Appearance Honesty-Trustworthiness Physical Abilities Emotional Stability Parent Relations Same-Sex Relations Opposite-Sex Relations  Overall Perceived Positive Assets.  Following from analyses conducted for self-  esteem/self-concept (as described above), another A N C O V A was conducted to evaluate changes in perceptions of overall positive attributes or assets as measured by the D A P Total Asset score over time across W T M L and control subjects. Time 2 D A P Total Asset scores served as the dependent variable, program participation was the independent variable, and Time 1 DAP Total Asset scores were the covariate.  Changes in overall perceived positive assets as a result of  program participation were not statistically significant after controlling for initial perceived positive asset scores  (/?^change=  -001, Fchange(l,39) = .391, p = .535) and only a 0.1% increase in  variance in overall perceived assets resulted from program participation.  Internal Assets. Analyses investigated changes in overall internal assets (Internal Asset score) and the scales that comprise it (Commitment To Learning, Positive Values, Social Competencies, Positive Identity) as a result of participating in W T M L . A series of A N C O V A s were conducted with Time 2 levels of D A P internal assets and subscales treated as the dependent variable, program participation as the independent variable and Time 1 internal assets as the covariate. Results indicated that the effects of program participation were not significant when initial Internal Asset scores and Internal Asset scales (Commitment To Learning, Positive Values, Social Competencies, Positive Identity) were controlled for.  See Table 4.4 for a  summary of statistical results. Although not statistically significant, participation in W T M L led to a 2.3% increase in variance for the overall Internal Asset scores. In terms of specific internal assets areas in which change occurred, a 4.1% increase in variance was observed in the Positive Identity domain. External Assets. Changes in overall perceived external assets (External Asset score) and the scales that form the external asset composite (Support, Empowerment, Boundaries & Expectations, Constructive Use of Time) resulting from W T M L participation were investigated. A series of A N C O V A s were conducted to determine the impact of W T M L  (program  participation; independent variable) on Time 2 External Asset and scale scores (dependent variable) after controlling for initial or Time 1 External Asset and scale scores (covariate). The impact of W T M L was not statistically significant on External Asset and scale scores after controlling for initial scores on those measures. See Table 4.4 for a summary of statistical results. Contexts. A series of A N C O V A s explored changes in perceived positive assets across contexts (Personal, Social, Family, School, Community) as a result of participating in the  W T M L program. For these analyses. Time 2 D A P Context scores were treated as the dependent variables, program participation was the independent variable, and Time 1 DAP Context scores were the covariate. Results were statistically nonsignificant across all contexts (see Table 4.4 for a summary of statistical results), although notable increases were found for a few context scores. The W T M L program led to increases in variance in perceived positive assets on the girls' Personal (2.8%) and Social (1.9%) context scores. Perceived Positive Assets Summary. Similar to the outcomes on the SDQ-II, the effects of program participation were not significant when initial scores on all scales the D A P were controlled for. However, notable increases in variance of over 1% as a result of W T M L were found on the Internal Asset score (2.3%), Positive Identity (4.1%), Personal context (2.8%), and Social context (1.9%) scales. See Table 4.4 for statistical results.  Results of Statistical Analyses for D A P Domain  AR'  df  AF  P  Total Asset Score  .004  1  .391  .535  Percentage Change +.4%  Internal Asset Score Commitment To Learning Positive Values Social Competencies Positive Identity  .023 .006 .010 .008 .041  1 1 1 1  1.971 .433 .717 .533 2.697  .168 .514 .402 .470 .109  +2.3% +.6% +1% +.8% +4.1%  External Asset Score Support Empowerment Boundaries & Expectations Constructive Use of Time  .003 .009 .001 .009 .003  1 1 1 1  .320 1.232 .058 .853 .289  .575 .274 .810 .361 .594  -.3% -.9% -.1% -.9% +.3%  Context Areas Personal Social Family School Community NoteA^=42. *p < .05. **p<.001.  .028 .019 .019 .006 .001  1 1 1 1 1  1.740 1.647 2.574 .585 .120  .195 .207 .117 .449 .731  +2.8% + 1.9% -1.9% -.6% +.1%  Qualitative Feedback From Program Participants Twenty-two of the 31 Grade 7 girls (71% of the participating students) at the four program sites were in attendance on the final day of the program at their respective schools to complete a final questionnaire about the W T M L program.  Student responses to these final  questions regarding the program provide initial data regarding how the program was received by the participants. In response to the question, "Would you recommend the Welcome To M y Life program to a friend?", all 22 (100%) of the students who completed the quesfionnaire responded that they would.  In response to the question, "Do you feel more prepared to enter high school as a result of participating in this program?  Why or why not?".  A 92% interrater agreement rate was  established for categorizing responses for this question. Eighteen (82%) of the girls responded that they felt more prepared to enter high school as a result of participating in the program. Of those students, the most commonly cited reasons included gaining an increased awareness of the issues they may encounter when they enter high school (n=8), developing new skills for coping with the transition (e.g., ways of handling peer pressure; n=4), and increasing their level of selfconfidence/self-awareness (n=4).  These areas were consistent with the goals of the W T M L  program. Four students (18%) responded that they did not feel more prepared for transitioning to high school or provided a vague/unclear response (e.g., "maybe", "yes and no"). Responses included still having many unanswered questions and concerns about high school (n=l) or already knowing the information that was covered in the program (n=l). Four students provided no reasons or uncodable reasons ("I'm not going yet.") for their response. The third question presented to the students was, "How has participating in the Welcome To M y Life program helped you (e.g., feel better about yourself, solve problems with peers better, improve your relationships with your parents, improve your body image, gain a better understanding of the demands of high school)?". Answers were categorized as indicating (a) improved relationships (e.g., with parents or friends), (b) increased understanding or preparedness for high school, (c) increased self-esteem/self-awareness, or (d) vague/no response. Interrater reliability for these categorizations was 87%. When a respondent's answer addressed more than one category of improvement, it was included in all of the relevant categories. Students reported that the program increased their understanding of or prepared them for transitioning to high school (n=10), increased their self-esteem/self-awareness  (n=6), and  improved their relationships or relationship skills (n=3). Five responses were coded as vague (e.g., "all", "yes", "don't know") and three students did not provide an answer for the question. Overall, the results indicate that the majority of participants reported that they would recommend the W T M L program to a friend (100%) and that they believed it provided them with assistance in areas consistent with the goals of the program (82%). Implementation Integrity Implementation integrity, or the degree to which a program or intervention was administered as intended, was measured through questionnaires completed by the program facilitators following each module. Time Spent Preparing for Modules.  The program developers expected the program  facilitators to spend at least an hour and a half preparing for each module of the W T M L program. To determine how much time they actually spent preparing for the modules, the program facilitators were asked following each session how much time they spent preparing during the past week with their co-facilitators (0-15 minutes, 15-30 minutes, 30-60 minutes, 60-90 minutes, over 90 minutes) and on their own (0-15 minutes, 15-30 minutes, 30-60 minutes, 60-90 minutes, over 90 minutes). Responses are summarized in the table below.  Time Spent Preparing For Modules Independent Preparation Time (min) 0-15  15-30  Module 1  30-60  60-90  6(86%)  1(14%)  Co-Facilitation Preparation Time (min) 0-15  15-30  30-60  60-90  7(100%)  Module 2  3 (50%)  3 (50%)  3 (50%)  1 (17%)  Module 3  4 (80%)  1 (20%)  2 (40%)  3 (60%)  Module 4 2 (33%)  2(33%)  2(33%)  4(67%)  2(33%)  Module 5 1(14%)  3(43%)  3(43%)  6(86%)  1(14%)  Module 6 1 (16%)  4(67%)  1(16%)  5(83%)  1(17%)  Module 7 1 (25%)  2 (50%)  1 (25%)  3 (75%)  1 (25%)  2 (33%)  Although statistical analyses were not possible, an inspection of the data presented in the table suggests three general trends regarding the time facilitators reportedly spent preparing for the modules. First, facilitators generally reported spending more time preparing for the initial modules than they did preparing for the latter modules. As presented in Table 4.5, 100% of facilitators reported preparing 30-90 minutes independently for the first module, where as 87% and 75% of facilitators reported preparing 0-30 minutes for the sixth and seventh module, respectively.  Likewise, in reports of time spent preparing with a co-facilitator, 100% of  facilitators reported spending 15-30 minutes preparing with a co-facilitator for the first module, whereas the majority of facilitators spent only 0-15 minutes preparing for the fifth (86%), sixth (83%), and seventh (75%) modules. Second, facilitators typically reported spending more time preparing for modules independently {Mdn = 15-30 minutes) than preparing with a co-facilitator  (Mdn = 0-15 minutes) across the modules. Third, the facilitators reported spending less time preparing for facilitating the modules than the program developers had intended. As seen in Table 4.5, only one facilitator, for one module, reported spending over 60 minutes independently preparing and none reported spending over 60 minutes preparing with a co-facilitator. Activities Administered.  Three components of the program were assessed  for  implementation integrity: module specific activities, routine activities that were to be completed as part of the routine for every module, and management guidelines or components that were to be used at the facilitators' discretion. Module specific activities included activities, lessons, or games designed for particular modules that were specific to the theme of the individual module. This included activities like making friendship bracelets, a discussion regarding personal values, and a lesson on the media influences on female adolescent body image. Routine activities were components that were to be included in the routine of every module, such as reviewing the previous module's themes, completing module evaluations, working on the community service project, and having the participants reflect in their journal.  The third area assessed was  management guidelines or program themes. They were activities to be incorporated into all of the modules as appropriate, including, for example, using the choice stone to deal with misbehaviour when applicable and to delegate turns during discussions, emphasizing the importance and the roles of adults in the participants' lives, and following the designated time frame for activities.  Facilitator reports regarding each of these three types of activities are  discussed in the subsections that follow. Module Specific Activities. The number of activities designed specifically to address the themes of individual modules that were facilitated was assessed by asking facilitators to indicate " Y E S " or " N O " regarding whether each specific activity was implemented during the module.  Typically, one or two facilitators completed the questionnaire after a module at each program site. Responses were investigated between facilitators at the same school for each module when more than one facilitator returned a questionnaire for a given module at a school. When there was disagreement between raters, the average number of activities reported across raters was computed. At two school sites, program facilitators completed questionnaires for all the modules. The facilitators reported implementing 84.6% of the planned activities at one site and 92.3% of the activities at the other over the course of the entire program. The other two schools did not have a facilitator submit an implementation questionnaire for every module.  One reported  facilitating 100% of the designated activities for the four of seven modules for which they completed a questionnaire and the other reported completing 100% of activities during six of seven modules for which they completed a questionnaire. Overall, implementation integrity was strong across program sites based on facilitator reports. Reported implementation integrity varied somewhat across modules and schools. This data is presented in Table 4.6.  Reported Rates of Activity Completion SCHOOL A  SCHOOL B  SCHOOL C  SCHOOL D  100% (n=2)  100% (n=2)  100% (n=l)  100% (n=2)  100% (n=l)  83.33% (n=2)  MODULE 1  78.57% (n=2)  MODULE 2  100% (n=2)  MODULE 3  100% (n=2)  MODULE 4  100% (n=2)  100% (n=l)  100% (n=2)  100% (n=l)  MODULE 5  87.50% (n=2)  100% (n=l)  100% (n=2)  75% (n=2)  MODULE 6  66.67% (n=2)  100% (n=l)  100% (n=l)  83.33% (n=2)  MODULE 7  75%  TOTAL  84.60%  100% (n=l)  (n=2)  100% (n=2) 100%  100%  92.30%  Although some variability in implementation across schools and modules was observed, most modules were reported to be delivered in their entirety. In only two cases were modules presented at less than 80%. These results suggest that implementation was high and poor implementation integrity was not likely to be the cause of the statistically insignificant findings on outcome measures. Routine Activities. Every module's plan included four activities that were expected to be completed: reviewing the past module, having participants write reflections in their journals, working on the community service project, and completing module evaluations.  For each  module, respondents were asked to indicate whether or not each of these activities were completed ("YES"/"NO").  Facilitators reported a high rate of adherence to the routine and  consistent components of the program.  Reported Rates of Routine Activity Completion SCHOOL A  SCHOOL B  SCHOOL C  SCHOOL D  100% (n=2)  100% (n=2)  100% (n=l)  100% (n=2)  100% (n=l)  83.33% (n=2)  MODULE 1  78.57% (n=2)  MODULE 2  100% (n=2)  MODULE 3  100% (n=2)  MODULE 4  100% (n=2)  100% (n=l)  100% (n=2)  100% (n=l)  MODULE 5  87.50% (n=2)  100% (n=l)  100% (n=2)  75% (n=2)  MODULE 6  66.67% (n=2)  100% (n=l)  100% (n=l)  83.33% (n=2)  MODULE 7  75% (n=2)  TOTAL  84.60%  100% (n=l)  100% (n=2) 100%  100%  92.30%  Each module, excluding the first module, was intended to start with a review of the themes and activities addressed in the previous module. This was reported to occur in 100% of four modules at one school, 100% of five modules at a second school, and 83.3% of six modules at a third school. At the fourth school, the previous module was reported by facilitators to be reviewed in five modules; however, there was disagreement as to whether it was reviewed for the sixth module. Completing module evaluations was another component with which there was high adherence. Module evaluations were reported to be completed by the participants for 100% of modules at all four program sites. Living the Learning, a community service project, was a component that was to be worked on during every module.  Program participants, with the guidance of program  facilitators, initiated a community service project and developed it through out the program's  duration. Two schools reported working on the community service project during 100% of four and six modules, respectively, and one reported working on the project during 85.7% of seven sessions. For the fourth school, there was interrater disagreement for one of the modules, with one facilitator reported that this component occurred and one reporting it did not.  The  community service project was reported to be facilitated at that site consistently during the remaining six modules. Participants were asked to write reflections on the module in their journal during each module.  According to reports from facilitators, this component was consistently facilitated.  Among school sites, three schools completed reflections for 100% of modules that they completed questionnaires for and one completed reflections in 85.7% of modules. Overall, the four routine components were reported to be faithfully implemented by facilitators in this program. Management Guidelines and Program Themes. This area assessed the presence of program components that were not designated activities, but that were to be followed at the discretion of the facilitators. Facilitators circled " Y E S " or " N O " to indicate the presence or lack of each component in this area (See Appendix E). In the past, the program developers had reported to the program evaluator that they were concerned that the choice stone was not being used in relevant situations. The choice stone was to be used when a conflict, misbehaviour, or chaotic atmosphere arose. A facilitator was to the choice stone to the participant or participants behaving inappropriately to remind them that they have the ability to choose to continue the undesirable behaviour or choose a more prosocial behaviour.  During this evaluation, the choice stone appeared to be a component that was  variably, but generally infrequently utiUzed. Across the four program sites, two schools reported  never using the choice stone (0%), one school reported using the choice stone during one of six modules (16.7%), and one school reported using the choice stone during all seven modules (100%). Concerns had been raised during previous implementations of the W T M L program that time was not always managed as intended during the modules.  For example, it had been  observed that certain activities initiated a good discussion and facilitators opted to continue with the conversation rather than complete the other activities. During previous implementations, facilitators were encouraged to use a timer that was provided and set it at the beginning of each activity to aid in time management.  Facilitators for this implementation had expressed to the  program developers that they preferred the time limits to not be so formal and opted to not use the timers, which the program developers permitted. During this implementation of the program, respondents indicated that they adhered sufficiently to the time guidelines laid out in the program manual. Across program sites, all faciUtators (100%) reported keeping activifies to within a few minutes of the designated time frame across all modules (100%). Although this was the case, perhaps because the program developer emphasized the importance of following the time guidelines during facilitator training, anecdotal reports provided on the questionnaires by facilitators indicated this was an area of difficulty.  Some facilitators noted that they and the  participants did not enjoy having to end a conversation or activity that was progressing well to move onto the next activity due to time guidelines and would have preferred more flexibility and discretionary judgement regarding timing issues. An overarching goal of the W T M L program was to emphasize the importance and role of adults in the participants' lives. Facilitators were instructed to incorporate this theme throughout the program, where appropriate, and preferably to incorporate that message into every module.  Facilitators reported incorporating this theme into 100% of six modules at one school, 85.7% of seven modules at a second school, 75% of four modules at a third school, and 57% of seven modules at the fourth school, reflecting considerable variability in implementation across sites. Treatment Acceptability As previously described, the degree to which intervention stakeholders, in this case the program facilitators, view an intervention as important, reasonable, fair, and likely to succeed has significant implications for the success the program will have on the recipients.  If the  program is believed to have high treatment acceptability, it is more likely to be followed as planned and in an enthusiastic manner. W T M L program developers had expressed concerns that some of the facilitators may not have valued social-emotional learning, the particular goals of the W T M L program, or individual modules or activities. During the implementation of W T M L that was evaluated for the present thesis, there was a high degree of treatment acceptability reported by the program facilitators at the four program sites across many components of treatment acceptability, as discussed in the subsections that follow. Perceived Importance of Social-Emotional Skills Training For the Transition from Elementary School to Secondary School. In order to assess their overall view of the importance of social-emotional training in schools, facilitators were asked to evaluate the degree to which they agreed with the following statement, "I think that social-emotional learning and skills training are critical to the transition of girls from elementary school to high school." Responses were made on a 5-point scale ranging from "strongly disagree" to "strongly agree". Five of the seven facilitators (71%) "Strongly Agreed" that such programming was critical for preparing Grade 7 girls for the transition and the other two facilitators (29%) "Agreed" that it was  important. Overall, these results suggest that the program facilitators valued the overall focus of the program as part of the school's efforts. Effectiveness of Training and Preparedness for Facilitation.  Following each module,  program facilitators were asked two questions that addressed (a) the degree to which they felt prepared to facilitate the W T M L program given the training they received, and (b) whether or not they felt they had the requisite skills to facilitate the module. Responses to both questions were marked on a 5-point scale, ranging from "Strongly Disagree" to "Strongly Agree". Reports indicated that across modules, facilitators agreed or strongly agreed that they had the requisite skills and were adequately prepared to present the program as a result of the training they received. Table 4.8 Reported Preparedness To Facilitate Modules "I believe I had the necessary skills to facilitate the activities in this module."  "Upon completion of the training I received, I felt prepared to facilitate the session."  MODULE 1  n=7  4.57 (Range = 4-5)  4.70 (Range = 4-5)  MODULE 2  n=6  4.83 (Range = 4-5)  5.00 (Range = 5)  MODULE 3  n=5  4.60 (Range = 4-5)  4.60(Range = 4-5)  MODULE 4  n=6  5.00 (Range = 5)  5.00 (Range = 5)  MODULE 5  n=7  4.70 (Range = 4-5)  4.70 (Range = 4-5)  MODULE 6  n=6  5.00 (Range = 5)  4.83 (Range = 4-5)  MODULE 7  n=4  5.00 (Range = 5)  4.50 (Range = 3-5)  4.81  4.76  MEAN  Support for specific modules. Program facilitators were asked how much they agreed with the following statement, "I believe in the value of all the activities in this module." Responses ranged on a five-point scale from "strongly disagree" (1) to "strongly agree". Responses were averaged for each module. Results, as presented in Table 4.9, indicated that the facilitators believed there was value in the activities they led across modules. Table 4.9 Support For Module Activities "I believe in the value of all the activities in this module."  MODULE 1  n=7  4.29 (Range 2-5)  MODULE 2  n=6  4.67 (Range 4-5)  MODULE 3  n=5  4.40 (Range 3-5)  MODULE 4  n=6  4.83 (Range 4-5)  MODULE 5  n=7  4.83 (Range 4-5)  MODULE 6  n=6  5.00 (Range 5)  MODULE 7  n=4  5.00 (Range 5)  MEAN  4.72  Perceived enjoyment by the girls of activities. The program facilitators were asked to indicate on a 5 point scale how much they agreed ( 1 = Strongly Disagree, 5 = Strongly Agree) to the following statement "The activities in the module were engaging and enjoyed by the girls." Results are presented in Table 4.10. Overall, the facilitators perceived the Grade 7 girls to enjoy and be engaged in the activities in each module.  The overall perceived enjoyment of the  program, obtained by averaging the ratings across all the modules, was 4.53.  Responses  generally indicated that the facilitators believed that the Grade 7 girls enjoyed the W T M L program. Table 4.10 Perceived Enjoyment of Activities "The activities were engaging and enjoyed by the girls." MODULE 1  n=7  4.43 (Range 3-5)  MODULE 2  n=6  5.00 (Range 5)  MODULE 3  n=5  4.20 (Range 3-5)  MODULE 4  n=6  4.30 (Range 4-5)  MODULE 5  n=7  4.76 (Range 4-5)  MODULE 6  n=6  4.30 (Range 4-5)  MODULE 7  n=4  4.75 (Range 4-5)  MEAN  4.53  Quality of Facilitator Training and Program Manual Dane and Schneider (1998) cited the quality of facilitator training and program manuals to be important influences on implementation integrity, specifically in terms of promoting implementation integrity.  Given the importance of these components, they recommend  investigation into these two areas as part of program evaluations. Facilitator training and the program manuals are described and informally evaluated in the following paragraphs. Facilitators for the program were comprised of U B C Trek students and volunteer adult members of the community (Wisdom Champions). The U B C Trek students were typically new to the program for each facilitation of the program, although the Wisdom Champions may have  been involved in previous facilitations of the program. The facilitators for each program site remained consistent for the duration of the program. Prior to the W T M L program's commencement, program facilitators completed 26 hours of training. The training focused on an analysis of the content of the curriculum (including demonstrations of the facilitation of activities by the program developers), facilitation training, cultural diversity and special needs training (provided by the VSB), community service learning and reflection instruction, and team building and planning for program activities with the high school mentors and Wisdom Champions. Shelley - I'm not sure what more to add re: facilitator training...should I ask the Y W C A for more information? A l l individuals involved in facilitative components of W T M L also received a training manual ( Y W C A , n.d.).  The training manual addressed the overarching philosophies of the  Y W C A and the program, cited research that supports the need for programming for girls during the transition to high school, provided the rationale for the development of the program and the chosen topics, and gave descriptions of each module and activity. A sufficient explanation was provided for the theoretical framework and the Search Institute's community assets research that guided the goals of the program, module topics, and activities. For example, the overall program was created to enhance the Grade 7 girls' perceived positive assets as defined by the Search Institute assets. Particular module topics were chosen to address categories of assets and areas that local research has found to affect girls in Grade 7. From there, specific module activities aimed to improve specific assets within the broader category assets of the module.  The  descriptions for each module and the activities that comprised it were consistently detailed, with observable steps, topic guidelines, and time recommendaUons clearly and straightforwardly established. It is the program evaluator's view that the manual would be a helpful resource for  the facihtators to refer back to as they proceed with facilitating the program as it is highly readable and explicit.  CHAPTER 5 Discussion The field of social-emotional programming has gained a higher profile in recent years with the increasing documentation of its importance and successful outcomes in preventing risk factors and enhancing protective factors.  The development of the field has no doubt been  strengthened by the increased research scrutiny to which these programs have been exposed. The transition from elementary school to high school is one that is difficult for many students.  As summarized in previous chapters, students are at risk for decreases in self-  esteem/self-concept, academic functioning, high risk behaviour (e.g., alcohol and tobacco use) and peer relations.  The W T M L program was developed to prepare Grade 7 girls for the  transition from elementary school to high school. The present study explored the effect of the WTML  program participants'  comparison subjects.  self-esteem/self-concept  and perceived assets, relative to  Although results indicated statistically non-significant effects of the  program for both self-concept/self-esteem and for perceived developmental assets, some small changes in particular areas of self-esteem, self-concept, and perceived assets were observed. Additionally, participants reported high levels of satisfaction with the program. Facilitator reports of implementation integrity and treatment acceptability also suggested promising results to support the continued use and evaluation of the program. A summary of these findings is presented below, followed by a discussion of the limitations of the study, exploration of the evaluation process, and recommendations for future research. Outcome Evaluation Self-Esteem and Self-Concept. The results of the present evaluation indicated that the effect of the W T M L program was not statistically significant for any of the self-concept/self-  esteem domains assessed. However, increases of over 1 % in variance were observed as a result of program participation in the areas of Emotional Stability, Opposite-Sex Relations, and Physical Abilities. This finding suggests that although levels of statistical significance were not obtained, possibly due to study limitations which are discussed in subsequent paragraphs, the program led to potentially meaningful increases in variance in some areas of the participants' self-concept. Further evaluation efforts, with a larger sample, are warranted. Findings for Program Effects on Perceived Positive Assets. Results of statistical analyses indicated that the effects of program participation were not statistically significant for reported positive assets. Although not statistically significant, overall perceived internal assets and positive identity increased in variance 2.3% and 4.1%, respectively, as a result of program participation. Across specific contexts, increases of over 1% of variance were also observed for personal and social context areas. These context areas, as opposed to community and school contexts, may have seen greater improvements in perceived assets as they are more consistent with the topics and goals of the W T M L program (e.g., friendship skills, peer and parental relations, self-esteem, development of personal values). Contrary to expectations, results of the present outcome evaluation indicate that the W T M L program had no significant impact on student self-perceptions nor their perceived assets.. However, further evaluation of this program may be warranted, given the limitations of the present evaluation, namely its small sample size and the lack of random assignment to group conditions. A small sample size can make it difficult to detect statistically significant changes if they are small or moderate, whereas with larger samples, small but consistent changes are more likely to be detected.  In addition, the imbalance of groups in the sample, which consisted of  more comparison than program participants, is another barrier to detecting significant results.  The short period of time between the completion of the pre-post questionnaires may also have contributed to individuals scoring similarly across time. First, perhaps not enough time passed for significant actual change to occur.  Second, girls may have remembered how they had  previously answered questions and purposely answered the same way in an effort to maintain consistency.  Third, the outcome measures examined in the present study, although highly  relevant to the program content, may be tapping constructs that are more difficult to change in a short period of time. Finally, it is also possible that the true impact of the W T M L program will be evident as students move into and through the transition period, when they attempt to make use of the skills and supports provided by W T M L . Again, further outcome evaluation efforts are warranted in order to fully evaluate the impact of W T M L . Participant Satisfaction. Program participants across the four program sites answered three questions following the final module regarding their satisfaction with the program. Specifically, if they would recommend it to a friend, if they felt more prepared to transition to high school as a result of W T M L , and in what ways participating in the program helped them. Reports indicated that the vast majority of the participants enjoyed the program and felt it made a meaningful contribution towards preparing them for the transition to high school. This finding suggests that changes and improvements may have occurred in the program participants in a manner not detected by the outcome assessment measures due to either the constructs that they measured or the sampling limitations in the study. Li any case, this level of reported enjoyment and perceived benefit from program participation support its continued use (and evaluation) of the W T M L program. Liiplementation Integrity and Treatment Acceptability. Implementation integrity, or the degree to which the program was administered faithfully, and treatment acceptability, or the  degree to which the program was viewed by facilitators to be fair, feasible, and likely to succeed, were assessed through facilitator responses to questions following each module. The time that facilitators spent preparing to facilitate modules independently and with their co-facilitator was assessed. Although not statistically analyzed, the data suggested a few trends.  First, facilitators reported spending more time preparing initial modules than latter  modules. This may be a result of having increased comfort and experience with the program, hence requiring less time to prepare for the module. Second, facilitators typically reported spending more time preparing for modules independently than preparing with a co-facilitator. This may have resulted from the increased convenience of independent preparation, rather than having to coordinate and plan meeting times to prepare with a co-facilitator.  Third, the  facilitators reported spending less time preparing for facilitating the modules than the program developers had intended. This may be due to facilitators being able to sufficiently prepare for modules in less time than expected or facilitators not putting adequate time into preparation. Additional questions regarding preparedness to facilitate modules based on their skills and the training they received indicated that facilitators generally felt apdy prepared to facilitate the modules, which would lend support to the first reason. Implementation integrity was generally found to be strong across program sites for module-specific activities, routine activities, and themes that were to be incorporated into each module. This may have been due to the implementation promotion strategies, like the high quality of facilitator training and program manual, which have been found to promote implementation integrity (Dane & Schneider, 1998) and were assessed to be strong in this evaluation.  There were two specific program components for which there was more variable or infrequent adherence: use of the choice stone and emphasis of the importance and role of adults in the participants' lives.  During the evaluation, reports indicated variable, but generally  infrequent use of the choice stone across program sites. Potential reasons for its under use may be confusion regarding its purpose, facilitators choosing other methods for handling misbehaviour situations or situations not arising where it was appropriate to use it.  The  incorporation of the importance of adults in the participants' lives into modules was variable across schools. Such variability could be attributed to a number of factors. Perhaps at some schools there was less opportunity to discuss the issue, hi other cases, it may have been more difficult for particular facilitators to incorporate this component into the modules without explicit directions because it required one to "think on their feet". It is important to note that although reports of implementation integrity from the program facilitators were strong, there are limitations to this method of assessing implementation. Selfreports of implementation, as opposed to more objective measures of implementation (e.g., an outside observer complete implementation checklist), are considered less robust as they are more prone to inaccuracies in ratings. For example, facilitators may be concerned that reporting poor implementation will reflect poorly on them and lead them to rate implementation as being stronger than it was.  In other circumstances, facilitators may have variable memory or  awareness of the degree to which components were administered faithfully. On questions which addressed treatment acceptability, facilitators reported supporting and valuing the program's goals and activities. They also reported feeling prepared to facilitate the modules based on their skill level and the training they received. Facilitators also reported perceiving the modules to be enjoyed and engaging for the participants.  It is important to remember that subjective  impressions of a program's effectiveness are a necessary but not sufficient component of a program's success. As previously mentioned, the D A R E program has been viewed as successful by many stakeholders, although the program has not been found through objective measures to reduce drug use (Berg, 1997; Curtis, 1999; Donnermeyer et al., 1997). Thus, it is important for facilitators to support a program, but as an evaluative tool, it is no guarantee of program impact. The Evaluation Process Much research has investigated the conditions that enable and hinder the successful implementation of school-based social-emotional programs (i.e., treatment acceptability, principal and teacher support).  Less research, however, has been reported in the social-  emotional literature on the factors that facilitate the successful evaluation of such programs. One could logically hypothesize that the conditions that facilitate or inhibit the implementation of a program would be similar to the ones that facilitate or inhibit the evaluation of it. hi the paragraphs that follow, strengths and limitations of the current study, as well as the evaluative process in general, are discussed. Recommendations for future research as well as suggestions for conducting researcher-community partnerships are addressed. Differing Views On Research Between Stakeholders.  Partnerships between multiple  stakeholders have the potential for many significant benefits, which may exceed individual, uncoordinated, and separate efforts targeted at the same goal. Indeed, in the present evaluation study, the collaboration between researchers, schools and program developers allowed for an independent evaluation of the W T M L program and made possible the present Masters thesis research. Still, partnerships between researchers and practitioner agencies do not exist without challenges and require strong communication and sometimes significant compromise among stakeholders. The present study was no different in this regard. Across stakeholders, there were  variable ideas and points of view regarding what should constitute research and program evaluation. In the paragraphs below, several of these issues are discussed. What is program evaluation? A significant roadblock that arose in the current evaluation project was that of competing agendas across stakeholders and varying ideas of what components and standards constituted a competent program evaluation. In terms of research designs, university researchers and research organizations typically employ scientifically rigorous research designs, with emphasis on such things as the statistical strength of randomization, control groups, pre- and post-test comparisons, implementation integrity, and treatment acceptability, and with a clear recognition of the limitations to validity and reliability that the absence of these components create. In this evaluation, the program developers and school principals did not fully share the researchers' understanding of the importance of ensuring these factors were maintained and how a failure to do so could significantly weaken an evaluation. Rather, our community and school partners were particularly interested in consumer satisfaction with the program (e.g., what did the students and facilitators like about the program) and expressed concerns regarding requests from researchers to form appropriate comparison or control groups who would complete pre-/postsurveys but not participate in the W T M L program. As noted in Chapter 1, investigation into the experiences of the comparison group is an important component in program evaluation to increase the interpretability of findings. Both program developers and school administrators were hesitant about approaching students and parents of students who were not participating in the program to complete pre- and post-program surveys, viewing such efforts as outside of their role and territory and not likely to be successful. Accordingly, and despite the provision of incentives to encourage non-program-participants to take part in the evaluation, little promotion  of such efforts was undertaken at the school level. Future collaborative efforts of this sort may benefit from initial discussions and clarifications regarding foci and expectations in conducting such evaluation research, with each group benefiting from a fuller understanding of the goals and necessary components or foci of the other. Despite some discomfort about the formation a comparison group, the final sample included few subjects who participated in the W T M L program who also received parent permission to participate in the study, and many more girls who did not participate in the W T M L program. The small sample size, with few subjects in the "treatment" group, seriously hampered the likelihood of detecting differences between experimental and control groups on questions of interest. Although the creation of the treatment and comparison groups through a self-selection process is consistent with how the program is "typically" facilitated in a naturalistic setting, future evaluations of the W T M L program would be stronger and more experimentally rigorous if conducted with a randomized control or "wait list" design, as described previously, and with a larger sample size. At the same time, it is important to underscore the mutual benefits of such collaborative efforts and the gradual processes through which both partners grew in the partnership. Indeed, in the present project, there were several instances in which consensus was reached regarding aspects of research design despite initially differing views. For example, the idea of pre-testing was not initially seen as being crucial to the evaluation by program developers, although this component was viewed by the evaluator as being critical given the lack of random assignment in the sample. Through discussions between the program developers and evaluators, both parties came to an agreement that appropriate pre-testing was important and, eventually, it was built into the evaluation.  In some cases, although dates were established with partners for completion of pre-test surveys prior to the start of the W T M L program, these dates were not always honoured. At one school, for instance, actual testing was cancelled with the expectation that such testing could be completed after the program had begun.  Similarly, the need to distribute and later collect  returned parent permission forms prior to the start of testing or the commencement of the program itself was not always fully appreciated by school partners. A related area of difficulty arose regarding differing expectations for the requirements, steps and time lines necessary for the research process. When conducting research on their own, the program developers would typically be able to begin the evaluation process immediately by having facilitators or participants of the programs complete questionnaires. For research with in the university setting, the research process also includes the additional and sometimes lengthy preliminary and requisite step of securing ethical approval.  This component impacted the  expected timelines that the different partners entered the partnership with. Through ongoing partnerships, over a longer period of time or across multiple projects, both school and community partners become increasingly aware of the processes established by research institutions for the completion of ethical research with underage subjects. Differing opinions among stakeholders were also evident regarding the type of measures that should be used to evaluate a program.  Program evaluators favoured the use of  psychometrically sound, objective, and standardized instruments, which measure constructs that would theoretically be impacted by the program. Alternatively, program developers viewed open-ended questions to the program facilitators and participants about their enjoyment of the program activities to be more meaningful for evaluating the program's success.  Despite  discussions of how the use of objective measures would increase the generalizability of findings  and how it is difficult to gamer robust findings from open-ended survey questions, both parties' opinions remained generally intact throughout the process, owing to a large extent to their differing views on the purpose of the evaluation. Whereas the program evaluators focused on the idea that evaluation should address the effectiveness of the existing program in its current state, program developers, understandably, emphasized the need for evaluation to serve the purpose of program development and improvement, as well as gauge participant enjoyment of the program. Although both are valid and important goals, coming to an acceptable integration of the two was a challenge in the present evaluation. In order to honour the needs of both groups, program developers administered their previously developed surveys to facilitators and participants to meet their goals and evaluative needs, and program evaluators conducted a quasi-experimental evaluation of the outcomes of the program. These issues highlight the discrepancy that often arises in community-university partnerships and community-based research projects where theoretically optimal conditions are not always readily available or feasible for practice in the "real world". Indeed, the Society for Prevention Science (2004) underscores the need to evaluate both efficacy (impact when the program is offered under relatively ideal circumstances), and effectiveness (impact when the program is implemented in real world settings).  Although incongruencies between the  "experimental" and "real world" research arise which require researchers to be flexible and make compromises when conducting research in natural settings, researchers must determine in which areas these accommodations may be made and to what degree they can be permitted without significantly compromising the integrity of the  findings.  For example, a researcher may  appreciate that in some circumstances having a control group is less important than having all members of a target group have access to an intervenfion. However, how that decision limits  potential research findings needs to be acknowledged and partners may be able to use more creative research designs and evaluation techniques to decrease of the impact of the missing component. Essentially, all parties, regardless of research background or knowledge, seek to maximize the likelihood of scientifically sound research outcomes, and to minimize effort, resistance, and inconvenience to participants. One of the most important outcomes of ongoing community/school-university research partnerships is the opportunity to work together to achieve these goals through a growing mutual understanding of differing priorities, foci and needs. Communication Among Stakeholders, hi the present evaluation project, difficulties arose in balancing clear channels of communication so that all parties were kept informed of changes and so that an appropriate distance was maintained for the purpose of researcher objectivity. Additionally, when there are multiple stakeholders participating on a project, it can be difficult to ensure that all parties are kept up to date with changes and new information. Two examples were particularly illustrative of this issue. In one case, the program evaluator contacted the principals of the schools where the program was to be conducted based on the list provided by the program developers, only to learn that two of the schools had opted to not participate in the W T M L program at that time.  A second incident involved a reorganization of the program  modules by the program developers. Because the program evaluator was not informed of these changes, the implementation integrity forms to be filled out by the program facilitators following each module did not reflect the most current version of the program. These particular examples reflect difficulties that did not seem to arise from negative intentions.  Rather, program  developers were simply not used to having to inform others outside of the agency of changes to the program. Likewise, the program developers likely felt at times that they were not fully aware  of the progress being made in securing ethical approval and the interactions between the schools and the program evaluator. The lack of clear lines of communication also contributed to difficulties between the program evaluator and school principals. Due to the demanding schedules of school principals, it was often very difficult for the program evaluator to reach and coordinate meetings with the principals. For example, it took over a month to reach one school principal. Many principals expressed frustration and feelings of being overburdened with the tasks placed on them by the program developers and the program evaluator, commenting on the fact that they had not been aware of the effort that would be required on their part to have the program and evaluation facilitated at their schools. Some school principals had views of research that differed from that of the evaluator. For example, on a few occasions data collection had to be rescheduled because the schools cancelled the day before the questionnaires were to be completed. In one school, after two last minute cancellations (e.g., because there were prior scheduled school activities occurring at the same time) and rescheduling, it was determined that the pre-test questionnaires could not be completed before the program was to start and therefore the school was unable to participate in the outcome assessment evaluation. In this case, the school administrator did not view it as problematic to administer the surveys after the program had started. Given the short length of the program, the evaluator did not feel that it would be suitable to collect data after the program had started. At another school, despite ongoing conversations with the school principal, the evaluator was not informed that consent forms were not being returned for the evaluation until the day before pre-test data were to be collected, leaving no opportunity to employ additional strategies  to encourage the Grade 7 girls to return the forms. At this school, girls were also not enrolling for the W T M L program. To address this issue, the program developers and the school principal scheduled the first module of the program during school hours, rather than after school, in order to have all of the Grade 7 girls participate in the program with the hopes that many of the girls would enjoy the experience and then choose to continue participation. Unfortunately, the presentation was scheduled during the time that the girls were to complete the pre-test questionnaires, and the evaluator was not informed of the change until the day before, at which point there was no other appropriate date to reschedule the data collection prior to the program commencement. It appeared that many of the principals did not fully understand the differing roles of the program developers and evaluators and therefore believed that if changes were made or information was provided to one, it was conveyed to the other, which was not always the case. Through the process of the evaluation, communication improved as parties gained a better understanding of the entire process and their roles in it. In the future, it would likely be beneficial for the researcher and program developers to attend meetings together with the school, rather than separately. In addition, it may be advisable for program and evaluation forms to be handed out to students at the same time. These strategies may help clearly define the roles for school contacts. Variable School Support. Across the school sites, there was varying support for the W T M L program and the evaluation of it.  At one extreme, one principal expressed strong  support for the program and the benefit it would have on the Grade 7 girls. Not surprisingly, this principal was more accommodating and supportive of the evaluative process, allowing greater flexibihty around testing days and times, honouring time lines and meeting commitments, and actively encouraging Grade 7 girls to participate in the W T M L program as well as the evaluation  project.  At other schools, there was greater resistance to both the W T M L program and  evaluation as expressed to the program developers and the evaluator. Although this was not assessed formally or explicitly, these factors likely impacted the program and the evaluation. As described in Chapter 1, principal support (Elliot, 1998; Kam et al., 2003) for a program, a good fit between a program and a school or culture (Dane & Schneider, 1998; Smith et al., 2004), and readiness to change (Elias, Zins, Graczyk, & Weissberg, 2003) are associated with implementation integrity and successful outcomes. Employee Tumover. Employee turnover has been cited as leading to difficulties in the faciUtation of school-based social-emotional programming (Elias, Zins, Graczyk, & Weissberg, 2003). Given the number of different stakeholders involved in the W T M L program, this issue was particularly relevant to the current evaluation. In particular, the W T M L program facilitators included volunteer U B C Trek students and community members ("Wisdom Champions") who were predominantly new to the program at each administration (i.e., each academic term at UBC). The inherent problem with this situation was that new facilitators lacked experience with the W T M L program specifically and potentially program facilitation in general, had variable levels of experience working with children, and possibly differentially supported the program itself A l l of these factors make it likely that the facilitation of the program would be less fluent than if it was presented by facilitators who had previously administered the modules. Given that the W T M L program is relatively short (6-8 sessions) and each session was typically being presented for the first time by a facilitator, this may hinder the impact of the program itself on the participants given the learning curve before facilitators were proficient at facilitating the modules.  As well, there was a significant degree of turnover among the Y W C A staff who developed the W T M L program during the course of the evaluation due to unforeseen circumstances and temporary contract positions. This impacted communicative channels and also decreased the shared knowledge base among the evaluator and program developers. For example, although a protocol and timeline had been developed for approaching the schools regarding evaluation project, these procedures were not always adequately communicated or followed when passed on to a subsequent employee. Finally, in an effort to provide the W T M L program to as many students as possible, the program was administered at a number of different and typically new schools each term, each with unique concerns and different levels of investment in the program.  Although this is  obviously consistent with the program goals of dispensing transition preparation materials to as many Grade 7 classroom as possible, it creates obstacles to facilitating and evaluating the program as each new administrator needs to become familiar, not only with the program itself, but also with the demands of evaluation procedures. As a first time experience in community-university partnerships for both parties, there were predictably unexpected roadblocks for which neither party was prepared. Expectedly, for each stakeholder there were instances when he/she felt inadequately informed about the activities of other stakeholders as those stakeholders performed what they felt to be their role in the process. However, as the partnership progressed and the parties gained a broader and deeper awareness of each other's role and fluency with the evaluation process, these roadblocks became increasingly predictable and as such, preventable. Over time, a greater appreciation developed among stakeholders for the differing needs and viewpoints of each party, in addition to the demands being placed on them. Having now completed the process, all the stakeholders would  likely now be better able to participate in more effective initial planning, collaboration, and consultation to allow the process to run more smoothly. Random Sampling And As Large A Sample As Possible. Researchers understand the statistical importance of a large sample size as " N " relates to power in statistical analysis. For other stakeholders, interventions may be viewed more simply as "working" or not. hideed, some stakeholders may not understand all the components that contribute to whether an intervention is deemed successful or unsuccessful and may fail to appreciate that appraisals of success may be conceptualized as existing on a continuum. For example, whether or not an intervention or phenomenon is viewed as being statistically significant is impacted by the size of the sample used in the research. In a large sample, small changes may be observed across subjects and because the changes are consistent across subjects, the changes may be deemed statistically significant.  In contrast, when a sample is small, changes need to be larger to be deemed  statistically significant. In the present evaluation, for example, one school principal re-scheduled pre-testing to the lunch hour, noting that by forcing students to give up their lunch hour to complete the questionnaire, the program evaluator would be assessing the students who were really committed to participating in the research, not the students who would misbehave or who were just doing it for the incentives. From a research point of view, feedback from all students would have been preferable and more representative.  Effective collaboration requires time and ongoing  communication between partners in order to understand competing needs and foci that can compromise the quality of the evaluation process. There may have also been selection biases that existed regarding the students who participated in the W T M L program. Although the program developers viewed the program as  one that should be available to all Grade 7 girls and especially for those at risk, one principal felt that the opportunity to participate in the program should be provided as a reward for students who had demonstrated good behaviour. The principal also felt that some of the higher risk students may make participating in the program less enjoyable for the better behaved students who had '"earned" the right to participate. This principal also suggested that many of the higher risk students had access to other resources in the school and community, so the W T M L program would be better utilized by the well-behaved students. Recommendations For Future Research and Contributions to the Research Literature The integration of research and practice and more specifically, researchers and community-based program developers and practitioners, requires a joint relationship based on the premise of mutual development and benefit. Such initiatives enable continued relationships in which each party learns from the other and are likely to yield greater advantages than fragmented efforts directed at the same goal. Community/school-university partnerships allow for research to continually inform practice, including ongoing improvement of the program. Ultimately, successful joint ventures between research organizations and community partners require compromise. This can be difficult as parties enter the relationship with their own values, beliefs, and expectations around social-emotional functioning, interventions, and evaluation, hi each partnership, the members must decide with which compromises they feel comfortable making. robust.  If researchers compromise too much, the outcomes of the evaluations will be less However, if researchers do not accept the limitations in the community to what is  feasible and available and refuse to make compromises, community partners may be less inclined to want to collaborate with research institutions.  Of primary importance is that these  partnerships continue, rather than divide, to narrow the gap between research and practice in the educational and social-emotional learning field. For subsequent evaluations of the W T M L , it would be advisable for the evaluator and program developers to approach schools and attend school meetings together so that the programming and evaluating efforts appear less fragmented. This may also aid in making the roles of program and evaluation staff more clear to school personnel. A second recommendation would be to have consistent stakeholders when possible. Therefore, involved persons would be familiar with the processes and procedures for the program and evaluation across program implementations and across time (e.g., same schools). In the case of this evaluation, having all new schools at each facilitation led principals to feel overwhelmed by their role and not have fluency with the procedures. As well, it would be beneficial during future evaluations to explore other comparison group options (e.g., waitlist, random assignment, alternative treatment group) and use a larger sample to strengthen the research design and increase the likelihood of yielding statistically significant results on outcome measures. Conclusion Program evaluation is a crucial component of any prevention or intervention program. There exists an ethical responsibility for such evaluations to occur, along with the current political  Zeitgeist  that is pushing for the adoption and dissemination of evidence-based programs.  As such, it is important in the early stages of program use and development to evaluate the effectiveness of a program. Strategies such as this may aid in narrowing the research-practice gap because evaluation is built into the development of the program from the initial stages and evaluation occurs immediately in the setting where it will be disseminated. Evaluation of the effectiveness of the W T M L program, conducted in partnership with the Vancouver School Board  and Y W C A , provided an initial look at the impact of the program on Grade 7 girls in the areas of self-esteem, self-concept, and perceived positive assets, as well as qualitative assessments participant satisfaction and facilitator treatment acceptibility and implementation integrity. Results of the present evaluation were mixed.  On one hand, reports of participant  satisfaction, implementation integrity, and treatment acceptability indicated that the program was well received and faithfully implemented as intended, providing some evidence for the face validity of the program. On the other hand, the present outcome evaluation failed to demonstrate statistically significant changes on measures of self-concept, self-esteem, or perceived positive attributes. Drawing definitive conclusions regarding the impact of the program should be cautioned, however, because a number of factors (e.g., sample size, unequal participant and comparison groups, lack of random assignment) may have limited the likelihood of yielding statistically significant results in the present study. Future studies should be directed to use the effect sizes derived in this study for power analyses to determine the sample size that would be needed to obtain statistically significant results to determine if sampling limitations led to the nonsignificant findings or if the program is not effective in those areas. Alternatively, focus groups may be conducted with former program participants to gain a further understanding of the areas that they believe they improved in as a result of the W T M L .  This will help determine if meaningful change is occurring in the areas  intended by program developers or if change is occurring in less predictable domains. These efforts will additionally aid in determining the appropriateness of the evaluation measures. Further evaluation efforts may consider the use of waitlist control groups or comparison with a control group receiving an alternative evidence based treatment.  Finally, and perhaps most  importantly, future research should be directed at examining the impact of the W T M L program  on how participants subsequently navigate that actual transition period, as this is where the program may be likely to evidence its greatest impact.  References  Aber, J.L., Jones, S.M., Brown, J.L., Chaudry, N . , Samples, F. (1998). Resolving conflict creatively: Evaluating developmental effects of a school-based violence prevention program in neighborhood and classroom context. Development and Psychopathology, 10, 187-213. Akos, P. & Galassi, J.P (2004). Middle and high school transitions as viewed by students, parents, and teachers. Professional School Counseling, 7,212-221. Allinder, R . M . & Oats, R.G. (1997). Effects of acceptability on teachers' implementation of curriculum-based measurement and student achievement in mathematics computation. Remedial and Special Education, 18,113-120. Barone, C , Aguirre-Deandreis, A. I., and Trickett, E. J. (1991). Means-end problem-solving skills, life stress, and social support as mediators of adjustment in the normative transition to high school. American Journal of Community Pscyhology, 19, 207-225. Battistich, V . (2000). The use of implementation data in assessing the effectiveness of the Child Development Project. Presented at the meeting of the Society for Prevention Research, June 2000, Montreal. Developmental Studies Center. B . C . Ministry of Education (2006.). British Columbia Ministry of Education Curriculum Integrated Resource Packages. Retrieved July 15, 2006, from http://www.bced.gov.bc.ca/irp/irp.htm. B.C. Ministry of Education. B.C. Education Performance Standards. Retrieved July 11, 2006, from http://www.bced.gov.bc.ca/perf stands/. Berg, K.F. (1997). Evaluation Report for the Hawai'i Department of Education Drug Abuse Resistance Education (DARE) Program for SY 94-95 and 95-96. Hawaii, U S A : University of Hawai'i, Curriculum Research & Development Group. Bemdt, T. J. & Keefe, K . (1995). Friends' influence on adolescents' adjustment to school. Child Development, 66, 1312-1329. Biglan, A., Mrazek, P.J., Camine, D., Flay, B.R. (2003). The integration of research and practice in the prevention of youth problem behaviours. American Psychologist, 58, 433-440. Brooks-Gunn, J. (2004). Don't throw out the baby with the bathwater: Incorporating behaviour research into evaluations. Social Policy Reports, Published by the Society for Research in Child Development (pp. 15-16). Brown, J.L., Roderick, T., Lantieri, L . , Aber, J.L. (2004). The Resolving Conflict Creatively Program: A school-based social and emotional learning program. In J.E. Zins, R.P  Weissberg, M . C . Wang, & H.J. Walberg (Eds.), Building Academic Success on Social and Emotional Learning. What Does the Research Say? (pp. 151-169). New York, N Y : Teachers College Press. Calhoun Research and Development & C. Lang Consulting (2005). Girls in Canada 2005. A report prepared for the Canadian Women's Foundation. Cameron, R., Manske, S., Brown, K.S., John, M.A., Murnaghan, D., & Lovato, C. Integrating public health policy, practice, evaluation, surveillance, and research: the school health action planning and evaluation system. American Journal Of Public Health, 97, 648-654. Cantin, S. & Boivin, M . (2004). Change and stability in children's social network and selfperceptions during transition from elementary to junior high school. International Journal of Behaioral Development, 28, 561-570. Carney, J. (2000). Bullied to death: Perceptions of peer abuse and suicidal behaviour during adolescence. School Psychology International 27(2), 213-223. Christenson, S.L. & Havsy, L.H.. (2004). Family-school-peer relationships: Significance for social, emotional, and academic learning. In J.E. Zins, R.P. Weissberg, M . C . Wang, & H.J. Walberg (Eds.), Building Academic Success on Social and Emotional Learning (pp. 59-75). New York, N Y : Teachers College Press. Christenson, S.L. & Peterson, C.J. (1998). Family, school, and community influences on children's learning: A literature review (Report No. 1). Parents are Teachers Project. Minneapolis: University of Minnesota Extension Service. Clayton, R. R., Cattarello, A . M . & Johnstone, B. M . (1996). The effectiveness of Drug Abuse Resistance Education (Project DARE): 5-year follow-up results. Preventive Medicine, 25, 307-318. Coie, J.D., Dodge, K.A., & Kupersmidt, J.B. (1990). Peer group behavior and social status. In S.R. Asher & J.D. Coie (Eds.), Peer Rejection In Childhood (pp. 17-59). New York: Cambridge University Press. Collaborative for Academic, Social, and Emotional Learning, The. (n.d.). SEL Competencies. Retrieved on July 20, 2006 from www.casel.org. Cook, T.D. (2004). Beyond advocacy: Putting history and research on research into debates about the merits of social experiments. Social Policy Reports, Published by the Society for Research in Child Development (pp.5-6). Cottingham, P. (2004). Why we need more, not fewer, gold standard evaluations. Social Policy Reports, Published by the Society for Research in Child Development (pp.13).  Curtis, C.K. (1999). The efficacy of the Drug Abuse Resistance Education Program (DARE_ in West Vancouver school. Part I, attitudes toward DARE: An examination of opinions, preferences, and perceptions of students, teachers, and parents. West Vancouver, B C : West Vancouver Police Department. (ERIC Document Reproduction Service No. ED440324). Datta, L. (2003). Avoiding unwarranted death by evaluation. The Evaluation Exchange, 9, 5. Dane, A.V., & Schneider, B . H . (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18 23-45. DeRosier, M.E., Kupersmidt, J.B., & Patterson, C.J. (1994). Children's academic and behavioral adjustment as a function of the chronicity and proximity of peer rejection. Child Development, 65, 1799-1813. Deihl, L . M . , Vicary, J.R., & Deike, R.C., (1997). Longitudinal trajectories of self-esteem from early to middle adolescence and related psychosocial variables among rural adolescents. Journal of Research on Adolescence, 7, 393-411. Donnermeyer, J. F. & Wurschmidt, T. N . (1997). Educators' perceptions of the D.A.R.E. program. Journal of Drug Education, 27, 259-276. Duchesne, S., Larose, S., Guay, F., Vitaro, F., & Tremblay, R.E, (2005). The transition from elementary to high school: The pivotal role of mother and child characteristics in explaining trajectories of academic functioning. International Journal of Behavioral Development, 29, 409-417. Drug Abuse Resistance Education (n.d.). www.dare.com.  D.A.R.E. Retrieved December 10, 2007, from  Dusenbury, L., Falco, M . , & Lake, A . (1997). A review of the evaluation of 47 drug abuse prevention curricula available nationally. Journal of School Health, 67, 127-132. Dusenbury, L . & Falco, M . (1995). Eleven components of effective drug abuse prevention curricula. Journal of School Health, 65, 420-425. Dukes, R.L. Stein, J.A., & Ullman, J.B. (1997). Long-term impacts of Drug Abuse Resistance Education (D.A.R.E.): Results of a 6-year follow-up. Evaluation Review, 21, 483-500. Egan, S. K. & Perry, D. G. (1998). Does low self-regard invite victimization? Developmental Psychology, 34, 299-309. Elias, M . (2004). Strategies to infuse social and emotional learning into academics. In J.E. Zins, R.P. Weissberg, M . C . Wang, & H.J. Walberg (Eds.), Building Academic Success on Social and Emotional Learning (pp. 113-134.). New York, N Y : Teachers College Press.  Elias, M.J., Zins, J.E., Graczyk, P.A., Weissberg, R.P. (2003). Implementation, sustainability, and scaling up of social-emotional and academic innovation in public schools. School Psychology Review, 32, 303-319. Elliott, D. (1998). Blueprints for violence prevention. Golden, CO: Venture. ElUott, S.N., Witt, J.C., & Kratochwill, T.R. (1991). Selecting, implementing, and evaluating classroom interventions. In G . Stoner, M.R. Shinn, & H . M . Walker (Eds.), Interventions for achievement and behavior problems (pp. 99-135). Silver Spring, M D : National Association of School Psychologists. Ennett, S. T., Tobler, N . S., Ringwalt, C. L . & Flewelling, R. L . (1994). How effective is drug abuse resistance education? A meta-analysis of Project D A R E outcome evaluations. American Journal of Public Health, 84, 1394-1401. Eslea, M . , Menesini, E., Morita, Y., O'Moore, M . , Mora-Merchan, J., Pereira, B., & Smith, P. (2003). Friendship and loneUness among bullies and victims: Data from seven countries. Aggressive Behaviour, 30, 71-83. Ethier, K.A., Kershaw, T.S., Lewis, J.B., Milan, S., Niccolai, L . M . , & Ickovics, J.R. (2006). Self-esteem, emotional distress and sexual behavior among adolescent females: Interrelationships and temporal effects. Journal of Adolescent Health, 38, 268-274. Field A. (2005). Discovering Statistics Using SPSS (2nd Edition). London: S A G E Publications Ltd. Floyd, C. (1997). Achieving despite the odds: A study of resilience among a group of African American high school seniors. Journal of Negro Education, 65, 181-189. Fonagy, P., Twemlow, S., Vemberg, E., Sacco, F.C., & Little, T.D. (2005). Creating a peaceful school learning environment: The impact of an anti-buUying program on educational attainment in elementary school. Medical Science Monitor, 11(1), 317-325. Forgatch, M.S. (2003). Implementation as a second stage in prevention research. Prevention & Treatment, 6. Goodenow, C. (1993). The psychological sense of school membership among adolescents: Scale development and educational correlates. Psychology in the Schools, 30, 79-90. Greenberg, M.T., Domitrovich, C.E., Graczyk, P.A., Zins, J.E. (2005). The Study of Implementation in School-Based Preventive Interventions: Theory, Research, and Practice (Volume 3). Rockville, M D : Center for Mental Health Services, Substance Abuse and Mental Health Services Administration.  Greenberg, M.T., Kusche, C.A., & Riggs, N . (2004). The PATHS curriculum: Theory and research on neurocognitive development and school success. In J.E. Zins, R.P. Weissberg, M.C. Wang, & H.J. Walberg (Eds.), Building Academic Success on Social and Emotional Learning (pp. 170-188). New York, N Y : Teachers College Press. Greenberg, M.T., Weissberg, R.P., O'Brien, M . U . , Zins, J.E., Fredericks, L., Resnik, H . , Ellas, M.J. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58, 466474. Haney, P. & Durlak, J.A. (1998). Changing self-esteem in children and adolescents: A metaanalytic review. Journal of Clinical Child Psychology, 27, 423-433. Hawkins, J.D., Smith, B.H., & Catalano, R.F. (2004). Social development and social and emotional learning. In J.E. Zins, R.P. Weissberg, M . C . Wang, & H.J. Walberg (Eds.), Building Academic Success on Social and Emotional Learning (pp. 135-150). New York, N Y : Teachers College Press. Hazier, R.J. (1996). Breaking the Cycle of Violence: Interventions for Bullying and Victimization. Washington, DC: Accelerated Development. Heider, F. (1958). The psychology of interpersonal relations. New York: Wiley. Herrmann, D.S., McWhirter, J.J., & Sipsas-Herrmann, A . (1997). The relationship between dimensional self-concept and juvenile gang involvement: Implications for prevention, intervention, and court referred diversion programs. Behavioral Sciences and the Law, 15, 181-194. Hillegers, M.H.J., Burger, H., Wals, M . , Reichart, C.G., Verhulst, F.C. Nolen, W.A., & Ormel, J. (2004). Impact of stressful life events, familial loading and their interaction on the onset of mood disorders. British Journal of Psychiatry, 755,97-101. Hymel, S., Schonert-Reichl, K . & Miller, L . (2006). Hymel, S., Schonert-Reichl, K.A., & Miller, L.D. (2006). Reading, 'Riting, 'Rithmetic and Relationships: Considering the Social Side of Education. Exceptionality Education Canada, 16 (3), 149-192. Illinois Board of Eduction (n.d.). Illinois learning Standards. Retrieved July 15, 2006, from http://www.isbe.state.il.us/ils/social emotional/standards.htm. Isakson, K., & Jarvis. (1999). The adjustment of adolescents during the transition into high school: A short-term longitudinal study. Journal of Youth and Adolescence, 28, 1-26. Jones, M.L., Eyberg, S.M., Adams, C D . , & Boggs, S.R. (1998). Treatment acceptability of behavioral interventions for children: A n assessment by mothers of children with disruptive behavior disorders.  Kallestad, J. H . & Olweus, D. (2003). Predicting teachers' and schools' implementation of the Olweus Bullying Prevention Program: A multilevel study Prevention & Treatment, 6. Kam, C , Greenberg, M.T., & Walls, C.T., (2003). Examining the role of implementation quahty in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55-63. Kanof, M . E. (2003). Youth Illicit Drug Use Prevention: DARE Long-Term Evaluations and Federal Ejforts to Identify Effective Programs. Washington, D C : General Accounting Office. Kaplan, H. B . (1980). Deviant behavior in defense of self. New York: Academic Press. Khanlou, N (2004). hifluence on adolescent self-esteem, in multicultural Canadian secondary schools. Public Health Nursing, 21, 404-411. Kling, K.C., Hyde, J.S., Showers, C.J., & Buswell, B . N . (1999). Gender differences in self esteem: A meta-analysis. Psychological Bulletin, 125, 470-500. Krain, A . L . , Kendall, P.D., & Power, T.J. (2005). The role of treatment acceptability in the initiation of treatment for A D H D . Journal of Attention Disorders, 9, 425-434. Kyler, S.J., Bumbarger, B.K., & Greenberg, M . T . (2005). Technical assistance fact sheets: Evidence-based programs. Philidelphia, P A : Prevention Research Center for the Promotion of Human Development. Lameiras, P . M . & Rodriguez, C . Y . (2003). Age and sex differences in self-esteem among Spanish adolescents. Psychological Reports, 93, 876-878. Lord, S.E., Eccles, J.S., & McCarthy, K . A . (1994). Surviving the junior high school transition: Family processes and self-perceptions as protective and risk factors. Journal of Early Adolescence, 74,162-199. Lynam, D.R., Milich, R., Zimmerman, R., Novak, S.P., Logan, T.K., Martin, C , Leukefeld, C , & Clayton, R. (1999). Project D A R E : No effects at 10-year follow-up. Journal of Consulting and Clinical Psychology, 67, 590-593. McCall, R.B. & Green, B.L. (2004). Beyond the methodological gold standards of behavioral research: Considerations for practice and policy. Social Policy Reports, Published by the Society for Research in Child Development (pp.3-29). McDougall, P. & Hymel, S. (1998). Moving into middle school: Individual differences in the transition experience. Canadian Journal of Behavioural Science, 30, 108-120. McDougall, P., Hymel, S., Vaillancourt, T., & Mercer, L . (2001). The consequences of early childhood rejection. In M . Leary (Ed.) Interpersonal Rejection (pp. 213-247). New York: Oxford University Press.  McVey, G.L., Davis, R., Tweed, S., & Shaw, B.F. (2004). Evaluation of a school-based program designed to improve body image satisfaction, global self-esteem, and eating attitudes and behaviours: A replication study. International Journal of Eating Disorders, 36, l - l 1. Marsh, H.W., Parada, R.H., Yeung, A.S., & Healey, J. (2001). Aggressive school troublemakers and victims: A longitudinal model examining the pivotal role of self-concept. Journal of Educational Psychology, 93, 411-419. Marsh, H.W. (1990a) Self Description Questionnaire - II Manual and research monograph. San Antonio, T X : Psychological Corp. Marsh, H . W. (1990b). A multidimensional, hierarchical model of self-concept: Theoretical and empirical justification. Educational Psychology Review, 2, 71-77. Marsh, H.W. (1989). Age and sex effects in multiple dimensions of self-concept: Preadolescence to early-adulthood. Journal of Educational Psychology, Si,417-430. Martens, B.K., Witt, J.C., EUiott, S.N., & Darveaux, D. (1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16, 191-198. Metalsky, G. I., Joiner, T. E., Hardin, T. S. & Abramson, L . Y . (1993). Depressive reactions to failure in a naturalistic setting: A test of the hopelessness and self-esteem theories of depression. Journal of Abnormal Psychology, 102, 101-109. Miller, K.S., Forehand, R . M . , & Kotchick, B . A . (2000). Adolescent sexual behavior in two ethnic minority samples: a multi-system perspective. Adolescence, 35, 313-333. Millsap, M . A., Chase, A . , Obeidallah, D., Perez-Smith, A., Brigham, N . , & Johnston, K. (2000). Evaluation of Detroit's Comer Schools and Families Initiative. Cambridge, M A : Abt Associates. Mook, D. G. (1983). In defense of extemal invalidity. American Psychologist, 38, 379 - 387. Mohr, L . B . (1995). Impact analysis for program evaluation (2nd ed.). Thousand Oaks, C A : Sage Publications, Inc. MuUis, R.L. & Chapman, P. (2000). Age, gender, and self-esteem differences in adolescent coping styles. Journal of Social Psychology, 140, 539-541. O'Connell, P., Pepler, D.J., & Craig, W . M . (1999). Peer involvement in bullying: Insights and challenges for intervention. Jounal of Adolescence, 22437-452. O'Connell, P., Sedighdeilami, F., Pepler, D., Craig, W., Connolly, J., Atlas, R., Smith, C , & Charach, A . (1997). Prevalence of bullying and victimization among Canadian  elementary and middle school children. Elementary and Early Childhood Education, April, 1-12. O'Dea, J.A. & Abraham, S. (2000). Improving the body image, eating attitudes, and behaviours of young male and female adolescents: A new educational approach that focuses on selfesteem. International Journal of Eating Disorders, 28, 43-57. Olweus, D. (2004). The Olweus Bullying Prevention Programme: Design and implementation issues and a new national initiative in Norway. In P.K. Smith, D. Pepler, & K . Rigby (Eds.), Bullying in Schools: How Successful Can Interventions Be? (pp. 13-36). Cambridge, U K : Cambridge University Press. Parker, & Asher, (1987). Peer relations and later personal adjustment: Are low-accepted children at risk? Psychological Bulletin, 102, 357-389. Pellegrini, A.D., Bartini, M . , & Brooks, F. (1999). School bullies, vitctims, and aggressive victims: Factors relating top group affiliation and victimization in early adolescence. Journal of Educational Psychology, 91, 216-224. Pellegrini, A . D . & Long, J. D. (2002). A longitudinal study of bullying, dominance, and victimization during the transition from primary school through secondary school. .British Journal of Developmental Psychology, 20, 259-280. Pratt, S. & George, R. (2005). Transferring friendship: Girls' and boys' friendships in the transition from primary to secondary school. Children & Society, 19, 16-263. Quatman, T. & Watson, C M . (2001). Gender differences in adolescent self-esteem: A n exploration of domains. Journal of Genetic Psychology, 162, 93-118. Resnick, M.D., Bearman, P.S., Blum, R.W., Bauman, K.E., Harris, K . M . , Jones, J., Tabor, J., Beuhring, T., Sieving, R.E., Shew, M . , freland, M . , Bearinger, L.H., & Udry, J. (1997). Protecting adolescents from harm: Findings from the national Longitudinal Study on adolescent health. Journal of the American Medical Association, 278, 823-832. Rigby, K., (2003). Consequences of bullying in schools. Canadian Journal of Psychiatry, 48{9), 583-590. Rigby, K. (1998). Peer relations at school and the health of adolescents. Youth Studies Australia, 17, 13-18. Rigby, K. (2000). Effects of peer victimization in schools and perceived social support on adolescent well-being. Journal of Adolescence, 23, 57-69. Rigby, K (1993). School children's perceptions of their families and parents as a function of peer relations. The Journal of Genetic Psychology, 154, 501-513.  Rigby, K., & Slee, P.T. (1991). Bullying among Australian school children: Reported behaviour and attitudes toward victims. Journal of Social Psychology, 131{5), 615-628. Roberts, D.F., Henriksen, L., & Foehr, U.G. (2004). Adolescents and media. In R.N. Lemer & L. Steinberg (Eds.), Handbook of Adolescent Development (pp. 487-521). Hoboken, NJ: John Wiley and Sons. Robins, R.W., Trzesniewski, K . H . , Tracy, J.L., Goshng, S.D., & Potter, J. (2002). Global selfesteem across the life span. Psychology and Aging, 17, 423-434. Robinson, N.S., Garber, J., & Hilsan, R. (1995). Cognitions and stress: Direct and moderating effects on depressive versus externalizing symptoms during the junior high school transition. Journal of Abnormal Psychology, 104, 453-463. Search Institute. Introduction To Assets. Retrieved on June 25, 2006 from http://www.searchinstitute.org/assets/. Search Institute (2004). Developmental Assets Profile Preliminary User Manual. Search Institute, Minneapolis, M N . Salmivalli, C , Kaukiainen, A., Voeten, M . , & Sinisammal, M . (2004). Targeting the group as a whole: the Finnish anti-buUying intervention. In P.K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in Schools: How Successful Can Interventions Be? (pp. 251-273). Cambridge, U K : Cambridge University Press. Salmivalli, C , Kaukiainen, A . , & Voeten, M . (2005). Anti-bullying intervention: Implementation and outcome. British Journal of Educational Psychology, 75, 465-^87. Selfridge, J. (2004). The Resolving Conflict Creatively program: How we know it works. Theory Into Practice, 43, 59-67. Shavelson, R.J., Hubner, JJ.J., & Stanton, G.C. (1976). Validation of construct interpretations. Review of Educational Research, 46, 407-441. Shinn, M . (2003). Understanding implementation of programs in multilevel systems. Prevention & Treatment, 6. Simmons, R.G. & Blyth, S.A. (1987). Moving into adolescence: The impact of pubertal change and school context. Hawthorn, N.Y.: Aldine de Gruyler. Simmons, R.G. & Blythe, D.A. (1987). The effect of type of school environment upon attitudes toward school and upon the self-image. In R.G. Simmons & D.A. Blyth (Eds.). Moving into Adolesdcence: The Impact of Pubertal Change and School Context, (pp. 207-228). Hawthorn, N Y : Aldine de Gruyler.  Simmons, R.G., Blyth, D.A., Van Cleave, E.F., & Bush, D . M . (1979). Entry into early adolescence: The impact of school structure, puberty, and early dating on self-esteem. American Sociological Review, 44, 948-967. Simmons, R.G., Burgeson, R., & Blyth, D. (1987). Cumulation of change, hi R.G. Simmons & D.A. Blj^h (Eds.). Moving into Adolesdcence: The Impact of Pubertal Change and School Context, (pp. 291-306). Hawthorn, N Y : Aldine de Gruyler. Simmons, R.G., Bush, D., & Blyth, D.A. (1987). Gender differences in late childhood and early adolescence. In R.G. Simmons & D.A. Blyth (Eds.). Moving into Adolesdcence: The Impact of Pubertal Change and School Context, (pp. 51-102). Hawthorn, N Y : Aldine de Gruyler. Simmons, R.G., Carlton-Ford, S.L., Blyth, D.A. (1987). Individual change and recovery: Selfesteem. In R.G. Simmons & D.A. Blyth (Eds.). Moving into Adolesdcence: The Impact of Pubertal Change and School Context, (pp. 259-274). Hawthorn, N Y : Aldine de Gruyler. Simmons, R.G., Carlton-Ford, S.L., Blyth, D.A. (1987). Adjustment to the junior high school transition: The effect of school properties and individual resources on self-esteem. In R.G. Simmons & D.A. Blyth (Eds.). Moving into Adolesdcence: The Impact of Pubertal Change and School Context, (pp. 307-329). Hawthorn, N Y : Aldine de Gruyler. Slavin, R.E., (2008). Perspectives on evidence-based research in education: What works? Issues in synthesizing educational program evaluations. Educational Researcher, 57 (1), 5-14. Smith, P.K., Pepler, D., & Rigby, K . (2004). Bullying in Schools: How Successful Can Interventions Be? Cambridge, U K : Cambridge University Press. Smith, J.D., Schneider, B., Smith, P.K., & Ananiadou, K . (2004). The effectiveness of wholeschool antibullying programs: A synthesis of evaluation research. School Psychology, 33, 547-560. Sperber, A . M . , Peleg, A . , Friger, M . , & Shvartzman, P. (2001). Factors associated with daily smoking among IsraeU Adolescents: A prospective cohort study with a 3-year follow-up. Preventative Medicine, 33, 73-81. Spreat, S. & Walsh, D.E. (1994). Impact of treatment efficacy and professional affihation on ratings of treatment acceptability. Mental Retardation, 32, 227-233. Society for Prevention Research. (2004). Standards of evidence: Criteria for efficacy, effectiveness, and dissemination. Retrieved July 20, 2006 from http://www.preventionresearch.org/StandardsofEvidencebook.pdf Sterk, C.E., Klein, H . , & Elifson, K.W. (2004). Self-esteem and "at risk" women: determinants and relevance to sexual and HFV-related risk behaviors. Women & Health, 40, 75-92.  Sugai, G. & Tindal, G . A . (1993). Effective School Consultation: An Interactive Approach. Pacific Grove. C A : Brooks/Cole. Telzrow, C F . & Beebe, J.J. (2002). Best practices in facilitating intervention adherence and integrity, hi A . Thomas & J. Grimes (Eds.), Best Practices in School Psychology IV (pp. 503-516). Bethesda, M D : National Association of School Psychologists. Thombs, D.L. (2002). A retrospective study of D A R E : Substantive effects not detected in undergraduates. Journal of Alcohol and Drug Education, 46, 27-40. Thorton, W., Douglas, G.A., & Houghton, S.J. (1999). Transition through stages of smoking: the effect of gender and self-concept on adolescent smoking behavior. Journal of Adolescent Health, 25, 284-289. Tonkin, R.S., Murphy, A., Lee, Z., Saewyc, E., & The McCreary Centre Society (2005). British Columbia youth health trends, a retrospective, 1992-2003. Vancouver, B C : The McCreary Centre Society. Towberman, D.B. & McDonald, R . M . (1993). Dimensions of adolescent self-concept associated with substance use. Journal of Drug Issues, 23, 525-533. U.S. Department of Education (2002). No Child Left Behind: A Desktop Reference. Prepared by the Office of the Under Secretary. Jessup, M D : Education Publications Center. von Sydow, K., Lieb, R., Pfister, H., Hofler, M . , & Wittchen, H . (2002). What predicts incident use of cannabis and progression to abuse and dependence? A 4-year prospective examination of risk factors in a community sample of adolescents and young adults. Drug and Alcohol Dependence, 68, 49-64. Vreeman, R . C , & Carroll, A.E. (2007). A systematic review of school-based interventions to prevent bullying. Archive of Pediatric Adolescent Medicine, 161, 78-88 Waas, G.A. & Anderson, G.P. (1991). Outcome expectancy and treatment acceptability: Perceptions of school-based interventions. Professional Psychology: Research and Practice, 22, 149-154. Watkins, D., Dong, Q., & Xia, Y. (1997). Age and gender differences in the self-esteem of Chinese children. Journal of Social Psychology, 137, 374-379. Weissberg, R. P., & Durlak, J. (2005, August). Social and emotional learning for school and life success. Invited address for the Society for Community Research and Action ( A P A Division 27) Distinguished Contribution to Theory and Research Award at the Annual Meeting of the American Psychological Association, Washington, DC  Wentzel, K.R., Barr, C , & Caldwell, K . A . (2004). Friendships in middle school: hifluences on motivation and school adjustment. Journal of Educational Psychology, 96, 195-203. Wentzel, K.R. & Caldwell, K . (1997). Friendships, peer acceptance, and group membership: Relations to academic achievement in middle school. Child Development, 68, 11981209. West, S.L. & O'Neal, K . K . (2004). Project D.A.R.E. outcome effectiveness revisited. American Public Health Association, 94, 1027-1029. Wysong, E., Aniskiewicz, R., & Wright, D. (1994). Truth and D.A.R.E.: Tracking drug education to graduation and as symbolic politics. Social Problems, 41, 448-472. Young Women Christian Association of Vancouver (n.d.). Welcome To My Life facilitatior manual. (Available from the Y W C A of Vancouver, 535 Hornby Street, Vancouver, B C V6C 2E8). Zabinski, M.F., Wilfley, D.E., Calfas, K.J., Winzelberg, A . J . , & Taylor, C.B. (2004). A n interactive psychoeducational intervention for women at risk of developing an eating disorder. Journal of Consulting and Clinical Psychology, 72, 914-919.  Appendix A : Information on the Welcome To M y Life Program  Documents from Appendix A have been removed due to copyright restrictions. The information removed (pages 109-115) described the Welcome To M y Life program's development and goals, as well as, a provided a brief overview of each module's topics and activities. The original materials were obtained from the following source: Young Women Christian Association of Vancouver (n.d.). Welcome To My Life facilitatior manual. (Available from the Y W C A of Vancouver, 535 Hornby Street, Vancouver, B C V6C 2E8).  Appendix B: Consent and Assent Forms  THE U N I V E R S I T Y OF BRITISH C O L U M B I A  UBC  Department of Educatknal and Counselling P^chology, and ^)ecial Education Faculty ofEducation 2125MainMaD Vancouver, B.C. V6T1Z4  An Investigation of the Effects of the Welcome To My Life Program on the SocialEmotional Functioning of Grade 7 Girls Facing the Transition to High School. Parental Consent Form  Principal Investigators:  Shelley Hymel, Ph.D., Professor, Faculty of Education, Department of ECPS School Psychology/Human Development L(  Co-Investigator:  Ms. Nikki Hearle, Masters Student, School Psychology (HHHH)  Dear Parent(s) or Guardian(s):  I am a graduate student in School Psycholgy at the University of British Columbia (UBC). For my Masters thesis, under the supervision of Dr. Shelley Hymel, I am conducting a research project at your daughter's school. We would appreciate your and your daughter's assistance with this project. The following is information about the study to help you make your decision. As you know, your daughter's school has been chosen to participate in the YWCA Welcome To My Life program. This opportunity, available to all grade 7 girls at the school, will help girls develop the relationship, time management, and problem-solving skills, as well as, selfesteem necessary for a successful transition into high school next year. Welcome To My Life will take place after school for 2.5-3 hours, once weekly for 6-8 weeks. Modules relating to issues grade 7 girls deal with will be facilitated by trained UBC TREK students. Themes of the program are fear, trust, and choice, and how they relate to the students' future transition to high school. Program development was based on concerns expressed by other grade 7 girls during focus groups. A parent meeting will be established to provide more information about the program. Purpose of the Current Research:  Dr. Hymel and I will be evaluating the effectiveness of the Welcome To My Life program and are interested in finding out the effects of the program on measures of self-esteem, positive qualities, and support systems. If the program is found to be effective in its goals, it may become more broadly available for grade 7 girls in the Vancouver area.  To evaluate Welcome To My Life, the girls who participate in the program will be evaluated, as well as, girls who do not participate in the program. It is of interest to the current research project to collect data from girls who do not participate in the program to learn more about girls during this developmental period. It will also help the researchers understand if any changes or findings result from participation in Welcome To My Life or something else that is occurring developmentally during the grade 7 period. Whether or not you choose to have your daughter participate in Welcome To My Life, it would be extremely valuable to the research project to have your daughter complete the evaluation component. Participation in the study would involve completing 2 self-report, paper and pencil measures before and after the program is facilitated. Two 45-60 minute sessions, during class time, will be used to complete the evaluation. The measures relate to self-esteem, safety, and support systems across various contexts (e.g., home, school, community, among peers). Participation in the questionnaire portion is completely voluntary. You or your daughter may chose to withdraw from the study at any point without penalization. If you choose to not have your daughter participate, will remain in the classroom and quietly work on school work. Confidentiality:  High standards of confidentiality will be maintained for all data gathered from participants of this study. A numerical code will be used for identification and no identifying information will be on the questionnaires. Information will be kept in a secure location at UBC and only the researchers will have access to the data. All computer files will be password protected. Only group level results will be discussed in the completed study and your daughter will not be mentioned by name or any other identifying information. Raw data will be destroyed following the completion of the study. Risks Involved With Participation:  There exists a slight possibility that completing the designated questionnaires will bring up issues potentially leading to negative feelings, although this is unlikely. If your daughter feels any discomfort while completing the questionnaires, she may refuse participation or withdraw from the study without penalization. At all times while the questionnaires are being filled out, a teacher and a trained school psychology graduate student will be present should a situation arise. Incentive for Returning the Consent Form:  We are interesting in hearing from all grade 7 girls. After reviewing the information in this letter, please discuss this opportunity with your daughter and indicate your choice on the attached consent form (page 4). Please send the consent form back to your daughter's school whether or not she chooses to participate. If your daughter returns page 4 of this form completed (whether or not she chooses to participate), her name will be entered in a draw for a $25 gift certificate to a local music store (e.g.. Future Shop).  If you have any further questions or concerns regarding the study, please do not hesitate to contact Dr. Shelley Hymel at The Research Subject Information Line at the UBC Office of Research Services at 604-822-8598 can be contacted with concerns regarding the rights and treatment of human research participants. Thank you for your cooperation with this research project.  Sincerely,  Shelley Hymel, PhD Professor and Principle Investigator  Nikki Hearle, BA Masters Candidate and Co-Investigator  Please complete the section below the dotted line and send the form back to your daughter's school within the next week. Keep the top section for your own records. Thank you. Consent:  You and your daughter's participation in this project is entirely voluntary and she may refuse to participate or withdraw from the study at any time without negative consequences. Your signature indicates that you have read the consent form, talked about it with your daughter, and consent to participate in this study. It also indicates that you have kept a copy of this form for your personal records. I give my consent/ I do not give my consent (please circle one) for my daughter to participate. Parent/Guardian's Name (please print): Parent/Guardian's Signature: Date: Daughter's Name (please print first & last name):  K  K  Consent:  Your and your daughter's participation in this study is entirely voluntary and you may refuse to participate or withdraw from the study at any time without negative consequences. Your signature indicates that you have read the consent form, talked about it with your daughter, and consent to participate in this study. It also indicates that you have kept a copy of this form for your personal records. I give my consent/1 do not give my consent (please circle one) for my daughter to participate. Parent/Guardian's Name (please phnt): Parent/Guardian's Signature:  Date:  Daughter's Name (please print first & last name): Please return this form to your daughter's school within the next week. Thank-you!  T H E U N I V E R S I T Y OF B R I T I S H C O L U M B I A  UBC  Department of Educational and Counsdlii^ Psychology, and Special Education Faculty ofMjcatiai 2125 Main Mall Vancouvff,B.C.VOTlZ4  An Investigation of the Effects of the Welcome To My Life Program on the SocialEmotional Functioning of Grade 7 Girls Facing the Transition to High School. Research Assent Form for Questionnaire Group Principal Investigators:  Shelley Hymel, Ph.D., Professor, Faculty of Education, Department of ECPS School Psychology/Human Development Learning And Culture  Co-Investigator:  (HHHIH)  Ms. Nikki Hearle, Masters Student, School Psychology  (HHHH)  Why are we doing this study?  The transition from elementary school to high school is one that is difficult for many students. We would like to find out if we can help grade 7 girls develop skills to help make the change easier. By participating in this research study, you will help us build a program to help other girls your age. What will happen if I participate in this research study?  To participate in this study, you do not need to be participating in the YWCA Welcome To My Life Program. We want to hear from lots of girls, some that are involved with it, some that are not. During two 45-60 minute class time sessions, you will complete a pencil and paper questionnaire. You will be asked questions about yourself, how you feel about yourself, and about different areas of your life. These are not tests, so there are not "right" or "wrong" answers, we just want to learn more about how grade 7 girls think and feel. By completing the questionnaires, you are helping us gather valuable information. What happens if I don't want to participate?  Participation in this project is completely voluntary. It's OK to decide you do not want to be involved anymore and you will not be punished for quitting. All you have to do is let your teacher or the researcher know that you do not want to participate anymore and they will find something else for you to do.  Privacy/Rules of Confidentiality:  Your name will not appear anywhere on the questionnaire, so no one will find out how you answered questions. What you write will not be shared with your parents, friends, teachers, or school. Information collected will be l<ept in a secure location at UBC and only the researchers will have access to it. ASSENT:  You have been read the reasons why this research project is happening and you have been given the chance to ask questions and discuss this with your parent/guardian and the researcher. Are you willing to participate in filling out the questionnaires? Please check one: YES I want to participate in filling out the questionnaires. NO I don't want to participate in filling out the questionnaires.  Name of Child  Date  Name of person who obtained assent  Signature  Appendix C: Self-Description Questionnaire-II (SDQ-II) and Developmental Assets Profile (DAP)  Questionnaires from Appendix C have been removed due to copyright restrictions. The information removed (pages 124-131) was measures to assess study participants' selfesteem/self-concept {Self-Description Questionnaire - II; Marsh, 1990) and perceived positive attributes {Developmental Assets Profile; Search Institute, 2004). The original materials were obtained from the following sources: Marsh, H.W. (1990a) Self-Description Questionnaire - II Manual and research monograph. San Antonio, T X : Psychological Corp. Search Institute (2004). Developmental Assets Profile Preliminary User Manual. Search Institute, Minneapolis, M N .  Appendix D: Participant Satisfaction Questionnaire  1. Would you recommend the Welcome to M y Life program to a friend?  2. Do you feel more prepared to enter high school as a result of participating in this program? Why or why no?  3. How has participating in the Welcome To M y Life program helped you (e.g., feel better about yourself, solve problems with peers better, improve your relationships with your parents, improve your body image, gain a better understanding of the demands of high school)?  Appendix E: Implementation Integrity and Treatment Acceptability Forms  Dear Participating UBC Treic Student :  We are interested in your experiences teaching tiie Welcome to l^y Life program. Ttiis information is vital to us in evaluating just how useful this program is. Please take a few minutes to complete this survey. If you have additional comments, feel fr to write them on the back of this questionnaire. We welcome your feedback and ideas. Thanks for your input.  Strongly Disagree  1. I think that social-emotional learning and skills training are critical to the transition of girls from elementary school to high school. 2. I believe in the value of all the activities in this module. 3. Upon completion of the training I received, I felt prepared to facilitate the session. 4. I believe I had the necessary skills to facilitate the activities in this module.  Strongly Agree  i  2  3  i  2  3  4  5  i  2  3  4  5  i  2  3  4  5  2  3  4  5  2  3  4  5  5. The activities in the module were engaging and enjoyed by the girls. 6. The activities were successful in achieving their stated goals/purposes (e.g., learn other girls' names, identif/ their own values and how they fit into their life, understand healthy food choices).  Neutral  i i  4  5  7. I worked with my co-facilitator preparing for this module during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  60-90 minutes  Over 90 minutes  8. I prepared for this module on my own during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  60-90 minutes  Over 90 minutes  9. How many years of previous experience have you had working with children? None  1 year  2 years  3 years  4 or more years  Please briefly describe your previous experience with children and the ages of those children.  10. What activity was the most engaging and why?  11. What activity was the least engaging and why?  IZ.What was the best thing about this module for the grade 7 girls?  13. Additional overall comments about the session.  14. What are some recommendations you could provide for future facilitators of this module?  Implementation Integrity - Implementation Integrity refers to how closely the facilitation of a module followed the program design. This section is completed by a wisdom champion or facilitator. Module 1: 411 On Girls  The following components of the module were facilitated: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14.  Oven/lew of the program was given. YES "Move Your Butt". YES "Interview Pairs", YES "Boundaries". YES "Choice, Fear, and Trust in My World." YES "My Hopes....". YES "Roles & Responsibilities." YES Module Evaluation. YES "Living the Learning". YES The choice stone was appropriately used. YES Activities were kept to with in a few minutes of the designated time frame. YES The timer was set for most activities. YES The importance of and role of adults was discussed. YES Module Evaluation. YES  NO NO NO NO NO NO NO NO NO NO NO NO NO NO  Dear Participating UBC Trek Student :  We are interested in your experiences teaching the Welcome to My Life program. This information is vital to us in evaluating just how useful this program is. Please take a few minutes to complete this survey If you have additional comments, feel fre to write them on the back of this questionnaire. We welcome your feedback and ideas. Thanks for your input  Strongly Disagree  1. I believe in the value of all the activities in this module.  Neutral  Strongly Agree  1  2  3  4  5  1  2  3  4  5  3. I believe I had the necessary skills to facilitate the activities in this module.  1  2  3  4  5  4. The activities in the module were engaging and enjoyed  1  2  3  4  5  The activities were successful in achieving their stated goals/purposes (e.g., learn other girls' names, identify their own values and how they fit into their life, understand healthy food choices).  1  2  3  4  5  2. Upon completion of the training I received, I felt prepared to facilitate the session.  by the girls.  5.  6. What activity was the most engaging and why?  7. What activity was the least engaging and why?  8. What was the best thing about this module for the grade 7 girls?  9. Additional overall comments about the session.  Implementation Integrity - Implementation Integrity refers to how closely the facilitation of a module followed the program design.  Module 2: Girls Connected The following components of the module were facilitated (0=Not at all/None of the time; 3=Partially completed, some of the component/some of the time; 5=Completely facilitated, all of the components were facilitated/All of the time): Review of Module 1. "Web of Support." "Relationship Bracelets." "Reflections." Module Evaluation. Living the Learning. The importance of and role of adults was discussed. 8. The choice stone was appropriately used. 9. Activities were kept to with in a few minutes of the designated time frame. 1. 2. 3. 4. 5. 6. 7.  YES YES YES YES YES YES  NO NO NO NO NO NO  YES  NO  YES  NO  YES  NO  School: Dear Participating UBC Treic Student :  We are interested in your experiences teactiing tfie Welcome to My Life program. This information is vital to us in evaluating just how useful this program is. Please take a few minutes to complete this survey. If you have additional comments, feel fr to write them on the back of this questionnaire. We welcome your feedback and ideas. Thanks for your input  Strongly Disagree  1. I believe in the value of all the activities in this module.  Neutral  strongly Agree  1  2. Upon completion of the training I received, I felt prepared to facilitate the session.  1  2  3  4  3. I believe I had the necessary skills to facilitate the activities in this module.  1  2  3  4  4. The activities in the module were engaging and enjoyed by the girls.  1  2  3  4  5. The activities were successful in achieving their stated goals/purposes (e.g., learn other girls' names, identify their own values and how they fit into their life, understand healthy food choices).  1  2  3  4  6. I worked with my co-facilitator preparing for this module during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  60-90 minutes  Over 90 minutes  7. I prepared for this module on my own during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  8. What activity was the most engaging and why?  9. What activity was the least engaging and why?  60-90 minutes  Over 90 minutes  12. What are some recommendations you could provide for future facilitators of this module?  Implementation Integrity - Implementation Integrity refers to how closely the facilitation of a module followed the program design. Module 3: What I Believe The following components of the module were facilitated (0=Not at all/None of the time; 3=Partially completed, some of the component/some of the time; 5=Completely facilitated, all of the components were facilitated/All of the time): 1. 2. 3. 4. 5.  6. 7. 8. 9. 10.  Review of Module 2. "Let's Talk About Values." "What's Important to You?" "When Values Conflict Module Evaluation. Living the Learning. The importance of and role of adults was discussed during the module. The choice stone was appropriately used. Activities were kept to with in a few minutes of the designated time frame. Reflection.  YES YES YES YES YES YES  NO NO NO NO NO NO  YES  NO  YES  NO  YES YES  NO NO  School: Dear Participating UBC Treic Student :  We are interested in your experiences teaching the Welcome to My Life program. This information is vital to us in evaluating just how useful this program is. Please take a few minutes to complete this survey. If you have additional comments, feel fre to write them on the back of this questionnaire. We welcome your feedback and ideas. Thanks for your input  Neutral  Strongly Disagree  1. I believe in the value of all the activities in this module. 2. Upon completion of the training I received, I felt prepared to facilitate the session.  Strongly Agree  1  2  3  4  1  2  3  4  3. I believe I had the necessary skills to facilitate the activities in this module.  1  2  3  4  4. The activities in the module were engaging and enjoyed by the girls.  1  2  3  4  5. The activities were successful in achieving their stated goals/purposes (e.g., learn other girls' names, identify their own values and how they fit into their life, understand healthy food choices).  1  2  3  4  6. I worked with my co-facilitator preparing for this module during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  60-90 minutes  Over 90 minutes  7. I prepared for this module on my own during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  8. What activity was the most engaging and why?  9. What activity was the least engaging and why?  60-90 minutes  Over 90 minutes  12. What are some recommendations you could provide for future facilitators of this module?  Implementation Integrity - Implementation Integrity refers to how closely the facilitation of a module followed the program design. Module 4: The Sparkle In You The following components of the module were facilitated (0=Not at all/None of the time; 3=Partially completed, some of the component/some of the time; 5=Completely facilitated, all of the components were facilitated/All of the time): 1. 2. 3. 4. 5. 6. 7. 8.  Review of Module 3. "Eating Smart" "Let's Go Shopping." Reflection. Yoga. Module Evaluation. Living the Learning. The importance of and role of adults was discussed during the module. 9. The choice stone was appropriately used. 10. Activities were kept to with in a few minutes of the designated time frame.  YES YES YES YES YES YES YES  NO NO NO NO NO NO NO  YES  NO  YES  NO  YES  NO  Dear Participating UBC Treic Student :  We are interested in your experiences teaching the Welcome to My Life program. This information is vital to us in evaluating just how useful this program is. Please take a few minutes to complete this survey. If you have additional comments, feel fre to write them on the back of this questionnaire. We welcome your feedback and ideas. Thanks for your input  Strongly Disagree  1. I believe in the value of all the activities in this module.  Neutral  Strongly Agree  i  2  3  4  5  i  2  3  4  5  3. I believe I had the necessary skills to facilitate the activities in this module.  i  2  3  4  5  4. The activities in the module were engaging and enjoyed by the girls.  i  2  3  4  5  5. The activities were successful in achieving their stated goals/purposes (e.g., learn other girls' names, identify their own values and how they fit into their life, understand healthy food choices).  i  2  3  4  5  2. Upon completion of the training I received, I felt prepared to facilitate the session.  6. I worked with my co-facilitator preparing for this module during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  60-90 minutes  Over 90 minutes  7. I prepared for this module on my own during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  8. What activity was the most engaging and why?  9. What activity was the least engaging and why?  60-90 minutes  Over 90 minutes  12. What are some recommendations you could provide for future facilitators of this module?  Implementation Integrity - Implementation Integrity refers to how closely the facilitation of a module followed the program design. Module 5: Take A Look In The Mirror The following components of the module were facilitated (0=Not at all/None of the time; 3=Partially completed, some of the component/some of the time; 5=Completely facilitated, all of the components were facilitated/All of the time): 1. Review of Module 4. 2. "Media Deception." 3. "Sense of Belonging." 4. Reflection. 5. "Love Yourself." 6. "Positive Affirmations." 7. Module Evaluation. 8. Living The Learning 9. The importance of and role of adults was discussed. 10. The choice stone was appropriately used. 11. Activities were kept to with in a few minutes of the designated time frame.  YES YES YES YES YES YES YES YES  NO NO NO NO NO NO NO NO  YES  NO  YES  NO  YES  NO  Dear Participating UBC Treic Student :  We are interested in your experiences teactiing the Welcome to My Life program. This information is vital to us in evaluating Just how useful this program is. Please take a few minutes to complete this survey. If you have additional comments, feel fre to write them on the back of this questionnaire. We welcome your feedback and ideas Thanks for your input  Strongly Disagree  1. I believe in the value of all the activities in this module.  Neutral  Strongly Agree  1  2  3  4  5  1  2  3  4  5  3. I believe I had the necessary skills to facilitate the activities in this module.  1  2  3  4  5  4. The activities in the module were engaging and enjoyed by the girls.  1  2  3  4  5  5. The activities were successful in achieving their stated goals/purposes (e.g., learn other girls' names, identify their own values and how they fit into their life, understand healthy food choices).  1  2  3  4  5  2. Upon completion of the training I received, I felt prepared to facilitate the session.  6. I worked with my co-facilitator preparing for this module during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  60-90 minutes  Over 90 minutes  7. I prepared for this module on my own during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  60-90 minutes  Over 90 minutes  8. Was the timer set for activities? Not at all  For a few activities  For most activities  9. What activity was the most engaging and why?  For all activities  12. Additional overall comments about the session.  Implementation Integrity - Implementation Integrity refers to how closely the facilitation of a module followed the program design. This section is completed by a wisdom champion or facilitator.  Module 6: Laugh Out Loud The following components of the module were facilitated: Review of Module 5. "Telephone Tag." "Friendship Box." "Putting It All Together." Module Evaluation. Living the Learning. The importance of and role of adults was discussed during the module. 8. The choice stone was appropriately used. 9. Activities were kept to with in a few minutes of the designated time frame. 10. The timer was set for most activities. 1. 2. 3. 4. 5. 6. 7.  YES YES YES YES YES YES  NO NO NO NO NO NO  YES  NO  YES  NO  YES YES  NO NO  School: Dear Participating UBC Treic Student :  We are interested in your experiences teaching the Welcome to My Life program. This information is vital to us in evaluating just how useful this program is Please take a few minutes to complete this survey. If you have additional comments, feel fre to write them on the back of this questionnaire. We welcome your feedback and ideas. Thanks for your input Strongly Disagree  1. I believe in the value of all the activities in this module. 2. Upon completion of the training I received, I felt prepared to facilitate the session.  Neutral  Strongly Agree  1  2  3  4  1  2  3  4  3. I believe I had the necessar/ skills to facilitate the activities in this module.  1  2  3  4  4. The activities in the module were engaging and enjoyed by the giris.  1  2  3  4  5. The activities were successful in achieving their stated goals/purposes (e.g., learn other giris' names, identify their own values and how they fit into their life, understand healthy food choices).  1  2  3  4  6. I worked with my co-facilitator preparing for this module during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  60-90 minutes  Over 90 minutes  7. I prepared for this module on my own during the past week for: 0-15 minutes  15-30 minutes  30-60 minutes  8. What activity was the most engaging and why?  9. What activity was the least engaging and why?  60-90 minutes  Over 90 minutes  12. What are some recommendations you could provide for future facilitators of this module?  Implementation Integrity - Implementation Integrity refers to how closely the facilitation of a module followed the program design.  Module 7: So Much To Do, So Little Time The following components of the module were facilitated (0=Not at all/None of the time; 3=Partially completed, some of the component/some of the time; 5=Completely facilitated, all of the components were facilitated/All of the time): 1. 2. 3. 4. 5. 6. 7.  Review of Module 6. "A Balancing Act." "Welcome to my Life game." "Putting It All Together." Reflection. "Living the Learning. The importance of and role of adults was discussed during the module. 8. The choice stone was appropriately used. 9. Activities were kept to with in a few minutes of the designated time frame. 10. The timer was set for most activities. 11. Module Evaluation.  YES YES YES YES YES YES  NO NO NO NO NO NO  YES  NO  YES  NO  YES YES YES  NO NO NO  Appendix F: Ethics Approval  The University of British Columbia Office of Research Services and Administration  152  Behavioural Research Ethics Board  Certificate of Approval PRINCIPAL INVESTIGATOR  DEPARTMENT  Hymel, S.  Educ & Couns Psych & Spec Educ  NUMBER  B06-0631  INSTITUTIONfS^ WHERE RESEARCH WJLI BF CAR RIFD D U T  U B C Campus , Vancouver School Board, CO-INVESTIGATORS:  Hearle, Nikki Jade, Psychology SPONSORING AGENCIES  Unfunded Research TITUE ^•  ' A n Evaluationof the "Welcome to M y Life" Program APPROVAL DATE  TERM (YEARS)  1  DOCUMENTS INCLUDED IN THIS APPROVAL:  July 23, 2006, Consent form / Assent form / Questionnaires  CERTIFICATION:  The application for ethical review of the above-named project has been reviewed and the procedures were found to be acceptable on ethical grounds for research involving human subjects.  Approved on behm^oj me Behavioural Research Ethics Board by one of the following: Dr. Peter Suedfeld, Chair, Dr. Jim Rupert, Associate Chair Dr. Arminee Kazanjian, Associate Chair Dr. M . Judith Lynam, Associate Chair This Certificate of Approval is valid for the above term provided there is no change in the experimental procedures  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0054537/manifest

Comment

Related Items